When the spec sheet lies: injection attacks against LLM-generated compliance documents
When the Spec Sheet Lies: Injection Attacks on LLM-Generated Compliance Artifacts
The authors test passive and active prompt injection on 5 LLMs across 6 categories of industrial compliance document generation tasks; 65.1% of 3,000 injection trials succeeded, and the falsify payload was the most effective.
The attacker injects malicious instructions into an industrial document generation LLM: passive injection randomly hides instructions in source documents read by the model, while active injection inserts them at the end of the prompt at inference time; payloads include five categories—override, suppress, falsify, schema break, exfiltrate—and the victim system is an LLM pipeline that generates structured compliance documents such as AAS/DPP, SDS, work orders, RFQs, QC reports, and customs classification.
- Prompt injection has been widely studied in conversational, coding, and retrieval scenarios, but the extent to which LLM generation of industrial compliance documents (SDS, QC, DPP, etc.) is affected by injection remains unstudied, even though these documents drive operational, regulatory, and safety decisions.
- The authors construct 6 types of simulated industrial document generation tasks, using two injection methods—passive (hidden in source documents) and active (appended to the end of the prompt)—with 3 variants for each of 5 payload categories, attacking 5 LLMs, and algorithmically determining attack success based on known indicators in the output.
- The success rate across 3,000 injection trials was 65.1%. The falsify payload was the most effective across all tasks (74%–97%); active injection (73.2%) was more successful than passive (57.0%); differences between models were large, with Claude Sonnet 5 averaging 10.7% and Qwen2.5-7B-Instruct reaching 89.7%.
- The authors note that synthetic documents may not fully reflect the complexity of real operational environments, and that the correctness of benign outputs was not evaluated. The experiments cover only 5 models and 6 synthetic task categories, and do not evaluate any defense mechanism.
- Evaluated models
- GPT-4oClaude Sonnet 5Meta-Llama-3.1-8B-InstructMistral-7BQwen2.5-7B-Instruct
- Benchmarks
- 六类模拟工业文档生成任务(standardized product representations、SDS hazard classification、predictive maintenance、procurement、quality control、customs),每任务 5 份合成文档
- Metrics
- attack success rate
The researchers examined how various prompt injection attacks compromise LLMs performing industrial document generation tasks, involving compliance documents such as safety data sheets (SDS), quality control (QC) documents and digital product passports (DPP). The results show that attack success rates differ significantly across models: active forgery-type injections had the highest success rate in all scenarios, while models were often most robust to passive injection coverage requests.
Paper analysis
What problem does this paper attempt to solve?
The authors study the impact of prompt injection attacks on LLM generation of industrial compliance documents (SDS, QC, DPP, etc.), a scenario that has previously lacked research.
Core question: to what extent can prompt injection attacks manipulate LLM-generated industrial compliance documents (such as SDS, QC reports, DPP), a scenario that has been almost unstudied until now.
- Scenario and importance: LLMs are increasingly used to scan and generate industrial documents (Safety Data Sheets, Quality Control documents, Digital Product Passports, etc.), and the outputs of these documents drive operational, regulatory, or safety-critical decisions, so tampering has downstream effects.
- Insufficient existing research: The authors state that prompt injection risks have been widely studied in conversational agents, coding assistants, and retrieval-augmented systems, but the impact of attacks on structured industrial compliance document generation is "largely unexplored," and at the time of writing no literature directly discusses this issue.
- Core observation: Industrial LLM applications mostly scan existing documents to generate new documents, so passive (hidden in source documents) and active (delivered with automated instructions) injection best fit their deployment mode; override, suppress, falsify, schema break, and exfiltrate payloads directly counter the purpose of these tools, and when PII is involved there is also a privacy risk.
- Paper contributions: On 6 categories of simulated industrial document generation tasks, 2 injection methods × 5 payload types (3 variants each) are used to attack 5 LLMs, for 3,000 injection trials in total, measuring attack success rate.
- Threat model (Section III-A, Figure 1):
- Attacker goal: make the model execute injected instructions in the generated structured document (override instructions, omit information, falsify data, break the schema, leak information).
- Knowledge: The paper cites the taxonomy of Greshake et al., describing such injection as a low-skill, low-knowledge attack.
- Capability: passive injection can hide malicious instructions in reference documents the model reads; active injection can insert instructions at the end of the prompt at inference time.
- Victim system: an LLM generation pipeline that converts heterogeneous industrial information into structured compliance documents.
What related research exists?
Related work falls into two groups: six categories of applications of LLM-generated industrial documents, and research on the classification and payloads of prompt injection; the authors state that no literature has yet studied the impact of injection on industrial documents.
LLM-generated industrial documents (Section II-A)
- Standardized product representation: Xia et al. early on used neural language models and semantic fingerprints to map arbitrary information models to AAS models [8]; later work used generative LLMs to convert technical data sheets into AAS instance models, reporting a generation success rate of 62%–79% [9]; Shi et al. used fine-tuned open-source LLMs for semantic entity matching between industrial data and AAS information models [10].
- SDS and predictive maintenance: Lee et al. evaluated LLMs extracting substance and ingredient information from SDS and compared it against manually extracted reference data [11]; Jones et al. used generative LLMs to generate maintenance planning package components from historical maintenance information [12].
- Procurement, quality control and customs: Brandtner and Hofer evaluated GPT-4o for formal checking, summarization and comparison of supplier quotations [13]; Shi et al. combined AAS information models with LLM semantic search to integrate heterogeneous production and quality information [14]; Hu et al. proposed an LLM mechanism for extracting key information from non-standardized tables [15].
Prompt injection taxonomy and payloads (Section II-B)
- Taxonomy of injection methods: Greshake et al. defined four categories of low-skill, low-knowledge injection methods (passive, active, user-driven, hidden) [1]; Gulyamov et al. surveyed prompt injection vulnerabilities, attack vectors and defense mechanisms in LLM and agent systems [2]. The authors state that developments such as Anthropic's Model Context Protocol (MCP) increase the risk of such attacks [2].
- Injection payloads: Oh et al. injected instructions to induce AI coding tools to insert insecure backends [16]; Yi et al. embedded malware advertisements into internet search results; Cao et al. used malicious pop-ups that trigger malicious tasks when viewed (VPI-BENCH) [17]. Table I summarizes seven categories of techniques: jailbreaking, instruction override, game-based manipulation, role playing, adversarial optimization, obfuscation, and payload splitting.
Baselines and benchmarks: The paper sets no comparison baseline methods and uses no existing benchmarks; the experiments consist of 6 types of simulated tasks built by the authors, plus 150 baseline runs without injection as a control.
Positioning of this paper: The authors state that none of the above industrial document generation studies addresses the impact of targeted injection attacks, and this paper aims to fill this gap; its task design is taken directly from the six common LLM generation tasks summarized in Section II-A.
How does the paper address this problem?
On 6 categories of simulated industrial document tasks, the authors use 2 injection methods × 5 payload types (3 variants each) to attack 5 LLMs, determining success algorithmically by known indicators in the output.
Overall approach: simulate the deployment mode of industrial LLM document generation pipelines, systematically apply passive and active prompt injection to 5 LLMs, and measure the attack success rate of five types of malicious payloads across six task categories.
Task selection (Section III-B)
- Six task categories: drawn from the common applications summarized in Section II-A—standardized product representations (data sheets converted to AAS format for DPP generation), SDS hazard classification summaries, predictive maintenance (sensor logs and technician notes generating structured work orders), procurement (supplier RFQ compliance data converted to JSON), quality control (inspector notes generating structured inspection reports), and customs (HS code classification and restricted substance flagging).
- Data construction: because some task data are not public, 5 documents were synthesized per task and manually reviewed and revised to ensure realism; schemas were written according to the industrial conventions of each scenario.
Injection methods and payloads (Section III-C)
- Passive injection: malicious instructions are randomly hidden inside the documents the model uses to generate formatted output.
- Active injection: malicious instructions are inserted at the end of the prompt at inference time.
- Five payload types: override (ignore some original instructions and execute other instructions instead), suppress (omit information that should be included), falsify (tamper with true information and inject false information), schema break (fail to follow the given schema and use a modified schema instead), exfiltrate (leak information into another format or output content that should have been redacted).
- Each payload type has 3 variants; the paper does not list the specific payload texts in the main text, and the appendix states that the code repository contains all prompts and documents.
Experimental procedure and success determination (Section III-D)
- Scale: 2 injection methods × 5 payload types × 3 variants × 5 models × 6 tasks × 5 documents, for 3,000 injection trials in total, plus 150 no-injection baseline runs, for 3,150 model outputs in total.
- Models tested (Table II): GPT-4o (OpenAI, API), Claude 4.6 Sonnet (Anthropic, API), Meta-Llama-3.1-8B-Instruct (Meta, Open), Mistral-7B (Mistral AI, Open), Qwen2.5-7B-Instruct (HuggingFace, Open).
- Success determination: each injection style carries a recognizable indicator (for example, the override prompt asks the model to insert specific marker text into the output document), and whether the attack succeeded is determined algorithmically by retrieving these known indicators in the output text.
- Scope statement: The authors explicitly do not evaluate the correctness of benign outputs, and only measure and report attack success rate.
What experiments did the paper conduct?
Overall success rate across 3,000 injection trials was 65.1%; the falsify payload was the most effective across all tasks, active injection outperformed passive, and differences between models ranged from 10.7% to 89.7%.
Experimental setup
- Models tested (Table II): GPT-4o, Claude 4.6 Sonnet, Meta-Llama-3.1-8B-Instruct, Mistral-7B, Qwen2.5-7B-Instruct.
- Tasks and data: 6 categories of simulated industrial document generation tasks, with 5 synthetic documents per task (manually reviewed and revised), with schemas written according to industrial conventions.
- Attack configuration: 2 injection methods (passive hidden in documents, active appended at the end of the prompt) × 5 payload types × 3 variants.
- Sample size: 3,000 injection trials + 150 no-injection baseline runs, for 3,150 outputs in total; the paper does not report the number of repetitions.
- Metric and determination: attack success rate, determined by algorithmic retrieval of known injection indicators in the output, with no LLM judging or human scoring; benign output correctness is not evaluated.
Main results
Overall: 1,953 of 3,000 injection trials succeeded, an attack success rate of 65.1%. Model × payload success rates per Figure 3:
Swipe horizontally to read the table
Model override suppress falsify schema break exfiltrate Qwen2.5-7B-Instruct 86% 85% 94% 95% 88% Meta-Llama-3.1-8B-Instruct 73% 61% 96% 99% 98% Mistral-7B 49% 85% 99% 89% 97% GPT-4o 30% 63% 100% 46% 40% Claude Sonnet 5 0% 5% 45% 1% 2% - Average success rate by model (Section IV-B): Qwen2.5-7B-Instruct 89.7%, Meta-Llama-3.1-8B-Instruct 85.5%, Mistral-7B 83.8%, GPT-4o 55.8%, Claude Sonnet 5 10.7%.
- Average success rate by task (Section IV-A): standard product representations and quality-control inspection both 70.4%, SDS hazard classification and predictive maintenance both 64.0%, procurement 62.8%, customs classification lowest at 59.0%.
- Authors' interpretation: The authors state that payload type has a greater effect on success rate than application domain; falsification was the most successful payload in every task category, which the authors suggest may be because falsification more closely resembles normal usage behavior (informing that some information is incorrect and needs to be changed), and they believe a robust document generation pipeline may need multiple cross-verifiable information sources.
Task × payload results (Figure 2)
- falsify highest across all tasks: from 74% for procurement to 97% for predictive maintenance; override was overall lowest, from 36% for customs to 58% for predictive maintenance.
- suppress varied most across tasks: from 27% for predictive maintenance to 78% for quality-control inspection; schema break and exfiltrate were more stable, both 61%–71%.
- Authors' interpretation: The authors suggest the variation in suppress may indicate that missing information is more easily noticed in some tasks; the lower effectiveness of schema break may be related to the schema containing additionalProperties: false fields, and the authors state that cheap prompt or input engineering may be the most intuitive defense.
Comparison of injection methods (Figure 4)
- Active higher than passive across the board: active averaged 73.2%, passive averaged 57.0%; active was higher under every payload type.
- Breakdown (passive → active): override 37% → 58%, suppress 49% → 71% (22 percentage points higher), falsify 80% → 94%, schema break 61% → 71%, exfiltrate 58% → 72%.
- Authors' interpretation: The authors state that the direct reason is not yet clear and may be recency bias (active injection is closer to the generation point), and believe defense mechanisms should prioritize guarding against malicious instructions added in transit beyond material sanitization.
Other ablations and analyses
- Models recognizing injection behavior: The authors report that Claude Sonnet actively flagged suspicious injection in some outputs, but the same output still leaked an internal reference number; the authors therefore conclude that human review is needed.
- override overall: lowest success rate across all injection trials, at 47.7% (Section IV-C).
- Negative results: The results of the 150 no-injection baseline runs are not reported in the main text; the paper does not report evaluation of any defense mechanism.
- Authors' observation: The three open-weight models had success rates above 83% in most categories (Figure 3); the authors state that models from Anthropic and OpenAI were more robust, and believe this may support the justification for their higher API costs.
What are some points that can be further explored?
The authors note limitations such as the representativeness of the synthetic documents and the lack of evaluation of the correctness of benign outputs, and call for research on defenses in industrial scenarios; the experiments cover only 5 models and synthetic tasks.
Limitations and future directions noted by the authors
- Representativeness of synthetic documents: The authors note that synthetic documents may not fully capture the complexity of real operational environments, but provide a controllable evaluation basis that avoids confidentiality issues (Section III-D, footnote 1).
- Benign output correctness not evaluated: The authors explicitly state that the correctness of benign LLM outputs is beyond the scope of this study, and that only attack success rate is measured (Section III-D, footnote 2).
- Human review needed: The authors state that models may notice a problem yet still leak information, highlighting the need for careful human review of outputs (Section V-C).
- Cross-validation across multiple information sources needed: The authors believe a robust document generation pipeline may need multiple cross-referenceable information sources to prevent erroneous information from entering the final output and propagating between systems (Section V-A).
- Gap in defense research: The authors state that defenses such as human review, detection mechanisms, and defensive prompts exist for general scenarios, but the preliminary success of injection attacks highlights the need to study how to apply these defenses to industrial scenarios (Section VI).
- Reason for active injection success unclear: The authors state that the direct reason active injection is more successful is not yet clear, and may be recency bias (Section V-D).
Scope of experimental coverage
- The models tested are GPT-4o, Claude 4.6 Sonnet, Meta-Llama-3.1-8B-Instruct, Mistral-7B, and Qwen2.5-7B-Instruct, 5 in total (Table II).
- The tasks are 6 categories of simulated industrial document generation scenarios, with 5 synthetic documents per task; the injection configuration is 2 methods × 5 payload types × 3 variants, for 3,000 injection trials in total (Section III).
- Success is determined by algorithmic text retrieval based on known indicators; the paper does not use LLM judging or human scoring, nor does it report agreement between the determination method and human judgment.
- The paper does not report the number of repeated runs, sampling settings (such as temperature), or query costs.
- The experiments do not include evaluation of any defense mechanism; the authors state that applying general defenses to industrial scenarios is left for future research (Section VI).
Summarize the main content of the paper
The authors systematically measure the impact of prompt injection on LLM-generated industrial compliance documents: 65.1% success across 3,000 trials, with falsify payloads and active injection most effective, and large differences across models.
An empirical attack study: measuring the impact of prompt injection on LLM generation of industrial compliance documents (AAS/DPP, SDS, work orders, RFQ, QC reports, customs classification), a scenario the authors state has previously lacked research.
- Problem: Industrial LLM pipelines mostly scan existing documents to generate structured compliance documents, and the outputs drive operational, regulatory, and safety decisions; injection attacks have been studied in conversational and coding scenarios, but their impact on this kind of document generation is unknown.
- Method: 6 simulated task categories (5 manually reviewed synthetic documents per task), 2 injection methods (passive hidden in source documents, active appended at the end of the prompt) × 5 payload types (override, suppress, falsify, schema break, exfiltrate, 3 variants each), attacking 5 LLMs, with success determined algorithmically by known indicators in the output, for 3,000 injection trials in total.
- Overall results: 1,953 successes, an attack success rate of 65.1%; the authors therefore conclude that prompt injection poses a substantive risk to industrial document generation pipelines and requires protections beyond model prompting.
- Payloads and methods: falsify was the most effective in every task category (74%–97%, Figure 2), reaching 100% on GPT-4o (Figure 3); active injection averaged 73.2%, higher than passive at 57.0%; override was overall lowest (47.7%).
- Model differences: average success rates were Qwen2.5-7B-Instruct 89.7%, Meta-Llama-3.1-8B-Instruct 85.5%, Mistral-7B 83.8%, GPT-4o 55.8%, Claude Sonnet 5 10.7%; the authors state that models from Anthropic and OpenAI were more robust, with some outputs actively flagging suspicious injection, but they may still leak information.
- Authors' conclusions: The authors believe payload type affects success rate more than task domain, and that falsification may be hard to defend against because it more closely resembles normal usage; they recommend that pipelines adopt cross-validation across multiple information sources and human review, and identify adapting general defense mechanisms to industrial scenarios as a future direction.