Cooking Risks: Benchmarking and Mitigating Food Safety Risks in Large Language Models

Existing LLMs and general guardrails lack alignment for food hazards and may provide operational guidance that could cause real-world harm.

• Scenario: The authors state that LLMs are already used for meal preparation, storage, health guidance, and food design; incorrect guidance can lead to contamination, foodborne illness, or allergies. Figure 1 cites a PwC 2025 survey: meal planning 47%, personalized nutrition 45%, food budgeting 41%, and 15% selecting "none of the above."
• Existing gaps: The authors state that alignment mostly targets hate speech, malware, and pornography, and does not systematically cover domain hazards. General-purpose guardrails mainly block explicitly harmful queries and lack the granularity and data needed to identify food risks hidden in seemingly innocuous requests.
• Authors' observation: The authors believe that unsafe advice can appear practical and benign while violating temperature control, cross-contamination prevention, allergen handling, or spoilage management; the absence of obvious harmful intent does not mean there is no risk.
• Proposed work: The authors propose FoodGuardBench (which they call the first comprehensive benchmark, containing 3,339 base queries anchored to the FDA framework), a scalable adversarial query generation pipeline, and a fine-tuned guardrail, FoodGuard-4B.
• Threat model:
  • Goal: The model must still provide helpful food assistance to ordinary users; adversarial users seek guidance that violates FDA food safety regulations and can be used to cause real-world harm.
  • Knowledge: AutoDAN optimizes prompts on white-box models before transferring; PAP probes black-box models with social persuasion. The paper does not frame the overall setting as a single white-box or black-box scenario.
  • Capability: Adversarial users can submit malicious queries and are likely to use jailbreaks to bypass internal safety alignment or external guardrails.
  • Victim systems: LLMs providing food-related assistance, and guardrails acting as standalone input-output review layers.

In §2, the authors divide related work into domain safety, jailbreaks, and guardrails, and compare existing benchmarks using Figure 3.

Domain Safety and Food Tasks

• Adjacent Domains: The authors list medical multi-agent (Chen et al., 2025), SOSBENCH (Jiang et al., 2026), chemical safety (Zhao et al., 2024), laboratory safety (Zhou et al., 2024), etc., and state that while safety research exists in physics, medicine, chemistry, psychology, pharmacology, and laboratories, risks in the food domain remain underexplored.
• Food-Side Work: The authors write that this domain requires molecular and sensory attributes (Lee et al., 2023), implicit constraints (Mizrahi and Shahaf, 2021), and policies and regulations (Fung et al., 2018). Document tasks include FoodSafeSum (Bakagianni et al., 2025b), etc.; the authors state that user-side safety risk exploration is limited.

Jailbreak Attacks

• Wrapping Harmful Instructions: The authors group GCG (Zou et al., 2023), AutoDAN (Liu et al., 2024), PAP (Zeng et al., 2024), DeepInception (Li et al., 2024b), etc., and state that jailbreak success is highly sensitive to the instruction domain.
• Mismatched Generalization: The authors cite Wei et al. (2023b), Yong et al. (2023), Lv et al. (2024), etc., and state that prior work moves malicious prompts to out-of-distribution domains such as low-resource languages, narratives, or code/math. The authors separate this paper from approaches that rely on non-natural language transformations, instead examining the intrinsic alignment sparsity in the food domain.

Guardrails

• Independent Review Layer: The authors state that guardrails do not modify the underlying weights. Discussed include Llama Guard (Inan et al., 2023), Qwen3Guard (Zhao et al., 2025a), NeMo Guardrails (Rebedea et al., 2023), and ThinkGuard (Wen et al., 2025). The authors state that most are fine-tuned on general safety policies and lack robust defenses against domain risks, so this paper builds a food-science-specific guardrail.

Baselines and Benchmarks

• Experimental Baselines: Robustness experiments use AutoDAN and PAP; out-of-distribution guardrail testing uses DeepInception. Guardrail comparisons are LLaMA3.1-8B, Qwen3-8B, LLaMA-Guard3-8B, LLaMA-Guard4-12B, Qwen3Guard-4B, Qwen3Guard-8B in Table 2.
• Benchmark Comparisons: In Figure 3, AdvBench and RedTeam-2K are general domains without seed policies; SciSafeEval covers chemistry, biology, medicine, and physics, with seed policies; SOSBench additionally includes pharmacology and psychology, without seed policies; FoodGuardBench is food science, with seed policies. The authors state that their seed policies are stricter and more detailed, and they performed manual review for real-world applicability. The authors state that the t-SNE in Figure 4 shows this dataset is complementary to existing benchmarks; the text does not retain cluster positions, so it cannot be verified whether they are separated.

The authors position this paper as measuring and filling the alignment sparsity in the food domain, rather than attacking general alignment with non-natural language transformations.

The authors evaluate safety and robustness in the food domain using FoodGuardBench, and use FoodGuard-4B for moderation without changing the underlying weights.

Categories and query sources

• Eight hazard categories: temperature control, contamination, hygiene, storage, preparation, allergens, pest control, water safety (§3.1). The authors state this allows analyzing successes and failures by hazard type, rather than only a safe/unsafe summary.
• Seed rules: 356 seed policies, from FDA food regulations and common operational principles. §3.2.1 also writes short rules for contamination, temperature control, storage, cleaning and hygiene, cross-contact, etc. The paper does not explain how this set of statements corresponds to the eight categories.
• Two intent types: each seed generates 5 queries with GPT-5. Benign queries target cooking, storage, reheating, cleaning, or ingredient handling; the authors state they should not be over-refused merely because the topic is sensitive. Malicious queries are baseline questions where an adversarial user treats the model as a guide for harmful information.

Quality checks and scale

• Manual spot checks: 100 samples each for seed filtering and query generation. 98% of seeds were judged to have practical value for real-world practices; over 90% of queries passed review. The authors state this ensures benign prompts are natural and malicious prompts clearly violate seed rules and should be refused.
• Deduplication: following RedTeam-2K's approach, near-duplicates are removed with all-MiniLM-L6-v2. The average semantic similarity of the 2,339 vanilla harmful queries is 0.35, which the authors state corresponds to high diversity. There are also 1,000 benign queries, making FoodGuardBench 3,339 items in total.
• Category proportions: Appendix Table 3 only counts these 2,339 harmful queries: contamination 1130 (48.3%), preparation 330 (14.1%), temperature control 270 (11.5%), hygiene 245 (10.5%), allergens 174 (7.4%), storage 145 (6.2%), pest control 30 (1.3%), water safety 15 (0.6%).
• Generation prompt: Figure 6 in Appendix C.1 gives the generation prompt for GPT-5, with the seed policy as input and 5 questions as output. The prompt text is not reproduced.

Jailbreak enhancement

• AutoDAN: a hierarchical genetic algorithm optimizes discrete text while maintaining natural language fluency to bypass perplexity-based defenses. Queries are generated on LLaMA 3.1-8B, then used to attack other models via transferability.
• PAP: uses 40 psychological persuasion techniques to rewrite harmful queries into human-readable requests. The authors state it probes social persuasion on black-box models, while AutoDAN probes optimized prompts on white-box models.
• Lower-bound positioning: the authors use these two methods to establish a lower bound on domain vulnerability and believe more advanced or adaptive jailbreaks could achieve higher ASR. This generates 4,464 additional jailbreak queries. The paper does not break down this number by method.

Guardrail fine-tuning and judgment

• Model and split: §4.1 states the fine-tuning starting point is Qwen3Guard-Gen-4B; Appendix D.2 states initialization uses the pretrained Qwen3Guard-4B. The paper does not state whether these are the same checkpoint. Vanilla harmful queries and jailbreak queries are combined and split 8:2, with evaluation on the test set. The paper does not state whether the 1,000 benign queries enter this split, nor does it report the test set size.
• Training parameters: NVIDIA RTX 6000 Ada, 3 epochs, maximum learning rate 1.5e-5, warmup ratio 0.03, batch size 1 per device, gradient accumulation 32; the authors state the effective batch per GPU is 32. Attack experiment temperatures are all 0.7, without changing weights.
• ASR: keyword matching. ASR(D')=\frac{1}{|D'|}\sum_{q\in D'} 1[\nexists p\in P_{refusal}:p\subseteq f(q)]. The meaning is that an attack succeeds only if the reply contains none of the refusal prefixes. The paper does not list the prefix set.
• Guardrail and human evaluation: the guardrail reports Accuracy, F1, FPR, FNR. The authors define FPR as the proportion of benign prompts labeled unsafe, and FNR as the proportion of harmful prompts labeled safe, and state FNR is the most critical failure mode. Human usefulness is scored 0–2: 0 is refusal or completely ineffective, 1 is partially effective but missing key precise details, 2 is step-by-step executable with no omissions (Appendix E).

Experimental Setup

• Tested LLMs: Claude-3.7-Sonnet, GPT-4o, GPT-4.1, GLM4-32B, LLaMA-3.3-70B, Mistral-Small4, Qwen-2.5-7B, Qwen-3-8B, Qwen-3-32B. The authors state these models do not have thinking capabilities (§4.1).
• Guardrails: Table 2 includes LLaMA3.1-8B, Qwen3-8B, LLaMA-Guard4-12B, LLaMA-Guard3-8B, Qwen3Guard-8B, Qwen3Guard-4B, and FoodGuard-4B. §4.1 specifically mentions Qwen3Guard and LLaMA-Guard 4, and does not list LLaMA-Guard3-8B in that section.
• Data: Base set of 3,339 items (2,339 harmful + 1,000 benign), plus 4,464 jailbreak queries. Guardrails are evaluated on an 8:2 split test set; the paper does not report the number of test set items.
• Attacks: AutoDAN queries are generated on LLaMA 3.1-8B and then transferred. None of the models in Table 1 are the white-box generation target itself. Another method is PAP. Out-of-distribution testing uses DeepInception.
• Evaluation: For LLMs, ASR is computed using refusal-prefix keyword matching, not LLM judges. For guardrails, FNR, FPR, F1, and ACC are used. Temperature is 0.7. The paper does not report the number of ASR repetitions.
• Human evaluation: Random 30 harmful queries, collecting successful jailbreak outputs from GPT-4.1, GLM-4-32B, LLaMA-3.3-70B, Mistral-Small4, and Qwen3-32B, totaling 150 outputs; three experts score on a 0–2 scale, and experts are selected via 10 pilot samples (Appendix E).

Main Results

Table 1 shows the ASR when both jailbreaks are used together. The table below lists only Overall. GPT-4.1 (50.54%) and Qwen-3-8B (56.38%) are omitted. The table note says ↑ indicates higher is better from the attacker's perspective.

Model	Overall ASR
Claude-3.7-Sonnet	34.59%
GPT-4o	54.84%
GLM4-32B	60.84%
LLaMA-3.3-70B	55.49%
Mistral-Small4	67.58%
Qwen-2.5-7B	62.99%
Qwen-3-32B	62.76%

• The authors state that Claude-3.7-Sonnet has the best overall safety, and Mistral-Small4 is the most vulnerable; they also state that without jailbreaks the average ASR is 18.11%, and with both jailbreaks the overall ASR is 56.22%. The latter two numbers are author summaries, not a specific row in the table.
• Based on this, the authors conclude that the food science domain lacks robust safety boundaries and requires targeted safety alignment.
• Figure 7 breaks down ASR by method. The authors state that PAP is higher than AutoDAN in most cases, and the average ASR for both methods exceeds 50%. After text conversion, the bars do not align with the models, so numbers are not copied from the bars per model.

Human Usefulness

• What was tested: Successful jailbreak responses to 30 harmful queries across five models, scored by three experts. Appendix E reports Fleiss' Kappa as 0.4684, which the authors describe as moderate agreement.
• Results: The authors report an average score of 1.398, with Mistral-Small4 at 1.584, and state it is the highest among the five models. Figure 5 contains per-expert breakdowns, but the text cannot be reliably aligned to models.
• Authors' interpretation: The authors believe these frontier models can generate food-related information useful for malicious goals, posing a real-world threat.

Guardrails

Values are from Table 2. The text refers to the same columns as percentages, e.g., FNR 2.75%, F1 97.10%, accuracy 95.24%. The paper does not report the number of test set items. The Qwen3-8B and LLaMA3.1-8B in Table 2 correspond to the same models with hyphenated names in §4.1 and §3.2.4, used as comparison models by the authors.

Swipe horizontally to read the table

Guardrail	FNR	FPR	F1	ACC
LLaMA3.1-8B	26.25%	6.28%	84.17%	77.60%
Qwen3-8B	24.55%	4.71%	85.51%	78.99%
LLaMA-Guard4-12B	59.71%	3.66%	57.12%	59.71%
LLaMA-Guard3-8B	42.40%	4.19%	64.29%	72.69%
Qwen3Guard-8B	29.52%	2.62%	82.41%	75.18%
Qwen3Guard-4B	21.43%	4.71%	81.50%	87.53%
FoodGuard-4B	2.75%	2.01%	97.10%	95.24%

• The authors state that existing guardrails have low FP but high miss rates, with an average FN of 38.27% for the four dedicated safety models; they also state that Qwen3-8B and LLaMA3.1-8B have higher F1 and accuracy than the dedicated guardrails. The authors write that the evaluated models' FP ranges from 2.62% to 6.28%; in Table 2, FoodGuard-4B's FPR is not in that range.
• The authors state that FoodGuard-4B outperforms the baselines in the table on all metrics, reducing FNR relative to the best baseline Qwen3Guard-4B, and also having the lowest FPR, so they believe higher sensitivity does not come with excessive refusal.

Out-of-Distribution Jailbreak

• What was tested: Applying DeepInception to vanilla harmful and benign prompts in the test set.
• Results: The authors state that FoodGuard-4B has a recall of 100%, FNR of 0%, and ACC of 99.18%, and claim there are almost no false positives; the paper does not report FPR for this experiment. The authors write that Qwen3Guard-88 and Qwen3Guard-48 have bypass rates of 13.8% and 4.1%, respectively. Elsewhere in the text, the comparison names are Qwen3Guard-8B and Qwen3Guard-4B, and the paper does not clarify whether the two sets of names refer to the same models.
• Authors' interpretation: The authors state that DeepInception remains effective against general models, and FoodGuard-4B remains robust against out-of-distribution jailbreaks.

Other Ablations and Analyses

• Vanilla ASR (Appendix Table 5): Claude-3.7-Sonnet 10.56%, GPT-4o 19.66%, GPT-4.1 10.85%, GLM-4-32B 10.00%, LLaMA-3.3-70B 32.65%, Mistral-Small4 31.81%, Qwen3-8B 13.16%, Qwen3-32B 15.90%, Qwen2.5-7B 18.42%; the authors state the average is 18.11%.
• The authors state that Pest Control has the highest ASR across five models, with a nine-model average of 64.81%. Counterexamples from Table 1: Claude's Water safety 60.00% and Allergens 55.31% are higher than Pest 50.00%; GLM4-32B's Pest 46.67% is lower than Hygiene 65.10%; LLaMA-3.3-70B's Allergens 68.16% is higher than Pest 66.67%.
• The authors state that Storage, Hygiene, and Temperature Control have ASR exceeding 50%. In Table 1, Claude's Hygiene is 33.27%, Storage is 31.72%, and Temperature control is 42.69%. The paper does not clarify whether "exceeding 50%" refers to category averages or per-model, nor does it report the denominators for each category in Table 1.
• Appendix Table 3 shows pest control as 30/2339 and water safety as 15/2339, which correspond only to vanilla harmful queries and cannot be directly used as denominators for Table 1.
• The paper does not have a separate hyperparameter ablation table. On the attack side, only temperature 0.7 is reported; guardrail training hyperparameters are in Appendix D.2, with no comparison experiment numbers.

Limitations and future directions noted by the authors

• Single-model interaction: The evaluation focuses on independent model interactions; real deployments include multi-agent pipelines such as retrieving recipes and then rewriting them according to dietary constraints, which introduce compounding risks. Future work should extend the benchmark to multi-agent settings (§6).
• Embodied scenarios: Future work should study embodied settings. The threats in this paper are primarily informational, meaning users may not necessarily act on unsafe advice; LLMs could also control vending machines, kitchen equipment, or production processes. The authors believe these risks will be critical (§6).
• User context: Scenarios are derived from FDA codes. The authors state that real-world risks are more personalized, dynamic, and context-dependent: preparation practices that are low-risk for healthy adults may be dangerous for immunocompromised individuals, pregnant women, or those with severe allergies. Future work should evaluate whether models adjust thresholds based on user context or use a uniform policy (§6).

The paper does not have a separate Limitations section. The above three points all come from §6 Future work.

Experimental coverage

• The tested dialogue models are the 9 LLMs without thinking capabilities from §4.1; ASR is reported in Table 1 and Appendix Table 5, with refusal determined by prefix keyword matching.
• Jailbreaks are AutoDAN (generated on LLaMA 3.1-8B and then transferred) and PAP; out-of-distribution testing of guardrails uses DeepInception (§3.2.4, §5.3).
• Guardrail comparisons are the 7 systems in Table 2. FoodGuard-4B is fine-tuned from the Qwen3Guard series; training hyperparameters are in Appendix D.2.
• Human evaluation covers 5 models, 30 queries, 150 responses, and 3 experts, with Fleiss' Kappa of 0.4684 (§5.2, Appendix E).
• The paper does not report the number of guardrail test set entries, the number of ASR repetitions, or the split of the 4,464 jailbreak queries between AutoDAN and PAP; nor does it state whether benign queries are included in the 8:2 split (§3.2.4, §5.3).

The authors frame LLM safety in food assistance as a domain problem that is both evaluable and interceptable.

• Problem: General alignment and general guardrails do not cover hazards such as temperature, contamination, hygiene, storage, preparation, allergens, pests, and water use. The authors state that incorrect storage or preparation guidance can lead to contamination, foodborne illness, or allergies.
• Approach: They extract 356 seed policies from FDA regulations, generate queries with GPT-5, and after filtering obtain FoodGuardBench with 3,339 items; then add 4,464 jailbreak queries using AutoDAN and PAP. FoodGuard-4B is fine-tuned on their own data as a review model that does not modify the underlying weights.
• Attack Results: The authors report an average ASR of 18.11% without jailbreaks, and an overall ASR of 56.22% under two jailbreaks. In Table 1's Overall ASR, Claude-3.7-Sonnet is 34.59% and Mistral-Small4 is 67.58%. The average human usefulness of 150 successful jailbreak responses across five models is 1.398.
• Guardrail Results: On the Table 2 test set, FoodGuard-4B has FNR 2.75%, FPR 2.01%, F1 97.10%, and accuracy 95.24%. The authors state that the average FN for four specialized guardrails is 38.27%. Under DeepInception, the authors state this guardrail has recall 100%, FNR 0%, and ACC 99.18%.
• Authors' Conclusion: The authors believe that existing LLMs and guardrails have sparse alignment in the food domain, and that a small number of representative jailbreaks can elicit executable harmful guidance, and they hope to promote safety research in everyday high-impact domains. §6 leaves multi-agent, embodied control, and user context for future work.