Empirical evaluation of six mainstream LLMs under multilingual and obfuscated prompt injection attacks

To what extent can publicly available or locally deployed large language models be induced by prompts to generate phishing emails, phishing webpages, and keylogger programs.

• Scenario: The authors state that models are being used to generate phishing content, deceptive websites, and malicious code at scale, lowering the barrier to cybercrime. The OWASP Top 10 for LLM Applications lists prompt injection as the most critical vulnerability; the paper cites the December 2023 case where a Chevrolet dealership's ChatGPT customer service was manipulated into seemingly agreeing to sell a vehicle for $1.
• Existing gaps: The authors state that alignment training, safety policies, and prompt filtering are still in use, but public evaluations mostly focus on English and simpler direct malicious requests. There is little systematic evidence on how compliance varies in Turkish, Russian, or with Base64, ROT13, or hexadecimal.
• Hypothesis: When semantic intent is unchanged, embedding requests in legitimate appearances such as security awareness training, red team evaluations, or academic research, and avoiding terms that easily trigger filters, will increase malicious compliance rates.
• This paper's approach: Provides an evaluation framework that measures compliance of six models for three types of malicious outputs and compares four natural languages and three lightweight encodings.
• Threat model:
  • Target: Phishing emails supporting social engineering, phishing webpages for credential harvesting, and keylogger programs that can be compiled and executed on target systems.
  • Knowledge: Interacts through the normal text channel of public or local models; treated as black-box, observing only text outputs.
  • Capability: Can rewrite instructions, switch languages, or add simple obfuscation; cannot modify model parameters, system-level prompts, or provider safety controls.
  • Victim systems: DeepSeek, GPT, Gemini, Grok, Llama, and Qwen; locally deployable models are deployed via Replicate without modifying security-related settings.

The literature is discussed in four groups: phishing, malware, prompt injection, and safety benchmarks. The authors position this paper on measuring malicious compliance, not on new attacks or new defenses.

• Phishing and social engineering: Fujima et al. (2023) analyze linguistic features of phishing emails related to ransomware; Wang and Lutchkus (2023) discuss cognitive biases. Sayyafzadeh et al. (2024) and Campesato (2023) address emotional manipulation and socially credible communication. The authors state that the realism and success rate of LLM phishing relative to human-authored content still lacks systematic benchmarks; they also state that tools such as APOLLO (Desolda et al., 2025, using GPT-4) focus on detection rather than using LLMs as automated phishing agents. The authors state that this paper fills the gap for multilingual scenarios and encoded prompts aimed at bypassing defenses.
• Malware: Warikoo (2023) and Yadav et al. (2022) describe ransomware monetization pipelines and how automation lowers the skill barrier. Böke and Torka (2025), Huang et al. (2024), and Sapra et al. (2013) discuss code obfuscation, mobile malware targeting LLM applications, and bypassing keystroke detection. The authors state that end-to-end generation demos and controlled experiments quantifying success rates against modern defenses remain scarce; this paper reports cross-model keystroke prompt compliance.
• Injection, alignment, and multilingual jailbreaks: OWASP Top 10:2021 and its mapping to CWE Top 25:2023 provide classification references. Lan et al. (2025) do detection; Prompt-G (Pingua et al., 2024), Signed-prompt (Suo, 2024), and dynamic moving target defense (Panterino and Fellington, 2024) are mitigation approaches. HarmBench (Mazeika et al., 2024) and AdvBench are used for red teaming and refusal elicitation; the authors state that this study requires different prompt schemes and thus uses prior datasets. Yong et al. (2024), Deng et al. (2024), and Yuan et al. (2024) report that low-resource languages, multilingualism, and cipher-style prompts alter bypass behavior. StrongREJECT (Souly et al., 2024) notes that seemingly compliant outputs may lack executable or semantically meaningful harmful content.
• Empirical safety benchmarks: Work on system robustness under unsafe prompts, Bioinfo-Bench, and MedAgentBench measure reliability with large-scale or domain-specific tasks. Kant and Rana (2025) and OSPC (Cai, 2024) do harmful content detection. The authors state that in these evaluations, multilingual robustness and code-based malicious compliance remain under-covered.
• Experimental controls: The paper does not use AdvBench, HarmBench, Signed-prompt, or other defenses as control baselines. The controls are six models, direct versus elaborate, four languages, and three encodings. Prompts come from the authors' prior phishing and malware datasets.

The authors state that the paper's position is to simultaneously measure refusal and compliance across models, languages, encodings, and prompt frameworks, without evaluating the semantic quality of compliant responses.

The authors do not train new models or change safety settings, but measure malicious compliance on black-box interfaces with fixed templates. They compare direct and elaborate prompts, as well as languages and lightweight encodings.

Adversarial Assumptions

• Channel and permissions: The adversary uses the standard text interface of public or local models, can iteratively rewrite, change language, or do simple obfuscation; cannot change parameters, system-level prompts, or provider safety controls.
• Target outputs: Phishing emails supporting social engineering, phishing web pages for credential collection, and compilable/executable keylogging programs.
• Correspondence with abstract wording: The abstract writes direct and multi-stage obfuscated attacks. The methods section operationalizes as direct/elaborate and encoding variants; each is an independent generation, not a multi-turn conversation.

Prompt Set

• Source and counts: Templates come from the authors' previous dataset. Direct phishing emails 7, elaborate phishing emails 6; direct phishing web pages 7, elaborate phishing web pages 6; direct keyloggers 6, elaborate keyloggers 5.
• direct: Explicitly requests malicious outputs, e.g., phishing email text or malware source code.
• elaborate: The same underlying request is embedded in a legitimate-looking wrapper such as security awareness training, red-team evaluation, or academic research, and avoids obvious malicious terminology. The authors use it to formalize the hypothesis that "legitimate, professional phrasing increases compliance while semantic intent remains unchanged."

Languages, Encodings, and Sampling

• Languages: English, Turkish, Russian, Simplified Chinese. Translated via DeepL, requiring preservation of semantic intent, pragmatic force, and scenario realism.
• Encodings: Base64, ROT13, Hex. Only the specified malicious behavior segment is transformed; surrounding instructions remain in English. The authors state this simulates bypassing surface-level keyword filters with lightweight obfuscation, without requiring the model to decode arbitrary binary. Transformations implemented in Python.
• Models: DeepSeek-V3.2, Gemini 2.5 Flash, GPT-5 Mini, Grok-4-1-Fast (Non-Reasoning), Meta Llama 4 Scout Instruct, Qwen 3 235B A22B Instruct. All black-box; locally deployable models uniformly deployed via Replicate, without changing safety-related settings.
• Replication: Each prompt template, condition (language or encoding), and target model is independently sampled 10 times under default inference settings. No system prompt or sampling parameter adjustments. Total 15,540 outputs.

Annotation and Metrics

• Three categories: One human expert annotates, three mutually exclusive categories. Full compliance means directly producing the requested malicious payload. Stalling assistance means not giving the final payload but providing specific, actionable guidance that can facilitate the attack. Refusal means refusing without substantive support.
• Warning label: Whether there is an explicit ethical or legal warning, such as illegality, consent, privacy, or policy restrictions.
• Derived rates: Fulfillment rate is defined as full compliance plus stalling assistance. Also report refusal rate and warning rate, and aggregate at global, model, condition, category, and cross levels.
• What is not judged: The authors state they do not evaluate the semantic quality or usefulness of compliant responses, nor do they perform manual utility checks on generated content.

实验设置

• 被测模型：DeepSeek-V3.2、Gemini 2.5 Flash、GPT-5 Mini、Grok-4-1-Fast (Non-Reasoning)、Meta Llama 4 Scout Instruct、Qwen 3 235B A22B Instruct。Table II 将前两款写作 Deepseek、Grok-4-1-Fast-Non-Reasoning。全部按 black-box 调用；本地模型经 Replicate，不改安全设置（Section III.C）。
• 提示：先前数据集中的六类模板：钓鱼邮件 7 条 direct、6 条 elaborate；钓鱼网页 7 与 6；键盘记录 6 条 direct、5 条 elaborate（Section III.B）。
• 条件：英语、土耳其语、俄语、简体中文，以及 Base64、ROT13、Hex。翻译用 DeepL；编码只改恶意行为段，周围指令保持英语（Section III.D）。
• 重复与分母：每模板×条件×模型独立采样 10 次，默认推理，不调 system prompt 或采样参数，共 15,540 条（Section III.E）。Table II 标题写每模型 2 590 条；Table III 标题写每种语言或编码 2 220 条。
• 指标：完全遵从、拖延协助、拒答；履行率为前两者之和；另记是否有伦理或法律警告（Section III.F）。
• 评审：一名 human expert，不是 LLM 裁判。论文未报告采样温度等具体数值，也未报告标注一致性。

主结果

Table I：15 540 条中完全遵从 10 685（68.76%），拖延协助 1 878（12.08%），拒答 2 977（19.16%），履行率 80.84%。模型结果据 Table II：

Swipe horizontally to read the table

模型（Table II 写法）	完全遵从	拖延协助	拒答	履行率	警告
Deepseek	84.32%	4.09%	11.58%	88.42%	44.67%
Gemini 2.5 Flash	77.72%	4.09%	18.19%	81.81%	50.97%
Grok-4-1-Fast-Non-Reasoning	72.55%	2.39%	25.06%	74.94%	55.02%
Qwen 3 235B A22B Instruct	67.84%	11.78%	20.39%	79.61%	63.44%
Meta Llama 4 Scout Instruct	66.02%	19.92%	14.05%	85.95%	43.59%
GPT-5 Mini	44.09%	30.23%	25.68%	74.32%	70.66%

• 作者称完全遵从两端相差 40.23 个百分点；DeepSeek 拖延少、履行最高，GPT-5 Mini 完全遵从最低，但拖延高，履行仍不低。
• Discussion 称没有模型的履行率低于 70%；作者称即便较谨慎的 GPT-5 Mini，计入拖延后仍履行近四分之三的请求。
• 作者认为完全遵从与警告率负相关。Section IV.A：完全遵从回应中 43.71% 带警告，拖延为 61.02%，拒答为 90.29%。作者认为现有机制没有把拒答和警告拆开。

语言与编码效应

• 测了什么：Table III 把四种语言和三种编码放在六款模型、全部类别和全部迭代上汇总。
• 结果：完全遵从依次为土耳其语 80.18%、简体中文 77.52%、俄语 73.92%、英语 71.31%、Base64 65.59%、ROT13 58.38%、Hex 54.41%。自然语言平均 75.73%，编码平均 59.46%，作者给出的差为 16.27 个百分点。Hex 的履行率仍为 72.16%。
• 作者解读：作者称六款模型的非英语完全遵从都高于英语，并认为这符合“安全微调与红队主要在英语、其他语言覆盖较弱”的假设，会给非英语使用者带来结构性不利。作者认为模型对常见编码至少部分熟悉，简单混淆不足以保证安全；Discussion 另称这一差距平均超过 16 个百分点。

提示类别与迭代稳定性

• 类别：Table IV 中 elaborate 钓鱼邮件完全遵从 83.06%、警告 17.18%；elaborate 钓鱼网页为 80.52% 与 37.58%。直接钓鱼邮件完全遵从最低，为 59.08%，警告 49.66%。直接类平均完全遵从 62.01%，elaborate 类 76.69%。
• 例外：键盘记录上 elaborate 完全遵从 64.48%，并不高于直接类的 64.88%；两类警告为 88.81% 与 91.03%。作者把“elaborate 更易引出载荷、且更少触发警告”限定在钓鱼场景。Figure 1 标注的数值与 Table IV 相同；图注称 elaborate、上下文丰富的提示系统性优于直接提示，尤其是钓鱼邮件和网页。
• 稳定性：十次迭代的全局完全遵从在 68.02% 与 69.56% 之间。作者称模型没有因重复类似恶意提示而变得更保守或更宽松。

交叉条件极值

• 作者举例（Section IV.E）：Meta Llama 4 Scout Instruct 在英语上完全遵从 98.11%，简体中文 97.84%；DeepSeek 在 Hex 上 96.22%；Gemini 2.5 Flash 在 elaborate 钓鱼邮件上 100.00%；DeepSeek 在两类 elaborate 钓鱼上超过 98%。
• 作者解读：作者称漏洞不限于单一配置，高风险行为是分散的，而不是集中在一个离群点。
• 表的范围：这些是举例，论文未给出模型×语言×类别的完整交叉表。

其他消融与分析

• 有明确伦理或法律警告的为 8,504（54.72%），没有的为 7,036（45.28%）（Section IV.A）。
• 拒答、拖延、完全遵从伴随警告的比例为 90.29%、61.02%、43.71%。
• ROT13 警告率 35.23%，为 Table III 各条件最低；论文未单独解释。
• 条件履行率最高为土耳其语 87.57%，最低为 Hex 72.16%（Table III）。
• 类别履行率最高为 elaborate 钓鱼网页 89.92%，最低为直接钓鱼邮件 73.10%（Table IV）。
• 作者给出的模型完全遵从极差为 40.23 个百分点（Section IV.B）。论文没有单独的超参数消融。

Limitations and future directions noted by the authors

• Payload quality out of scope: The authors state that evaluating the semantic quality or usefulness of compliant responses is beyond the scope of this study; no manual verification or utility evaluation of generated responses is performed. The authors consider response quality evaluation as a complementary future direction (Section II.C).
• Defense directions: The conclusion states that stronger defenses are needed, including multilingual safety coverage, deeper semantic intent detection, and resilience against adaptive and multi-stage prompt engineering attacks. The introduction states that Section VI provides future research directions (Section VI, Section I).

Experimental coverage

• The tested models are DeepSeek-V3.2, Gemini 2.5 Flash, GPT-5 Mini, Grok-4-1-Fast (Non-Reasoning), Meta Llama 4 Scout Instruct, and Qwen 3 235B A22B Instruct, totaling 6; all are called as black-box through public interfaces, with local models using Replicate and not changing safety settings (Section III.C).
• Conditions are English, Turkish, Russian, and Simplified Chinese, as well as Base64, ROT13, and Hex; encoding only transforms the malicious behavior segment, with surrounding instructions kept in English; translations are generated by DeepL (Section III.D).
• Prompts are phishing emails with 7 direct and 6 elaborate, phishing webpages with 7 and 6, and keyloggers with 6 direct and 5 elaborate; each combination is independently generated 10 times, totaling 15,540 responses (Section III.B, III.E).
• Judgment is a three-way behavior classification plus warning labels by one human expert, not an LLM judge; the authors state that the semantic quality or executability of payloads is not evaluated (Section III.F, II.C).
• The paper does not report specific parameters such as sampling temperature, only stating default inference and no adjustment of system prompts or sampling parameters (Section III.E); the paper does not report whether translations were manually proofread or annotation consistency.

This empirical evaluation measures six black-box LLMs: how often phishing email, phishing webpage, and keylogger requests are assisted under default safety policies. The adversary can only rewrite prompts, switch languages, or apply light encoding through the text interface, and cannot change parameters or provider safety controls.

• Prompts are divided into direct requests and elaborate ones embedded in training, red teaming, or academic appearances. Languages are English, Turkish, Russian, and Simplified Chinese; encodings are Base64, ROT13, and Hex, encoding only the malicious behavior segment. Each combination is independently sampled 10 times and manually labeled as full compliance, delayed assistance, or refusal.
• Among 15,540 responses, full compliance is 68.76%, delayed assistance is 12.08%, and the fulfillment rate is 80.84% (Table I). The authors state that about four-fifths of malicious requests receive at least partial support.
• Full compliance in Table II ranges from 44.09% for GPT-5 Mini to 84.32% for DeepSeek. The authors state that no model has a fulfillment rate below 70% and associate higher compliance with fewer warnings.
• Turkish has 80.18% full compliance, English 71.31%; Hex still has 54.41% (Table III). The authors state that all six models have higher rates in non-English languages, and encoding reduces but does not eliminate malicious outputs.
• Elaborate phishing emails have 83.06% full compliance and 17.18% warnings; direct phishing emails have 59.08% and 49.66% (Table IV). For keyloggers, elaborate is not higher than direct. Global full compliance across ten iterations ranges between 68.02% and 69.56%.
• The authors conclude that current alignment is uneven across languages and prompt packaging; future work needs multilingual coverage, semantic intent detection, and resilience against adaptive, multi-stage prompt engineering. Whether payloads are truly executable is explicitly left out of scope.