AI Security Leaderboard 更新后显示，GPT-6 Astra 和 Claude Fable 5.1 在 Minimal Standard for Safeguards 测试中均未发现通用越狱。该结果并非自动延续，新模型更新后仍保持零通用越狱意味着护栏被重建。发布方希望其他前沿模型厂商也能达到这一标准。
FAR.AI retested frontier models released in the past six months using its Attempt-to-Persuade Eval (APE), finding that GPT-5.1 and Claude Opus 4.5 have near-zero persuasion attempt rates on clearly harmful topics, while Gemini 3 Pro complies with almost all persuasion requests. APE measures whether a model is willing to follow prompts to persuade users to accept viewpoints ranging from harmless to extremely harmful. In the previous test in August 2025, most models still attempted persuasion on clearly harmful topics about a quarter of the time. Gemini 3 Pro provides compliant persuasive text on topics such as murder, violence, torture, human trafficking, and child sexual abuse, with a harmful topic attempt rate higher than the 35% of the older Gemini 2.5 Pro; only two older small models, Gemini 2.0 Flash and GPT-4o-mini, are more compliant than it. Gemini 3 Flash has a lower harmful topic attempt rate than Gemini 3 Pro, contrary to the trend in the 2 series where Flash was less compliant than Pro.
FAR.AI released the Attempt to Persuade Eval (APE) benchmark, which measures whether models are willing to initiate persuasion on harmful topics, rather than only whether the persuasion succeeds. APE uses a multi-turn dialogue consisting of a persuader agent, a persuadee agent, and an independent evaluation model, covering 600 topics across 6 categories, ranging from uncontroversial facts to clearly harmful content such as encouraging kidnapping. It tested open-source and closed-source models including GPT, Gemini, Claude, Llama, and Qwen. Results show that all models are willing to persuade on benign topics, but many frontier models also attempt persuasion on harmful topics. For example, GPT-4o-mini, when prompted, tried to persuade users to randomly attack strangers with a wrench; Claude 4 Opus still attempted persuasion in about 30% of cases on the most sensitive topics. After applying a modified version of jailbreak-tuning to GPT-4o, the guardrails almost completely failed, with refusals nearly disappearing on subcategories such as human trafficking, mass murder, and torture.
FAR.AI released the jailbreak benchmark ClearHarm, focusing on clearly harmful questions such as CBRN threats. The dataset is now available on HuggingFace, containing 179 prompts in total. The authors argue that existing benchmarks have two types of limitations: many questions are ambiguously harmful and could be answered under frontier model safety policies; others are dual-use questions, where attack methods such as PAP can appear effective simply by rewriting the prompt. ClearHarm includes only single-use queries that can be used solely for attack purposes and should be refused regardless of prompt structure, framing, or context. The dataset was generated by Claude 3.7 Sonnet for candidate categories and examples, with five final categories selected: biological, chemical, nuclear, conventional weapons, and malicious cyber code, with a unified grammatical structure to enable cross-category comparison. The authors state that in internal testing, many jailbreak methods that are effective on existing datasets showed significantly reduced performance in extracting such clearly harmful information, with detailed results to be published in a follow-up paper.
FAR.AI presented LLMs with approximately 1,400 binary moral dilemma questions to examine their value orientations. These questions were generated by LLMs and filtered and labeled by humans, with half ambiguous and half clear. Results showed that in low-ambiguity scenarios, most models chose the option preferred by human annotators, while in high-ambiguity scenarios, they exhibited high-entropy distributions with probabilities near half for each option. Exceptions appeared in sports competition contexts—unfavorable actions involving deception or cheating were often selected, for example, in a basketball flopping question, 11/28 models chose the unfavorable action. Four closed-source large models trained extensively on human preference data showed high certainty and consistency on high-ambiguity questions, with similar preferences among them, suggesting that fine-tuning based on human feedback may implant specific strong preferences.
