Jev 网关提示注入检测器基准测试

1. 06:40
   Cisco

   Jev 决策模型在内容安全任务上的对比评测：与传统分类器及 LLM 判官谁更强

   思科在 24 个危害类别、6 个评测数据集上对比了零样本决策模型 Jev（TypeSafe）与 Laya（ConvAI Innovations）和自训练的 1B 参数编码器分类器，在 0.5% 误报预算下，自训练模型在全部 6 个数据集上召回率最高，Jev 在 6 个中的 5 个 AUC 领先 Laya。Cisco 多轮测试集上，自训练模型召回 61%、Jev 34%、Laya 几乎为零。Jev 另与 Gemma 4 31B（关闭思考）在 8 个公开基准上对比，Jev 在 7 个基准 AUC 更高、1 个持平。

1. 01:43
   RedditReddit r/LocalLLaMA· Discussion

   Benchmarking decision models is fun - Clef Q8 vs Jev(opens the original post in a new tab)

1. Date not given
   Pillar Security

   Agents Without LLMs: What Jev and System One Models Mean for AI Safety

   Pillar Security analyzes TypeSafe's System One model Jev, arguing that this type of model, which does not generate text and only returns typed decisions with calibrated probabilities, removes the leverage points for three detection surfaces: prompt scanning, output filtering, and chain-of-thought monitoring. Jev was released on September 15, 2026, trained with RLCD, rewarding calibration rather than human preference, answering in parallel in a single forward pass, with a text input limit of 32k tokens. The authors say the attack surface shifts from the instruction channel to the data channel, equivalent to indirect prompt injection without prompts, and TypeSafe documentation also states that jev-1.13 does not treat state as adversarial by default. In a security operations alert triage evaluation, Jev's agreement rate with frontier references is 61.7%, at 0.3 seconds and $0.0001 per case, while Opus 5 achieves 66.2%, 15.1 seconds, and $0.0574. The article recommends moving defenses to the tool-call boundary and notes that its guardrails cookbook only covers jailbreaks and harmful content in chat text, not indirect injection or tool calls.

2. Date not given
   Jev Gateway StudySelected

   Benchmarking Jev against open-source decision model prompt injection detectors

   This benchmark compares Jev 1.13.0, Kev-4B, Kev-0.8B, Laya, GLiNER2.5-Decide, ProtectAI DeBERTa, Prompt Guard 2, and keyword rules on the same Apple M4 machine across two task types: document chunk injection and user message jailbreak. In the document chunk test, Jev achieves an AUC of 1.00, 100% recall at a frozen threshold with 8% false positives, Kev-4B achieves 0.90 and 70%, and Prompt Guard 2 achieves 0.83 and 19%; on covert attacks that do not directly mention AI, Jev achieves 97% recall, Kev-4B 62%, and the three dedicated injection classifiers perform near random levels. On 220 real human attacks from LLMail-Inject, Jev catches 95% with 0% false positives, Kev-4B catches 86% with 1% false positives; when used as an email assistant, gpt-oss-120b sends emails to the attacker's address in 20/220 cases, and with Jev in front, 0 pass through.

Current attention61·Peak in the comparable period62（Oct 12 17:00）·Change over the comparable past 24 hours +7%