Eight in Ten LLMs Leak System-Prompt Secrets: Nevan, an Agentic Capture-the-Flag Study
Eight in Ten LLMs Leak System-Prompt Secrets: Nevan, an Agentic Capture-the-Flag Study
Translation in progress — untranslated text is omitted.
Researchers used agentic capture-the-flag experiments to audit instruction hierarchy enforcement across 363 text-input-output models in the OpenRouter catalog, with about 80% of models persuaded to hand over secrets in their system prompts. In the experiments, each model's system prompt contained a hidden 16-character flag, and a human researcher-guided OpenCode agent driven by DeepSeek V4.1 Flash attempted to persuade the model to output the flag; flags were extracted from 296 of the 363 models, plus 4 models released after the catalog was frozen, for a total of 300 captures, of which 170 involved non-English language switching, most commonly French. The authors note that each capture only indicates that the model ranked a user request above system instructions at least once, and does not prove that it can resist untested attacks, will persistently leak semantically sensitive data, or is susceptible to harmful-content jailbreaks. The authors accordingly recommend that practitioners not store secrets in system prompts, enforce critical behaviors outside the model using output validation and deterministic checks, and not let LLM judges alone decide significant outcomes.