Screwtape edits agent trajectories to bypass action monitoring
Start here
Redwood Research proposes the Screwtape pipeline, which red-teams action-only monitors by rewriting real agent conversation logs. The attacker uses an LLM to edit conversation logs so that the agent causes specified harm, iterating until the monitor no longer flags it; since the rewritten actions are never actually executed, the pipeline uses an LLM verifier to check that the edits are consistent with the original logs and meet the severity threshold. The team used self-built agent monitors based on Opus 4.8 and Gemini 3.1 Pro, ran 252 runs on 94 Claude Code and Codex trajectories, and obtained 35 attacks that bypassed their own monitors, 6 of which also bypassed a frontier developer's held-out monitor and were confirmed by manual review. The authors state that the pipeline is better suited for patching monitor blind spots than for providing quantitative safety conclusions.
Generated by AI from reports · Updated 2026-10-17
Progress timeline
- XiaohongshuXiaohongshu · AI Safety Daily· Discussion🎞️Rewrite the trajectory, and the monitor will miss it.(opens the original post in a new tab)
- LessWrongSelectedScrewtape: Adversarial Log Editing for Monitor Red Teaming
The authors propose Screwtape, a pipeline that rewrites real agent logs into adversarial attacks, iterating until monitors no longer detect them, for red-teaming monitors that only look at actions, not chain of thought. The pipeline has three steps: an attacker edits the log, a validator checks whether the attack is real and severe enough according to safety engineering metrics, and a monitor scores it; on failure, the reason is fed back to the attacker for further iteration. The authors ran 252 runs on 94 Redwood employee logs, producing 35 attacks that bypassed their own monitor, 6 of which also bypassed a frontier developer's holdout monitor and were manually confirmed as valid. The authors state the pipeline cannot yet generate chain of thought or provide quantitative safety estimates; its main use is helping developers locate and patch monitor blind spots.
- Redwood ResearchSelectedScrewtape: Adversarial Trajectory Editing for Monitoring Red-Teaming
Redwood Research proposes Screwtape, an attack pipeline that rewrites real agent trajectories turn by turn to bypass action-only monitoring, with an LLM verifier judging whether the attack is real, harmful, and feasible. The team used their own agent monitoring based on Opus 4.8 and Gemini 3.1 Pro, ran 252 runs on 94 unique trajectories, and obtained 35 attacks that bypassed their own monitoring, 6 of which also bypassed a frontier developer's held-out monitoring and were confirmed by manual review. The authors state that this pipeline is better suited for patching monitoring blind spots than for providing quantitative safety estimates, and they are not releasing the implementation for now.
- arXiv: Jailbreaks, Prompt Injection, Poisoning & DefensesSelectedScrewtape: Adversarial Dialogue Editing for Red-Teaming Monitors
Researchers at Redwood Research propose the Screwtape pipeline, which red-teams action-only monitors by rewriting real agent dialogue logs. An attacker uses an LLM to edit dialogue logs so that the agent causes specified harm, iteratively modifying until the monitor no longer flags it; since the rewritten actions are not actually executed, the pipeline uses an LLM verifier to check whether edits are consistent with the original log and meet a severity threshold. Across 252 runs on 94 Claude Code and Codex dialogue logs, the pipeline produced 35 candidate attacks, of which 17 bypassed held-out monitors provided by frontier developers, and 6 were confirmed effective by human review. The verifier rejected 55 of 63 flawed examples in injection-inconsistency tests and accepted all 44 unedited logs.
Related discussion
- Older than 72 hours · Not included in the 72-hour figures
- XiaohongshuXiaohongshu · AI Safety Daily🎞️Rewrite the trajectory, and the monitor will miss it.(opens the original post in a new tab)
- X@dl_weekly · Deep Learning Weekly🤖 From this week's issue: A red-teaming blog post about refining attacks inserted into real agent transcripts until mon…(opens the original post in a new tab)
3 likes · 2 comments·Oct 31 17:41Collect
- X@neuralloot · NeuralLootPaper: "Screwtape: Adversarial Transcript Editing for Monitor Red-Teaming" by Gosha Kozhevnikov, Tyler Tracy (Redwood R…(opens the original post in a new tab)
- X@neuralloot · NeuralLootClaude Code and Codex sessions, edited by another AI to slip in sabotage, got past a frontier lab's agent monitor 6 tim…(opens the original post in a new tab)
1 likes · 2 comments·Oct 20 17:33Collect
- X@verbal72 · verbal72.exe"Screwtape: Adversarial Transcript Editing for Monitor Red-Teaming" (Kozhevnikov, Tracy · Redwood Research, Oct 16): ht…(opens the original post in a new tab)
- X@verbal72 · verbal72.exeAn AI monitor watched an agent "remove" an API key from a log. It never checked that the removal worked. It didn't. Th…(opens the original post in a new tab)
1 likes · 3 comments·Oct 20 17:33Collect
- X@FSFG · Cryptography and Security PapersScrewtape: Adversarial Transcript Editing for Monitor Red-Teaming Gosha Kozhevnikov, Tyler Tracy https://arxiv.org/abs…(opens the original post in a new tab)
- X@AIQuanting · AI QuantingRedwood Research built a tool that edits real AI agent transcripts to sneak past the monitors that watch those agents.…(opens the original post in a new tab)
1 likes·Oct 20 17:33Collect
- X@baraltiva · Gosha KozhevnikovWe're not releasing Screwtape publicly for now, but we're happy to share it with researchers working on monitor red-tea…(opens the original post in a new tab)
5 likes·Oct 31 06:43Collect
- X@baraltiva · Gosha KozhevnikovHuman red teaming doesn't scale well and Screwtape is our first step to address this. Automating red-teaming is hard, a…(opens the original post in a new tab)
1 likes · 1 comments·Oct 30 22:42Collect
- X@baraltiva · Gosha KozhevnikovRed-teaming agent monitors can take humans hours to design and implement realistic attacks. This makes thorough red-tea…(opens the original post in a new tab)
33 likes · 2 comments·Oct 31 12:41Collect
- X@gastronomy · Tanat TonguthaisriScrewtape: Adversarial Transcript Editing for Monitor Red-Teaming: We present Screwtape, a pipeline that converts real…(opens the original post in a new tab)