Skip to content
EventPast event

Screwtape edits agent trajectories to bypass action monitoring

3 reports3 reporting sourcesLatest report

Start here

AI overview

Redwood Research proposes the Screwtape pipeline, which red-teams action-only monitors by rewriting real agent conversation logs. The attacker uses an LLM to edit conversation logs so that the agent causes specified harm, iterating until the monitor no longer flags it; since the rewritten actions are never actually executed, the pipeline uses an LLM verifier to check that the edits are consistent with the original logs and meet the severity threshold. The team used self-built agent monitors based on Opus 4.8 and Gemini 3.1 Pro, ran 252 runs on 94 Claude Code and Codex trajectories, and obtained 35 attacks that bypassed their own monitors, 6 of which also bypassed a frontier developer's held-out monitor and were confirmed by manual review. The authors state that the pipeline is better suited for patching monitor blind spots than for providing quantitative safety conclusions.

Generated by AI from reports · Updated 2026-10-17

Progress timeline

Oct 17
  1. LessWrongSelected
    Screwtape: Adversarial Log Editing for Monitor Red Teaming

    The authors propose Screwtape, a pipeline that rewrites real agent logs into adversarial attacks, iterating until monitors no longer detect them, for red-teaming monitors that only look at actions, not chain of thought. The pipeline has three steps: an attacker edits the log, a validator checks whether the attack is real and severe enough according to safety engineering metrics, and a monitor scores it; on failure, the reason is fed back to the attacker for further iteration. The authors ran 252 runs on 94 Redwood employee logs, producing 35 attacks that bypassed their own monitor, 6 of which also bypassed a frontier developer's holdout monitor and were manually confirmed as valid. The authors state the pipeline cannot yet generate chain of thought or provide quantitative safety estimates; its main use is helping developers locate and patch monitor blind spots.

  2. Redwood ResearchSelected
    Screwtape: Adversarial Trajectory Editing for Monitoring Red-Teaming

    Redwood Research proposes Screwtape, an attack pipeline that rewrites real agent trajectories turn by turn to bypass action-only monitoring, with an LLM verifier judging whether the attack is real, harmful, and feasible. The team used their own agent monitoring based on Opus 4.8 and Gemini 3.1 Pro, ran 252 runs on 94 unique trajectories, and obtained 35 attacks that bypassed their own monitoring, 6 of which also bypassed a frontier developer's held-out monitoring and were confirmed by manual review. The authors state that this pipeline is better suited for patching monitoring blind spots than for providing quantitative safety estimates, and they are not releasing the implementation for now.

Oct 15
  1. arXiv: Jailbreaks, Prompt Injection, Poisoning & DefensesSelected
    Screwtape: Adversarial Dialogue Editing for Red-Teaming Monitors

    Researchers at Redwood Research propose the Screwtape pipeline, which red-teams action-only monitors by rewriting real agent dialogue logs. An attacker uses an LLM to edit dialogue logs so that the agent causes specified harm, iteratively modifying until the monitor no longer flags it; since the rewritten actions are not actually executed, the pipeline uses an LLM verifier to check whether edits are consistent with the original log and meet a severity threshold. Across 252 runs on 94 Claude Code and Codex dialogue logs, the pipeline produced 35 candidate attacks, of which 17 bypassed held-out monitors provided by frontier developers, and 6 were confirmed effective by human review. The verifier rejected 55 of 63 flawed examples in injection-inconsistency tests and accepted all 44 unedited logs.

Related discussion

Total 12 items

Attention trend

Hourly attention

Current attention 42·Peak in the comparable period 42 (Nov 13 03:00)·Change over the comparable past 24 hours 0%

0204060Nov 1303:00Nov 1312:00Nov 1320:00Nov 1405:00