Study: Separating self-critique from external proposals improves red-team attack success rate

1. 10:49
   XiaohongshuXiaohongshu · Liu Dongrui (Shanghai AI Lab)· Discussion

   UIUC: Self-Improving Red Teaming(opens the original post in a new tab)

1. 15:34
   arXiv

   The target agent is its own best critic but a poor proposer: self-improving red teaming

   Researchers found that in automated red teaming, target-session self-critique is effective while self-proposal is poor, and on this basis proposed a role-separated red teaming method that achieved 63.2% overall attack success rate (ASR) on DTAP-BENCH, clearly higher than the baseline. Self-critique is more effective because it is based on the target session's real-time execution context, including observations, constraints and available reasoning; removing reasoning reduces downstream attack success rate by 6%-17%. Self-proposal performs clearly worse because awareness of the adversarial goal induces explicit refusals and non-refusal deliberate sandbagging. The role-separated method has the target session diagnose the reasons for attack failure, while a separately instantiated attacker generates the next proposal; it was validated on DTAP-BENCH covering the Telecom, Workflow, CRM and Code domains, under both direct misuse and indirect threat threat models and across multiple Agent frameworks. The paper concludes that effective automated red teaming depends not only on model capability and search strategy, but also on how execution context and adversarial responsibility are allocated across different model sessions.

Current attention 28·Peak in the comparable period 28 (Nov 7 01:00)·Change over the comparable past 24 hours 0%