Black-Box Auditing of Cognitive Reliability in Multi-Agent Debate Distillation

The researchers propose a two-stage black-box audit framework, ER-Audit, for detecting cognitive reliability degradation in multi-agent debate distillation, that is, adaptation that maintains performance on monitored tasks while reducing the verifier's support for correct answers on hidden tasks. The paper considers an adversarial debater that manipulates debate arguments while defending correct answers on monitored tasks. ER-Audit compares frozen verifier checkpoints before and after adaptation: it first searches for non-degenerate counterexamples through semantically valid rewrites, and if no counterexamples are found, it then performs sequential hypothesis testing with independent rewrites and derives an anytime-valid confidence lower bound on the non-degeneracy probability, supporting data-dependent stopping within a limited budget. The team also releases two evaluation benchmarks that pair monitored and hidden task prompts based on shared context, and the code and benchmarks are open-sourced on GitHub. Experiments show that higher hidden-task accuracy can coexist with more non-degenerate counterexamples and a lower non-degeneracy lower bound; this divergence challenges the explanation based solely on catastrophic forgetting and indicates that auditing can reveal selective degradation on hidden tasks masked by aggregate performance improvements.