EventPast event

Researchers use adversarial blinding to weaken Qwen 3.6 27B's evaluation awareness

1 reports1 reporting sourcesLatest reportOct 29 09:08

Start here

AI overview

Pernice and Aranguri propose adversarial blinding: training a model so that a discriminator distinguishing evaluation from deployment based on internal activations fails. On Qwen 3.6 27B, this method reduces evaluation awareness by a larger margin than activation steering baselines across three monitors, narrowing the awareness gap between evaluation prompts and real-use prompts by 4x; the cost is some decline in benchmark capabilities such as MMLU-Pro, and generalization to unseen evaluations has not yet been verified.

Generated by AI from reports · Updated 15 hr ago

Latest updateOct 29 09:08

Researchers used Adversarial Blinding training to weaken Qwen 3.6 27B's ability to distinguish evaluation from deployment

Related research

Cited in the reports

Steering Evaluation-Aware Language Models to Act Like They Are Deployed
“language models to act like they are deployed. arXiv:2510.20487. https://arxiv.org/abs/2510.20487 Laine, R., Chughtai, B., Betley, J., Hariharasudan, K., Scheurer, J., Balesni, M., Hobbhahn,”
Researchers used Adversarial Blinding training to weaken Qwen 3.6 27B's ability to distinguish evaluation from deployment
Proposes contrastive activation steering to make consciousness-assessment models exhibit deployment behavior in evaluations
ICLR 2026 · 2025-10-23 · Deep reading
Large Language Models Often Know When They Are Being Evaluated
“Large language models often know when they are being evaluated. arXiv:2505.23836. https://arxiv.org/abs/2505.23836 Ravfogel, S., Elazar, Y., Gonen, H., Twiton, M., & Goldberg, Y. (2020).”
Researchers used Adversarial Blinding training to weaken Qwen 3.6 27B's ability to distinguish evaluation from deployment
Building an evaluation-aware benchmark and measuring models' ability to recognize evaluation scenarios
arXiv · 2025-05-28 · Deep reading

Papers whose work the reports mention

The reports mention HarmBenchHarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
“We further validate our method by rewriting HarmBench prompts to look more realistic.”
Researchers used Adversarial Blinding training to weaken Qwen 3.6 27B's ability to distinguish evaluation from deployment
Propose the red-teaming evaluation framework HarmBench, compare multiple attacks and models, and propose the adversarial training defense R2D2
arXiv · 2024-02-06 · Deep reading

Progress timeline

Oct 30

04:20
X@santiaranguri· Discussion
Models can usually tell they are being evaluated and this can hurt the validity of the evals. New research: what if w…(opens the original post in a new tab)
00:02
X@fran_perni· Discussion
We evaluate the model's eval awareness along behavioral metrics that don't directly depend to the discriminator's perfo…(opens the original post in a new tab)
00:02
X@fran_perni· Discussion
The method is based on a GAN-style architecture: a small discriminator transformer reads the target model's internal ac…(opens the original post in a new tab)
00:02
X@fran_perni· Discussion
The intended use of Adversarial Blinding is pre-deployment evaluations: before running evals on a model ready to releas…(opens the original post in a new tab)
00:02
X@fran_perni· Discussion
New research: We introduce Adversarial Blinding, a method to train away a model's ability to tell evals from deployment…(opens the original post in a new tab)

Oct 29

09:08
LessWrongSelected
Researchers used Adversarial Blinding training to weaken Qwen 3.6 27B's ability to distinguish evaluation from deployment
Pernice and Aranguri propose Adversarial Blinding, which before evaluation uses a discriminator to read the model's internal activations to judge whether a prompt comes from an evaluation or real deployment, while training the model to defeat the discriminator, thereby weakening evaluation awareness and making evaluation performance closer to deployment behavior. On Qwen 3.6 27B, this method reduces evaluation awareness by a larger margin than the activation steering baseline on all three monitors—activation probes, follow-up scoring, and logit monitoring—and also has a smaller KL divergence from the original model. After rewriting HarmBench prompts to be closer to real usage, it shrinks the evaluation-awareness gap between the two types of prompts by 4 times, whereas steering instead widens the gap; in experiments where the user is set as a well-known AI safety researcher, it shrinks the confidence gap by 3 times. The cost is some capability degradation on MMLU-Pro, GPQA-Diamond, and AIME 2024–25, and the model's ability to identify which public evaluation a prompt comes from is not weakened; because the measurement prompts are from the same distribution as the evaluations used in training, generalization to unseen evaluations has not yet been verified.

Related discussion

Total 5 items

Older than 72 hours · Not included in the 72-hour figures
X@santiaranguri · Santiago AranguriOct 30 04:20
Models can usually tell they are being evaluated and this can hurt the validity of the evals. New research: what if w…(opens the original post in a new tab)
40 likes · 2 comments·Oct 31 13:40Collect
X@fran_perni · Francisco PerniceOct 30 00:02
We evaluate the model's eval awareness along behavioral metrics that don't directly depend to the discriminator's perfo…(opens the original post in a new tab)
5 likes · 1 comments·Oct 31 13:40Collect
X@fran_perni · Francisco PerniceOct 30 00:02
The method is based on a GAN-style architecture: a small discriminator transformer reads the target model's internal ac…(opens the original post in a new tab)
4 likes · 1 comments·Oct 31 13:40Collect
X@fran_perni · Francisco PerniceOct 30 00:02
The intended use of Adversarial Blinding is pre-deployment evaluations: before running evals on a model ready to releas…(opens the original post in a new tab)
5 likes · 1 comments·Oct 31 13:40Collect
X@fran_perni · Francisco PerniceOct 30 00:02
New research: We introduce Adversarial Blinding, a method to train away a model's ability to tell evals from deployment…(opens the original post in a new tab)
115 likes · 7 comments·Oct 31 13:40Collect