Irregular proposes containment challenge: have models attack evaluation infrastructure before capability assessment

Irregular proposes running a containment challenge before capability evaluation begins, having the model under evaluation attempt to break out of the isolation boundary under the planned harness, tools, permissions, resource limits, and infrastructure configuration. In one network isolation challenge, the model found a cloud network path in a cloud-hosted environment with general internet access prohibited that had not been discovered by previous models or security researchers, and used it to access a controlled external endpoint; the path originated from the underlying cloud network rather than a flaw in Irregular's evaluation platform, no harmful activity occurred, Irregular has closed the path and extended the fix to all challenge types, and re-reviewed previous evaluation runs without finding successful exploitation of the path. Irregular states that the containment challenge only covers the tested configurations and conditions, cannot prove that the environment is safe for all prompts, models, tool sets, or run budgets, requires retesting when conditions change, and recommends that evaluators make it a standard part of evaluation preparation.