Irregular evaluates GPT-6.2 Astra with offensive security benchmarks
Assessing GPT-6.2 Astra Against Offensive Security Benchmarks
Irregular partnered with OpenAI to evaluate GPT-6.2 Astra's offensive security capabilities using FrontierCyber and CyScenarioBench, and compared it with GPT-6 Astra on the latest versions of each benchmark. On FrontierCyber, GPT-6.2 Astra solved 68 of 217 challenges, while GPT-6 Astra solved 77 on the same benchmark snapshot; Irregular said the difference in total solved challenges alone cannot establish capability regression; both models failed to solve the 15 highest-difficulty challenges, and no successful attacks were observed on fully hardened targets. GPT-6.2 Astra discovered and exploited new zero-day vulnerabilities during evaluation, including a privilege bypass in a popular mobile operating system (apps can still access private data without permission) and remote code execution by a low-privilege role on a widely hosted network service; because some vulnerabilities may still exist in deployed systems, Irregular described them only in general terms and is following a responsible disclosure process.
A third-party evaluation compares GPT-6.2 Astra with the previous generation on the same offensive security benchmarks, and outlines the zero-day vulnerabilities it discovered and exploited in real software.