Skip to content
TechTimes·Indexed by this site · Originally published· Evaluation →· Its place in the scan record →SelectedAttention80

OpenAI rates GPT-6.2 Astra's self-improvement capability as High; two second-stage tests did not meet the threshold

OpenAI Rates GPT-6.2 Astra High for Self-Improvement Though Two Second-Stage Tests Miss Cutoffs

AI summary

In the GPT-6.2 system card, OpenAI rates Astra's self-improvement capability as High, saying it is the first externally deployed model to reach that level, but two of five research engineering evaluations' second-stage tests did not meet the thresholds the card itself sets: NanoGPT at 42.87% (threshold 80%) and PostTrainBench Lite at 74.79% (threshold 90%), with only MLE-Bench Revised crossing the 90th percentile line at 96.06. The card does not explain how the five results are weighted, and the two first-stage tests also give no numerical thresholds. External evaluations point in different directions: Epoch AI says Astra outperforms GPT-6 Astra on all unsaturated benchmarks, OECI is 174 versus 169, and FrontierMath solved 7/68 Erdős problems; Vals AI gives 29.01% on a modified RSI index and considers it unlikely to fully automate AI research.

Why it was selected

Two of the five evaluations the system card lists itself fall below the thresholds it sets itself, from which readers can see that the High rating rests on a weighted judgment rather than on all items meeting the bar.

Read originaltechtimes.com