> ## Content Index
> Fetch the complete content index at: https://adjacent.media/llms.txt
> Use this file to discover other available public pages before exploring further.

# OpenAI's AGI Benchmark Score Depends Entirely on the Testing Harness
- URL: https://adjacent.media/signals/openais-agi-benchmark-score-depends-entirely-on-the-testing-harness/
- Published: 2026-09-07T16:14:10.000Z
- Updated: 2026-09-07T16:14:10.000Z
- Description: OpenAI’s claim that o1 achieved 99.9% on an AGI capability benchmark collapses when run through the benchmark’s own official software—dropping to 62.7%—exposing how evaluation methodology, not model capability, manufactured the headline.
- Author: Jonathan Greene
- Tags: #signal, theme-ai, model evaluation, capability claims, benchmark integrity

Source: [The Next Web](https://thenextweb.com/news/openai-astra-arc-agi-3-harness-62-7-vs-99-9-benchmark-revisions?ref=adjacent.media)

OpenAI's claim that o1 achieved 99.9% on an AGI capability benchmark collapses when run through the benchmark's own official software—dropping to 62.7%—exposing how evaluation methodology, not model capability, manufactured the headline. This reveals a structural problem in AI capability claims: the gap between marketing narrative and reproducible measurement is now wide enough that "AGI" has become whatever testing apparatus you bolt onto the model. The stunt undermines legitimate discussions about o1's genuine advances in reasoning while establishing a precedent where frontier labs control both the models and the evaluation criteria.