Source: Fortune
OpenAI's post-hoc revisions to evaluation standards for its flagship model undermine the credibility of comparative AI benchmarks when governments and enterprises need reliable performance data for deployment decisions. The pattern of metric adjustment after release—rather than pre-registration—mirrors gaming dynamics in academic publishing and creates incentives for labs to optimize benchmark design around their own capabilities rather than genuine capability measurement. Benchmark gaming erodes the technical consensus required to evaluate real safety tradeoffs between models.