Source: LessWrong
Frontier labs are attempting to use their own latest models to audit and improve safety in the next generation—a recursive strategy that assumes each generation can reliably catch problems the previous one missed. But if frontier models themselves are unreliable evaluators (prone to hallucinations, inconsistent reasoning, missing context), this approach creates a closed loop where systematic blind spots propagate forward rather than get corrected. This means safety oversight is collapsing into self-assessment by the very entities with financial incentives to declare their products safe.