Source: X
Daniel Kokotajlo's warning cuts deeper than typical AI capability talk. He's identifying a structural problem: the rate of model advancement has begun to exceed humans' ability to meaningfully audit or understand them, creating an asymmetry that matters for safety and governance. The second-order risk he's flagging is feedback-loop dependency. As researchers outsource more analysis to AI systems themselves, institutional knowledge about *why* models behave certain ways erodes, leaving fewer experts capable of catching genuine failures before deployment. This is a near-term coordination problem where the people building the most powerful models are admitting they've partially lost visibility into what they've built.