Source: LessWrong
The piece argues that the most consequential AI capabilities emerge from internal representations that frontier models build—the patterns they extract that produce rare moments of genuine surprise or breakthrough insight. This reframes AI progress: not as a linear climb on published leaderboards, but as an increasingly opaque process where the gap between what models can do internally versus what they're permitted to demonstrate externally becomes the real competitive frontier. The result is uncomfortable: we may have poor visibility into actual capabilities, which makes both safety assessment and competitive positioning harder than public metrics suggest.