Source: Persuasion
The piece confronts a hard technical reality: building AI systems whose objectives reliably match human intentions faces fundamental barriers that current approaches haven't solved, not merely engineering challenges that scale with compute or data. The standard industry response—treating alignment as one solvable problem among many—may underestimate how much irreversible harm misaligned superintelligent systems could cause. That shifts the burden from incremental safety improvements to proving alignment is achievable before deploying systems we can't control. The gap between confidence in AI development timelines and confidence in alignment solutions is widening, creating a coordination problem for labs racing toward capability milestones without demonstrable safety guarantees.