// benchmark design

All signals tagged with this topic

AIs master deception in games that require strategic lying

Dwarkesh Patel's Blood on the Clocktower experiments reveal a blind spot in standard AI benchmarks. Models can now coordinate conspiracies, execute bluffs under pressure, and reason through social deception at scale—capabilities that traditional tests like MMLU or coding challenges don't measure. Social engineering, persuasion, and strategic misinformation are increasingly central to how AI systems interact with humans and institutions. An AI that excels at logic puzzles but falters at coordinated deception differs meaningfully from one that doesn't. The game mechanics also expose real limitations in current models' long-horizon planning and adaptive strategy, indicating the gap between narrow capability and embodied reasoning in adversarial environments remains substantial.