Source: LessWrong
A hands-on test of four AI coding agents with real constraints—$100 budget, concrete deliverable—found that none produced genuinely usable output, despite passing basic functionality checks. This matters because it exposes the gap between "works in isolated tasks" and "ships a real product." Current agents excel at scaffolding and boilerplate but fail at debugging, architectural decisions, and the grinding work of polish that separates demos from deployable tools. The limiting factor isn't raw capability but integration, judgment, and the messy reality that software quality compounds through iterative refinement—something today's agents still can't do autonomously without human intervention.