Source: Marcus on AI
LLMs generate confident-sounding but medically incorrect information, creating real liability risks as patients increasingly turn to AI for preliminary health guidance. The core problem isn't knowledge gaps—it's that these systems have no mechanism to express uncertainty or refuse questions outside their competence. In healthcare, false confidence compounds harm. Systems adopting chatbot triage without human verification checkpoints are outsourcing diagnostic gatekeeping to technology that cannot distinguish between plausible-sounding fabrication and fact.