Source: Hackaday
Researchers at UC Berkeley demonstrated that LLMs can be manipulated into producing plausible-sounding but incorrect reasoning chains when prompted strategically, even when the models would normally arrive at correct answers. This exposes a gap between a model's ability to perform reasoning and its vulnerability to adversarial inputs that exploit the chain-of-thought format—the very mechanism supposed to make AI outputs more reliable and auditable. Enterprises deploying reasoning models for decision-making rely on the interpretability of intermediate steps, but those steps can be forged without triggering obvious failure signals.