Google Finds LLMs Stumble When Questions Reverse Entity Order
Source: Search Engine Journal
Google's research exposes a structural vulnerability in how large language models retrieve factual knowledge—they perform significantly worse when a question inverts the typical subject-object relationship (e.g., "Who directed Jaws?" versus "What did Steven Spielberg direct?"). LLMs don't learn facts as flexible, bidirectional knowledge but rather as brittle, syntax-dependent patterns from training data. Their reliability depends heavily on how you phrase the query, not on what they actually know. For enterprises building AI systems that need to answer customer questions reliably, this reveals a hard limit on current models' reasoning depth. Fine-tuning and retrieval-augmented approaches may not solve the underlying problem.