The single most consequential misunderstanding about large language models is that they "know" things the way a search engine or a database does. They don't. Understanding what they actually do — predict the next most statistically likely word, over and over — explains almost every strange thing they do: the confidently wrong answers, the inconsistency between two nearly identical questions, and why a five-word prompt and a five-paragraph prompt sometimes land you in the same place.
It's autocomplete, just much bigger
You've used a weaker version of this technology already: the predictive text on your phone that suggests the next word as you type. Type "The capital of France is" and it'll probably suggest "Paris" — not because your phone "knows" geography, but because that word follows that phrase constantly in the text it learned from. A large language model does the same thing, at a scale that's hard to compare to anything else: trained on a huge slice of publicly available text, and built to weigh not just the last word but the relevance of everything you've written so far, near or far, when predicting what comes next.
That's it. That's the mechanism. There's no lookup happening, no database being queried, no fact-checking step. Every response is a very sophisticated act of "what word most plausibly comes next, given everything so far."
Why this explains the weird failures
Ask an LLM about a well-documented topic — a major historical event, a mainstream programming concept, how photosynthesis works — and it's usually right, because the plausible continuation and the correct continuation are almost always the same thing when a topic is heavily represented in its training data. Ask it about a specific court case, an obscure historical date, or a niche academic paper, and the failure mode is different: it doesn't say "I don't have enough information." It produces the most plausible-sounding continuation anyway — a real-sounding case name, a specific-sounding date — because "plausible-sounding" is the only thing it was ever built to produce. It isn't lying, because lying requires knowing the truth and saying something else. It's doing exactly what it does everywhere else; it's just that here, plausible and correct have come apart.
Why it still feels intelligent
This isn't a reason to dismiss these tools — the same mechanism that produces confident fabrications also produces genuinely useful writing, explanation, and reasoning, because human-written text is full of examples of good reasoning, clear explanation, and careful argument, and predicting "what does a careful explanation of this look like" often produces one. The trick is knowing which mode you're in: for topics with deep, unambiguous representation in general knowledge, prediction and correctness line up well. For anything specific, recent, numeric, or obscure, they don't.
The mental model that actually helps
Stop picturing a librarian who looks things up. Picture a writer who has read an almost unimaginable amount and produces, instantly, the response that sounds most like what a knowledgeable person would say — without any step that confirms whether it's actually true. That model tells you exactly when to trust it at face value (drafting, brainstorming, rephrasing, explaining a well-known concept) and exactly when to double-check before you rely on it (citations, dates, statistics, anything where "sounds right" and "is right" might not be the same sentence).