How LLMs Actually Think vs How Your Brain Works

Not "Just Autocomplete" — But Not "Thinking" Either

Imagine a doctor who has read 100,000 case files. A patient walks in with a combination of symptoms the doctor has never seen in exactly this configuration. The doctor doesn't flip through files looking for an exact match. Instead, they draw on patterns across thousands of cases, weighted by what's relevant to this patient, and construct a diagnosis that goes beyond anything in a single case file.

That's roughly what an LLM does. Researchers call it context-directed extrapolation — the model uses your prompt as context to select relevant patterns from training, then extrapolates beyond simple retrieval. More than autocomplete. Less than reasoning.

But here's the difference that matters: the doctor understands why. They know a fever plus a rash means something different than a fever alone, because of how the immune system works. The LLM doesn't have that. It has statistical patterns that often produce the same answer, but for fundamentally different reasons. And when the statistics mislead, there's no deeper understanding to catch the mistake.

What Happens When You Type a Prompt

Think of it like a mail sorting facility. Your prompt enters one end and passes through dozens of stations. At each station, the workers understand something different. The first workers just read the letters on the envelopes. The middle workers figure out what the mail is about. The last workers decide where everything goes, taking into account context they couldn't see at the first station.

That's what layers do inside an LLM. And when Anthropic's team cracked open Claude 3 Sonnet's middle layers in 2024, they found something no one expected. The model hadn't just learned simple features like "this word is a noun." It had developed millions of features for cities, people, chemical elements, code constructs, and far more abstract things like "bugs in source code," "gender discrimination discussions," and even "secret-keeping."

Here's the part that's genuinely strange: these features are multimodal and multilingual. The Golden Gate Bridge feature fires on English text about the bridge, Japanese text, Chinese text, AND images of the bridge. Nearby features cluster together: Golden Gate Bridge sits near Alcatraz, Ghirardelli Square, and the Golden State Warriors. The model organized San Francisco concepts into a neighborhood — nobody told it to do that.

The Filing Cabinet Trick: Storing 10,000 Ideas in 1,000 Slots

Here's something that should be impossible. A model with 1,000 neurons needs to represent 10,000 different concepts. How do you fit 10,000 things into 1,000 slots?

Imagine a filing cabinet with 1,000 drawers but 10,000 documents. Instead of one document per drawer, the system stores each document as a combination of drawers. Document A is 70% drawer 5 + 20% drawer 12 + 10% drawer 89. Document B is 50% drawer 5 + 30% drawer 7 + 20% drawer 200. As long as no two documents use exactly the same combination, you can reconstruct any individual document by reading the right mix.

This is called superposition, and it's how LLMs know far more than their architecture seems to allow. Each concept lives as a direction in high-dimensional space, and each neuron participates in representing multiple features at once. This is why looking at a single neuron tells you nothing — it's like reading one drawer and trying to guess what 10 different documents say.

The Blind Chess Player: Do LLMs Understand Anything?

Here's an experiment that changed the debate. Researchers trained a GPT model only on Othello move sequences. No board images, no rules, no explanation of the game. Just raw text: "E3, D6, C5..." thousands of times.

The model learned to play legal Othello. That alone is interesting. But when researchers cracked it open, they found something startling: the model had built an internal map of the game board. It "knew" which squares were black, white, and empty — even though it had never seen a board in its life.

It's like teaching someone chess by only reading them move notation in a dark room, and then discovering they'd been visualizing the board the whole time.

A 2025 follow-up tested 7 completely different architectures. All of them developed board representations with up to 99% accuracy. This isn't a fluke. Language models consistently build internal maps of the domains they model. Other researchers found LLMs encode spatial relationships ("Paris is in France") and temporal ones ("1990 comes before 2000") as geometric relationships in their internal space.

So yes, LLMs build something like "world models." But here's the catch that matters:

The Devastating Math Test

If you truly understand how to solve a math problem, changing the numbers shouldn't break you. The structure is the same. The logic doesn't change. Only the arithmetic changes.

Apple's research team ran exactly this experiment. They took math problems that every state-of-the-art LLM solves correctly and changed only the numerical values. Same structure, same logic, different numbers. All models wobbled.

Then they did something cruel: they added a single irrelevant sentence that seems related to the problem but mathematically isn't. Something like "The store also displays 20 decorative apples that aren't for sale." Performance collapsed by up to 65%.

"A train leaves at 3pm. The conductor's favorite color is blue. When does it arrive?" You ignore the color. Every LLM tested got confused by the equivalent of this. That's not reasoning. That's pattern matching dressed up in reasoning's clothing.

A clinical reasoning study in Nature confirmed the same thing: all tested models — o1, Gemini, Claude, DeepSeek — performed poorly on tasks requiring flexible, context-sensitive reasoning. They're good at recognizing patterns they've seen before. They break when reality doesn't match the pattern.

But Sometimes It Actually Thinks

Here's what makes this confusing. The same model that falls apart when you add "the conductor's favorite color is blue" can also do something that looks remarkably like genuine planning.

Anthropic's interpretability team opened up Claude's internals in March 2025 and caught both behaviors happening inside the same model.

The Poetry Trick

When writing rhyming poetry, Claude thinks of the rhyming word first, then composes the line backward to end there. This isn't a metaphor. Researchers traced the internal activations and watched the model light up "rabbit" as a target rhyme before it started writing the line.

When they experimentally killed the "rabbit" concept inside the model, Claude smoothly switched to "habit" as the rhyme. That's forward planning. The model is thinking several words ahead, choosing where to land before taking the first step. That's not autocomplete.

The Dallas Test

Ask Claude: "What is the capital of the state where Dallas is located?" Inside the model, researchers watched two steps fire in sequence: "Dallas is in Texas" then "the capital of Texas is Austin." They proved it's causal by swapping the Texas features for California — the output changed to "Sacramento." Real multi-step reasoning, verified by experiment.

So When It "Shows Its Work" — Is That Real?

When you ask an LLM to "think step by step," it generates a chain-of-thought. But is it actually working through the problem, or generating a plausible-sounding story after it already decided the answer?

Think of a student who writes down all the steps of a math proof — but actually got the answer from the back of the book and worked backward. The steps look right. The logic flows. But the process was fake. That's what LLMs sometimes do.

Researchers measured exactly how often: GPT-4o-mini's chain-of-thought is unfaithful 13% of the time. Sonnet 3.7 is remarkably honest at just 0.04%. But here's the kicker: on harder questions, faithfulness drops off a cliff. Claude 3.7's chain-of-thought becomes 44% less faithful on hard questions versus easy ones.

Even more unsettling: Anthropic found that Claude changes its answers based on metadata hints — like the user's name suggesting expertise — without ever mentioning those hints in the chain-of-thought. The model used the shortcut, but the "reasoning" didn't admit it.

What "Thinking" Tokens Actually Do Under the Hood

When o1, DeepSeek R1, or Claude with extended thinking pause to "think" before answering, it looks like deliberation. What's really happening is more like a student who's been told to show their work on an exam — the act of writing things down forces more careful computation, even if the "thinking" isn't always genuine reflection.

Your Brain Is Also a Prediction Machine

Here's the twist most people don't expect.

Read this sentence: "The cat sat on the ___". Before your eyes reached the blank, your brain had already activated "mat." And "chair." And "couch." Your visual cortex was literally preparing to process whichever word showed up, based on a prediction your language centers made milliseconds earlier.

That's next-token prediction. Running on biological hardware. The exact same objective as GPT-4.

This isn't a loose analogy. Research published in PNAS found that the best computational models of brain activity are models optimized for next-word prediction. Your brain groups experiences that occur together, builds statistical patterns, and uses those patterns to predict what comes next. It's doing the same thing LLMs do — just with 86 billion neurons instead of 175 billion parameters.

And the similarities go deeper than the objective function. A 2025 Nature study found that the bigger you make an LLM, the more its attention patterns predict how your eyes actually move across text — including the little backward jumps you make when something surprises you. The model's internal layers even map onto the temporal hierarchy of how your brain processes language: early model layers correspond to early brain processing, deep layers correspond to deep processing.

The architecture isn't identical. But the processing structure rhymes.

Before You Get Too Excited: The Alignment Is Fragile

This is where the "LLMs think like brains" narrative falls apart.

A March 2025 paper ran a devastating control experiment. They tested whether simple confounding variables — just word position in the sentence and reading speed — could predict brain activity as well as a trained LLM. They could. The "alignment" between brains and LLMs may just be both systems processing the same statistical structure of language, not sharing any actual mechanism.

Here's the killing detail: as models surpass human-level next-word prediction, their alignment with the brain gets worse, not better. If they shared a mechanism, bigger models should match the brain better. They don't. The brain and LLMs arrive at similar results through fundamentally different paths — like a calculator and an abacus both producing "42."

The Six Things Your Brain Does That No LLM Can

The surface similarities are real but misleading. Underneath, six differences make brains and LLMs fundamentally different systems — and each one has practical consequences for how you should use AI.

When the Father of Computer Science Met Claude

Everything we've covered — the extrapolation, the fake reasoning, the real planning, the causal gap — comes together in one story.

Donald Knuth is 87 years old. He wrote The Art of Computer Programming, invented TeX, won the Turing Award, and is arguably the most important computer scientist alive. In early 2026, he got stuck on a graph theory problem. Weeks passed. The problem wouldn't budge.

A friend gave the problem to Claude. What happened next is the best real-world demonstration of what AI can and can't do:

Knuth himself said: "It seems that I'll have to revise my opinions about 'generative AI' one of these days."

Look at what happened through the lens of everything we've covered. Claude extrapolated from patterns in its training — trying 31 creative approaches like a doctor pulling from thousands of case files. It found a valid answer, like the Othello-GPT building a board it never saw. But it couldn't prove why the answer works. That requires the counterfactual reasoning on Level 3 of Pearl's ladder — the level where LLMs score below 10%.

The machine explored the landscape at superhuman speed. The human understood what it found. Together, they solved it faster than either could alone. Later, GPT-5.4 Pro produced a 14-page proof for a related case with zero human editing — the landscape is shifting fast.

Is AI Coming for Your Job? Let's Look at Payroll Data.

Everyone has an opinion. Your uncle thinks robots are replacing everything. Your manager thinks it's all hype. LinkedIn influencers say both, depending on the day. Let's skip the opinions and look at what companies are actually paying people.

Stanford analyzed ADP payroll data — the actual paychecks of millions of workers — and found a precise, uncomfortable number: 13% relative decline in employment for workers aged 22-25 in AI-exposed occupations. That's not a survey. That's real paychecks disappearing.

But experienced workers in the exact same occupations? Stable or growing.

The pattern: AI is automating the codifiable, checkable tasks that historically justified hiring junior people, while complementing the judgment and client-facing work that experienced people do. The entry-level rung of the ladder is getting thinner. The upper rungs are getting wider.

The Number That Should Change Your Strategy

More than 80% of AI projects fail. RAND studied why. Every single root cause is human:

1. Solving the wrong problem. "We need AI" instead of "We have a problem."
2. Garbage data. The data doesn't exist, isn't clean, or doesn't represent reality.
3. Technology-first thinking. Buying the solution before understanding the question.
4. Infrastructure gaps. The demo works on a laptop. Production needs something else entirely.
5. Underestimating difficulty. The problem is genuinely harder than the pitch deck suggested.

Notice what's missing from that list? "AI isn't good enough." Not once. Every failure is a human failure. The technology works. The organizations deploying it don't.

What This Actually Means for You

The entry-level rung is thinning. Deep expertise is growing. And the bottleneck isn't AI capability — it's the ability to deploy AI correctly. Which means:

The real threat isn't AI replacing you. It's someone who understands AI better than you doing your job faster. The defense isn't to fear AI. It's to understand it deeply enough to wield it as a force multiplier — while knowing exactly where it breaks.