Simple Machine Learning
A field journal · v.01
← Back
FiledSeptember 16, 2026·6 min read·By The Editor

How LLMs Work (and How ChatGPT Works)

A large language model guesses the next word, then does it again. Everything else — training, hallucinations, context windows, ChatGPT itself — is detail hung on that one loop.

llmbeginnerintuition

The one-sentence version

A large language model is a machine that reads some text and guesses what word comes next. That is the entire job. To write you a paragraph, it does that a few hundred times in a row, each time gluing its own guess onto the end and starting over.

Figure 1 · the whole machine
How a language model produces a sentenceA loop: the prompt goes in, the model scores every possible next word, one is picked, it is appended to the text, and the whole thing runs again.your prompt“The cat sat on the”score everythingevery word gets a numberpick one“mat”glue it on the end and run the whole thing again“The cat sat on the mat” → now predict the next word after thatONE WORD AT A TIMEthere is no plan, no draft, no outline
A language model does exactly one thing: given some text, guess what comes next. To write a paragraph it does that a few hundred times in a row, feeding its own output back in. Everything else you have heard about these models is detail hung on this loop.

Everything else — the size, the training cost, the personality, the refusals, the hallucinations — is detail hung on that loop. If you hold onto one idea from this page, hold onto that one.

What "guessing the next word" actually means

Ask it to finish "The cat sat on the ___" and it does not retrieve an answer. It produces a number for every single word it knows.

Figure 2 · what a prediction looks like
Next-word probabilitiesA bar chart of candidate next words after 'The cat sat on the': mat 41 percent, floor 18, couch 11, table 7, roof 3, idea 0.4.“The cat sat on the ___”what the model actually produces: a number for every word it knowsmat41.0%floor18.0%couch11.0%table7.0%roof3.0%idea0.4%… and so on, for every one of ~100,000 tokensIt never knows the answer is “mat”. It knows “mat” is the safest bet.
This list of numbers is the model's entire output. It is not a sentence, not a thought, not a lookup. It is a probability for every word in its vocabulary, and the highest one is usually — but not always — the one you get.
In plain English

The model has no answer. It has odds. "Mat" scores highest because, across a very large amount of text, "mat" followed "the cat sat on the" more often than anything else did. That is not knowledge of cats. It is a statistical record of how people write.

This is why the same question can get different answers on different days. If the top word is 41% and second place is 18%, a model set to pick with a bit of randomness will sometimes take the 18%. Same model, same prompt, different sentence.

Where the numbers come from

The model contains billions of adjustable numbers, called parameters or weights. Training is the process of setting them, and it works exactly like the arithmetic in gradient descent by hand: show it text, let it guess, measure how wrong the guess was, nudge every number a hair in the direction that would have been less wrong. Repeat until the bill is enormous.

Figure 3 · two completely different activities
Training compared with using a modelTwo lanes. Training: text goes in, the model guesses, it is corrected, and its weights change — repeated trillions of times, once, in a data centre. Using it: text goes in, the model guesses, and the weights never change.TRAINING — happens once, costs millionsshow it textit guessescorrect itweights changerepeat a few trillion timesUSING IT — happens every time you type, costs a fraction of a centyou typeit guessesyou readnothing changes
Training and using are not the same machine doing two jobs. Training rewrites the model; using it only reads. The model you talk to is frozen. It cannot learn from your conversation, remember you tomorrow, or update itself with something you told it — and every claim to the contrary is a feature built around the model, not inside it.
In plain English

Training and chatting are not the same machine in two modes. Training writes; chatting only reads. The model you talk to is frozen — it cannot learn from your conversation or remember you tomorrow.

That last point causes more confusion than anything else on this page, so be blunt about it: when a chatbot appears to remember you, something outside the model saved your text and pasted it back in. The model itself did not change. It cannot.

Why it sounds like it understands

Because predicting the next word well requires an enormous amount of structure.

To reliably finish "The capital of France is ___" you have to have encoded something about France. To finish "She opened the door and ___" you need something about doors, rooms and narrative. To finish a line of Python you need the language's grammar. None of that was taught deliberately. It fell out of the pressure to guess well, across trillions of words.

So the honest answer to "does it understand?" is: it has built whatever internal machinery makes the guessing work, and that machinery is far richer than a lookup table and far stranger than a mind. Both "it's just autocomplete" and "it thinks like a person" are wrong in the same way — they pick a familiar thing and insist the model is that.

Why it makes things up

Look again at Figure 2. Every word gets a score. There is no score for "I don't know."

Ask for a court case that does not exist, and the model does what it always does: produces the most plausible continuation. Plausible court cases have names, dates, and a citation format. So you get one, formatted perfectly, entirely fictional.

In plain English

A hallucination is not a malfunction. It is the system working exactly as designed, on a question where working as designed produces a lie. The model was trained to sound right, never to check.

We wrote a whole piece on this: why LLMs hallucinate.

The four things that actually limit it

  • The context window. It can only look at so much text at once. Beyond that boundary, material does not exist — it is not forgotten, it was never there.
  • The training cutoff. Its numbers were frozen on a date. Anything after that it can only know if you paste it in.
  • No arithmetic engine. It predicts the text of a calculation. Often that lands on the right answer, because it has seen a lot of arithmetic. Sometimes it does not.
  • No self-knowledge. It cannot tell you why it said something. Ask, and it will generate a plausible-sounding reason — which is a fresh prediction, not a report.

How the pieces fit

The loop in Figure 1 is the whole system. The rest of the vocabulary is names for parts of it:

Token
A chunk of text — roughly ¾ of a word. The model reads and writes tokens, not letters.
Parameter
One of the billions of numbers inside the model. Set during training, frozen afterwards.
Training
The one-off process that sets those numbers. Costs millions of dollars.
Inference
What happens when you type. Reads the numbers, changes nothing.
Context window
How much text it can look at in one go. Everything outside it does not exist.
Temperature
How adventurous the word-picking is. Low = predictable, high = surprising.
Hallucination
A confident guess that happens to be false. The default failure mode.

So how does ChatGPT work?

ChatGPT is this loop plus product engineering. A system prompt is pasted invisibly above your message. Your recent turns are pasted in too, which is what makes it feel like a conversation. Sometimes a search tool runs and its results are pasted in as well. Then the model does the only thing it does: predicts the next token, a few hundred times.

Pull those wrappers off and you have a next-word guesser. Leave them on and you have something that feels like talking to someone. The gap between those two sentences is the entire product.

← Archive— end of essay —Front page →