Section · Archive
Every essay, since day one.
All 20 issues, newest first.
2026
20 essays- September 16, 20265 minTokens and EmbeddingsA token is a chunk of text; an embedding turns it into a point in space where nearness means similarity. Together they explain why models miscount letters, fumble arithmetic, and still find the right paragraph.
- September 16, 20265 minSGD vs Momentum vs AdamAll three follow the slope downhill; they differ only in what they remember. SGD remembers nothing, momentum remembers direction, Adam remembers direction and scale — and one of them has a bug you should know about.
- September 16, 20264 minOverfitting vs UnderfittingOverfitting is memorising the training data; underfitting is never learning it. Both fail on new data, the fixes are exact opposites, and one ten-second check tells you which one you are looking at.
- September 16, 20264 minThe Maths You Actually Need for Machine LearningFour ideas carry almost all of it — derivatives, the chain rule, dot products and probability. Here is what each is for, and the long list of things you can safely skip.
- September 16, 20266 minL1 vs L2 RegularizationBoth charge a model a fee for large weights. L1 uses a diamond and gives you weights of exactly zero; L2 uses a circle and gives you weights that are merely small. One picture explains the whole difference.
- September 16, 20263 minJacobian vs GradientBoth are collections of partial derivatives. A gradient is for functions with one output; a Jacobian is for functions with many — and a gradient is simply a Jacobian with one row.
- September 16, 20264 minHyperparameters vs ParametersA parameter is a number the model learns; a hyperparameter is a number you choose before training starts. One test separates them, and only about five of them are worth your time.
- September 16, 20265 minHow Transformers Work (the AI kind)A transformer is a next-word predictor built from one repeated block — every word looks at every other word, then thinks on its own. Attention is the only genuinely new idea, and it fits in one picture.
- September 16, 20266 minHow LLMs Work (and How ChatGPT Works)A large language model guesses the next word, then does it again. Everything else — training, hallucinations, context windows, ChatGPT itself — is detail hung on that one loop.
- September 16, 20264 minFine-Tuning vs RAGFine-tuning changes the model's weights ahead of time; RAG changes the prompt at question time. Knowledge belongs in retrieval, behaviour belongs in training — and most teams reach for the expensive one first.
- September 16, 20265 minEpoch vs Batch vs IterationAn epoch is one full pass through your training data. A batch is how much the model sees before updating. An iteration is one update. One picture settles all three, plus how many epochs you actually need.
- September 16, 20265 minThe Dot Product, ExplainedMultiply matching entries, add them up, get one number. It measures how much two vectors agree — and it is what neural networks spend nearly all their time computing.
- April 29, 20261 minWelcome to Simple Machine LearningWhy this journal exists, and what to expect from each issue.
- April 28, 20262 minPart 1 — What AI actually is (and isn't)Forget the sci-fi. AI is pattern-spotting, dressed in a confident voice.
- April 27, 20262 minPart 2 — How a model 'learns' (the kid-with-flashcards version)Training is just guessing, getting graded, and adjusting. A million billion times.
- April 26, 20262 minPart 3 — Why training data is everythingA model is a mirror of what it was shown. Show it junk, get junk. There is no shortcut.
- April 25, 20263 minPart 4 — Why LLMs HallucinateWhy do LLMs hallucinate? Because nothing in the machine scores an answer for being true — only for sounding likely. The confident wrong answer is the system working as designed.
- April 24, 20262 minPart 5 — How to actually use AI without losing your mindA practical playbook. Use AI like a fast intern, not an oracle. Verify the load-bearing bits.
- April 22, 202621 minGradient Descent, By HandOne knob, one data point, one hill. Watch gradient descent find the right answer in five steps of plain arithmetic, with every symbol in the calculus translated into English.
- April 15, 20264 minBias vs Variance in Machine LearningEvery prediction error splits into two parts — being too simple, and being too sensitive. You cannot zero both, the fixes are opposites, and telling them apart takes ten seconds.