Transformers explained simply—and accurately

Most 'ELI5 transformer' posts either lie ('it thinks about the whole sentence') or drown you in matrices. You need a model accurate enough that the next article does not contradict it. Here is that model.

Text is chopped into tokens, not always into English words. Each token becomes a vector — a list of numbers that locates it among other tokens. Attention is a learned weighting: for this prediction, which earlier tokens should influence the mix? Stack those blocks and you get a transformer. The context window is how many tokens fit in that mix at once.

Attention is not understanding. Next-token prediction is not a search engine. Fine-tuning is not a moral education. If a system browses the web, that is a tool call sitting beside the transformer, not magic emerging from the weights.

Explain to a Builder-age learner why a model can lose the plot in a long chat. If you said 'it got tired,' try again. If you said 'the early tokens fell out of the window and attention has nothing honest to weight,' you are ready for the how-AI-is-built pillar.