How transformers work

A transformer is an architecture that represents text as tokens, mixes information with attention, and predicts the next token. 'Attention' means the model learns which earlier tokens matter for the current prediction — not that it is paying human attention.

Tokenize. Embed. Let each position look at others (attention). Push the mix through the same block many times. Decode. Context windows are the memory limit of that mix. Long documents fall off the edge; the model will still speak as if it read the part it dropped.

Give a short story, then ask which sentence a later pronoun refers to. Then add distracting sentences and see when the reference breaks. You just felt a context-window and attention failure without a lecture on matrices.

It is not a tiny person, a search index, or a guarantee of truth. Architecture explains capability and failure modes. It does not confer moral status or homework permission.