Transformer — next-token prediction

Clicked card's query: arcs glow to keys it attends to (real softmax)Each punched card carries a word and its 4-D embeddingResidual bypass carries the input around each layerA transformer decoder block as a workshop rig. A punch press tokenises words into punched cards; positional gears number them along the belt; a lift pipe carries them up through stacked layers, each with three multi-head self-attention panels, a feed-forward roller and a residual bypass rail; and an output roulette wheel chooses the next token by probability. Click any card to focus the query; the model is a real tiny transformer (4-D embeddings, three heads with their own Wq/Wk matrices and softmax), and the attention arcs and pointer update live.