Generation visualised
How does a language model generate new text?
Language models do not write sentence by sentence — but token by token.
The animation shows how language emerges step by step from probability distributions: the model computes possible next tokens and keeps picking new words from them.
With the temperature slider you can then observe yourself how deterministic and creative generation differ.
Animation: autoregressive generation — how a language model produces text token by token
Step 1 · 0%
from prompt to next token
Step 1 — We start with a sentence
We start with a sentence and let the model write the continuation.
Temperature:
T = 0.8
Context so far
—
Next token
—
Probability
—