Generation visualised
How does a language model generate new text?
Language models do not write sentence by sentence - but token by token.
The animation shows how language emerges step by step from probability distributions: the model computes possible next tokens and keeps picking new words from them.
With the temperature slider you can then observe yourself how deterministic and creative generation differ.
Animation: autoregressive generation - how a language model produces text token by token
Step 1 · 0%
from prompt to next token
Step 1 - We start with a sentence
We start with a sentence and let the model write the continuation.
Temperature:
T = 0.8
Context so far
-
Next token
-
Probability
-