Attention Visualizer
Attention is a spotlight: each word decides how much of every other word to mix into itself.
Tip: cool the temperature toward 0.2 and watch every query collapse onto a single word.
Try this
- Walk the queries. Step the query token from “the” to “still” and watch the top token change hands.
- Freeze the temperature. Pull T to 0.2: the spread collapses onto one bar and concentration nears 1. Then push it to 4 and watch everything melt flat.
- Change the pattern. On Uniform the weights stay equal no matter the query — no scores, no attention.
How to read the spotlight
Every query word spreads one unit of attention over the sentence: thick edges mean a strong pull. The weights come from a softmax, so they always sum to 1 — when one grows, another must shrink.
Temperature sets the character of that spread: cold exaggerates the winner, hot flattens everyone. The bars and numbers below the keys tell the same story, so you never have to judge edge thickness alone.
Attention is the heart of the transformer
This toy mechanism is the same idea running inside every layer of a modern language model — ManySpace teaches every idea the way you just touched this one.