Gradient Descent Playground
One walker, one bowl, one knob that matters. Find the step size that flies — and the one that explodes.
Tip: try lr = 0.5 — it lands exactly on the minimum in a single step. Then try 1.1 and watch it blow up.
Try this
- Crawl, then fly. Walk from x₀ = 0 with lr = 0.1 and count the steps home. Now restart with lr = 0.9. Same bottom, a fraction of the steps — step size is travel time.
- Find the magic step. Try lr = 0.5 from any start: one step, exact minimum. Then nudge to 0.6 and watch it overshoot-and-return. There is always a “just right” — and it is never the biggest stable step.
- Blow it up on purpose. Set lr = 1.1 and press “Run 100”. The dots flee the bowl and the DIVERGED banner earns its red. Every training run you have ever heard “explode” died exactly this death.
How to read a descent
Gradient descent is the engine inside nearly every trained model: stand somewhere on an error curve, measure the slope, and step downhill in proportion to it. The rule is one line — x ← x − lr·f'(x) — and the learning rate lr is the whole game. Small lr tiptoes safely but crawls; large lr leaps but risks overshooting the bottom entirely.
On this bowl (f(x) = (x−3)² + 2) the math is exact: lr below 1 always converges, lr = 0.5 lands on the minimum in one step, and lr above 1 diverges — each step leaps clean over the bottom to a worse spot, forever. Real models live in million-dimensional bowls, but the step-size bargain is identical: the most important knob in machine learning, felt in under a minute.
Optimization clicks once you have tuned it by hand
This playground is one small piece of how ManySpace teaches machine learning — every idea as something you can run, break, and tune back to life.