AI & MLCoreAIML-LM-004
The Language-Modeling Objective
Cross-entropy on text: the loss that trains next-token predictors, plus perplexity.
- Difficulty
- 2 / 5
- Importance
- 5 / 5
- Study time
- 1–2 h
- Kind of work
- Mixed
Why it matters
The learner can compute the LM loss for a tiny predicted sequence and read perplexity as average branching.