AI & ML
Sequence Modeling
Part of Deep Learning, level 3. About 8 hours of study across the topics below.
Sequences
- Sequences and OrderTime and order as structure: variable length, position, and why shuffling breaks meaning.1–2 h
- AutoregressionPredict the next from the past: the task behind sequences, time series, and language models.1–2 h
Recurrence
- The RNN ConceptRecurrence with shared weights: one cell, unfolded in time.1–2 h
- Hidden StateMemory as a vector: the state carries the past into the present step.1–2 h
- Vanishing Gradients in TimeLong-range amnesia: why distant past fades from the gradient.1–2 h
- LSTM and GRU: The Gating IdeaGates guard memory: the conceptual role of gated recurrence, not cell internals.1–2 h
- Sequence-to-Sequence ModelsAn encoder compresses an input sequence, a decoder generates an output sequence: the bottleneck attention was invented to fix.1–2 h
- State Space Models for SequencesLinear recurrences with structured, sometimes input-dependent, state updates: linear-time alternatives to attention.1–2 h