ManySpace
Türkçe Sign in Sign up

AI & MLRecommendedAIML-SYS-019

Speculative Decoding

A small draft model proposes tokens that the large model verifies in parallel, preserving the output distribution.

Difficulty
4 / 5
Importance
4 / 5
Study time
1–2 h
Kind of work
Mixed

Why it matters

The learner can vary the draft model's acceptance rate and predict the resulting speedup.

The route to this topic

Route AIML-SYS-019
What Is a Model? Tasks: Regression, Classification, and Beyond Features and Targets Probabilistic Prediction Classification Framing Linear Regression I: Prediction and Residuals Embeddings Tokens and Tokenization Scores and Logits Loss and Fitting: Squared Error Similarity in Vector Space Multiclass Scores and Softmax The Artificial Neuron Autoregression Objective Functions and Empirical Risk Query, Key, Value Activation Functions Next-Token Prediction Training vs Serving Similarity Scores The Multilayer Perceptron Temperature Weighted Aggregation Latency and Throughput Forward Propagation Top-k and Top-p Sampling Self-Attention Memory and Model Size Context and Conditioning The Attention Block The Causal Transformer Context Windows Caching and Reuse KV Caching and Batched LLM Serving Speculative Decoding