ManySpace
Türkçe Sign in Sign up

AI & MLRecommendedAIML-LM-015

Context Windows

Long context, real costs: memory, attention growth, and using the window well.

Difficulty
3 / 5
Importance
4 / 5
Study time
1–2 h
Kind of work
Mixed

Why it matters

The learner can explain quadratic attention cost and the KV-cache memory story qualitatively.

The route to this topic

Route AIML-LM-015
What Is a Model? Tasks: Regression, Classification, and Beyond Features and Targets Classification Framing Linear Regression I: Prediction and Residuals Embeddings Tokens and Tokenization Scores and Logits Loss and Fitting: Squared Error Similarity in Vector Space Multiclass Scores and Softmax The Artificial Neuron Autoregression Query, Key, Value Activation Functions Next-Token Prediction Similarity Scores The Multilayer Perceptron Weighted Aggregation Self-Attention The Attention Block Context and Conditioning The Causal Transformer Context Windows