ManySpace
Türkçe Sign in Sign up

AI & MLRecommendedAIML-GM-014

Discrete Latents and Learned Tokenizers

Vector quantisation turns continuous signals into codebook tokens that autoregressive and multimodal models can use.

Difficulty
4 / 5
Importance
4 / 5
Study time
1–2 h
Kind of work
Mixed

Why it matters

The learner can quantise encoder outputs to a codebook and explain the codebook-size versus reconstruction trade-off.

The route to this topic

Route AIML-GM-014
Natural Numbers What Is a Model? Variables & Constants Tasks: Regression, Classification, and Beyond Features and Targets Solving Linear Equations Function Notation & Composition Classification Framing Linear Regression I: Prediction and Residuals k-Means Complex Numbers Scores and Logits Loss and Fitting: Squared Error Basic Ratios Unit Circle The Artificial Neuron Linear Regression II: Least Squares Fitting Objective Functions and Empirical Risk Polynomial Functions Activation Functions Principal Component Analysis Likelihood Intuitive Limits Continuity Definition of the Derivative Differentiation Rules The Multilayer Perceptron Latent Representations Maximum Likelihood Estimation Bayesian Reasoning Chain Rule Antiderivatives & Indefinite Integrals Forward Propagation Mixture Models Intuition Derivatives of Transcendental Functions Riemann Sums & Definite Integral Computational Graphs Expectation-Maximization Fundamental Theorem of Calculus The Chain Rule in Neural Nets Integration by Parts Backpropagation Sample Spaces & Events Autoencoders Random Variables & Distributions Entropy & Mutual Information Variational Inference and the ELBO Variational Autoencoders Discrete Latents and Learned Tokenizers