ManySpace
English Giriş yap Kayıt ol

Yapay ZekâÇekirdekAIML-TRF-010

Encoder-Only Models and Masked Language Modeling

BERT-style bidirectional pretraining: predict masked tokens, then reuse the representations.

Bu konunun henüz Türkçe çevirisi yok, bu yüzden İngilizce gösteriliyor.

Zorluk
2 / 5
Önem
5 / 5
Çalışma süresi
1–2 sa
Çalışma türü
Karma

Neden önemli

The learner can construct a masked-LM training example and explain why bidirectional context helps understanding tasks but not free generation.

Bu konuya giden rota

Rota AIML-TRF-010
What Is a Model? Tasks: Regression, Classification, and Beyond Features and Targets Classification Framing Linear Regression I: Prediction and Residuals Embeddings Tokens and Tokenization Scores and Logits Loss and Fitting: Squared Error Similarity in Vector Space Multiclass Scores and Softmax The Artificial Neuron Autoregression Objective Functions and Empirical Risk Query, Key, Value Activation Functions Next-Token Prediction Likelihood Similarity Scores The Multilayer Perceptron Cross-Entropy Loss Weighted Aggregation The Language-Modeling Objective Self-Attention The Attention Block The Feed-Forward Block Encoder and Decoder Concepts Encoder-Only Models and Masked Language Modeling