Dikkat Mekanizması

Mechanism

1. Query, Key, ValueThree roles, one retrieval: what the query asks, what keys offer, what values deliver.1–2 sa
2. Similarity ScoresHow well does each key match? Query-key alignment, scored.1–2 sa
3. Scaling and SoftmaxScores into weights: why divide by root-d, and the sum-to-one guarantee.1–2 sa
4. Weighted AggregationThe mixture is the message: values averaged by weights into one output.1–2 sa

Sequence Attention

1. Self-AttentionA sequence attending to itself: every token contextualized by weighted others.1–2 sa
2. Multi-Head AttentionParallel views, one output: heads attend differently, then concat and project.1–2 sa
3. MaskingInformation control: causal and padding masks decide what may attend to what.1–2 sa
4. Cross-AttentionQueries from one sequence attending to keys and values from another: how decoders read encoders and text reads images.1–2 sa
5. Efficient AttentionAttention's quadratic cost and the fixes: IO-aware kernels, sparse and sliding windows, and linear approximations.1–2 sa