ManySpace
Türkçe Sign in Sign up

AI & ML

Multimodal Learning

Part of Perception, level 6. About 4 hours of study across the topics below.

Representations

  1. Multimodal RepresentationsCombining images, text, and audio: joint embedding spaces, fusion strategies, and alignment.1–2 h
  2. Contrastive Image-Text PretrainingAligning image and text encoders on web-scale pairs, enabling zero-shot classification by text prompts.1–2 h

Generation

  1. Vision-Language ModelsConnecting a vision encoder to a language model so it can answer questions about images.1–2 h
  2. Text-to-Image GenerationText-conditioned diffusion: a text encoder, cross-attention conditioning, and guidance.1–2 h