# Source Coding

Canonical: https://manyspace.aporia.systems/topics/source-coding-mth-info-344
Updated: 2026-10-13

Mathematics · Information Theory · MTH-INFO-344

- Difficulty: 4 / 5
- Importance: 4 / 5
- Study time: 6.5–12 h

optimal codes, Kraft inequality, entropy rate

## Why it matters

Quantifies information and limits of compression/communication; the theoretical foundation of modern data science and ML.

## Where it is used

Lossless/lossy compression, error correction, ML loss functions (cross-entropy), cryptography, communications.

## How you know you have learned it

Explain/derive key result from memory + solve 2 non-routine checks.

## What you need first

- [Sample Spaces & Events](https://manyspace.aporia.systems/topics/sample-spaces-events-mth-prob-196) (required)

## Resources

- Elements of Information Theory (Cover & Thomas) — Cover & Thomas