AI & ML
Responsible and Reliable ML
Part of Evaluation & Trust, level 9. About 11 hours of study across the topics below.
Fairness Privacy
- Dataset BiasRepresentation, measurement, and history: how datasets encode their world.1–2 h
- Fairness ConceptsParity notions and their impossibility: what fairness math can and cannot promise.1–2 h
- PrivacyMemorization, minimization, and differential privacy as a promise shape.1–2 h
- Fairness Criteria and Their IncompatibilityDemographic parity, equalised odds, and calibration cannot all hold when base rates differ.1–2 h
Safety
- Adversarial ExamplesEvasion and poisoning: small perturbations, flipped decisions.1–2 h
- HallucinationUngrounded generation: detection, measurement, and mitigation that actually helps.1–2 h
- Specification Gaming and Reward HackingOptimisers exploit gaps between the stated objective and the intended one.1–2 h
Oversight
- Interpretability: Goals and LimitsAttribution, probes, and the limits of explanations.1–2 h
- Human OversightReview, abstention, escalation: autonomy levels with honest handoffs.1–2 h
- Documentation, Governance, and Risk ManagementModel cards, datasheets, and risk-management frameworks that make ML systems accountable.1–2 h
- Scalable Oversight and Dangerous-Capability EvaluationSupervising systems whose outputs humans struggle to check, and evaluating for high-risk capabilities before deployment.1–2 h