AI & ML

Security and Privacy

Part of Evaluation & Trust, level 9. About 9 hours of study across the topics below.

Privacy

  1. Differential PrivacyA formal privacy guarantee: outputs change little whether or not one person's data is included; DP-SGD for training.1–2 h
  2. Federated LearningTraining across devices without centralising data, and what it does and does not protect.1–2 h
  3. Membership Inference and Model ExtractionPrivacy and IP attacks through the model's outputs: was this record in training, and can the model be copied?1–2 h

Threats And Robustness

  1. Threat Modeling for ML SystemsAttack surfaces across the ML lifecycle: data, training, model artifact, inference API, and integrations.1–2 h
  2. Adversarial Training and Certified RobustnessTraining on worst-case perturbations, and proving robustness within a radius.1–2 h
  3. Data Poisoning and BackdoorsCorrupting training data so a model fails generally, or behaves normally except on a hidden trigger.1–2 h

Llm And Content Security

  1. Prompt InjectionUntrusted text treated as instructions: direct and indirect injection, and why it is hard to eliminate.1–2 h
  2. Jailbreaks and Red TeamingEliciting behaviour a model was trained to refuse, and systematic adversarial testing before release.1–2 h
  3. Content Provenance and WatermarkingMarking and tracing generated content: statistical watermarks, signed provenance metadata, and why detection is fragile.1–2 h