AI Ecosystem CourseLearn with Q
AI Ecosystem · In Development

Post-Training, Adaptation, And Evaluations

Learn how models are adapted for particular tasks and checked before release.

About This Course

Introduction

A broad model may be capable and still be a poor fit for a product. It may return the wrong format, invent details, mishandle exceptions, or take too much time and money. Post-training changes learned behavior after pretraining. Evaluation helps establish what improved, under what conditions, and what may have become worse.

We will connect supervised fine-tuning, preference optimization, reinforcement learning, parameter-efficient adaptation, distillation, and compression to evaluation. Each method addresses particular problems. Your project is a twenty-five-case evaluation packet and a model-selection memo grounded in the results.

Review Modules 1, 13, and 14 for training, loss, gradients, data splits, parameters, context, and generation. Study adaptation first, evaluation design next, then the complete case packet. Keep an unchanged baseline. Without it, it is much harder to know whether an intervention helped.

Planned Lessons

  1. Diagnose The Failure Before Choosing TrainingFind out why an AI answer is poor before deciding whether the model needs more training.
  2. Supervised Fine-Tuning, Or SFTSee how teaching a model with good examples can improve its answers on a particular task.
  3. Preference Data And The Reward-Model ApproachExplore how people’s comparisons of answers can help guide model training.
  4. Direct Preference Optimization, Or DPOSee how a model can learn directly from examples of preferred and less useful answers.
  5. Reinforcement Learning And Verifiable RewardsExplore how training can use results that can be checked, such as whether an answer is correct.
  6. Parameter-Efficient Fine-Tuning And LoRACompare changing a small part of a model with updating all of its learned numbers.
  7. Catastrophic Forgetting, Regressions, And Retained CapabilityCheck whether a model loses useful skills while learning to perform a new task.
  8. Distillation, Pruning, Quantization, And Speculative DecodingCompare ways to make models smaller, less costly to run or faster to answer.
  9. Define “better” As A Set Of Task ConditionsDefine what a good result means for the people and task an AI tool will serve.
  10. Case Design, Slices, And CoverageChoose test cases that cover the users, situations and errors that matter.
  11. Rubrics, References, And Human ReviewUse clear marking criteria and example answers to make human reviews more consistent.
  12. Small Samples And Honest UncertaintyJudge how much confidence a small set of test results can support.
  13. Latency, Cost, Quality, And Successful WorkCompare AI tools through useful results, waiting time and the money spent delivering them.
  14. Red Teaming, Adversarial Tests, And Controlled RecoveryTest likely ways a system could fail and whether it can recover safely.
  15. Offline Evaluation, Shadowing, And Production ExperimentsCompare testing a model before release with observing it in use and running live comparisons.
  16. A Model-Selection Memo That Supports A DecisionWrite a clear recommendation that explains which model to choose and why.