How Foundation Models Learn And Adapt

A museum team has a notice: the sculpture room closes at 3 on Saturday, while the café stays open until 5. The team wants a short visitor summary and 2 opening-hours fields. 3 proposals advertise the same foundation-model family. Can the team treat those proposals as the same system, or choose

the one with the largest parameter count? Begin with what the broad model makes possible. A task-specific model might take a fixed table and predict whether a transaction is fraudulent. A broadly pretrained language model learns useful patterns across many sequences.

1 learned base can support summarizing the museum notice, extracting its times, translating a question, or drafting code. Each use still needs a clear task and an appropriate test. Foundation models can also work with images, audio, or several input types; this course first develops the language-model chain.

In this classroom example, keep the notice fixed. For summarization, the task is to preserve which closing time belongs to which place. For extraction, the task asks for separate sculpture-room and café fields. An output that assigns 5 o’clock to the sculpture room fails either

task even if its sentence is fluent or its fields are well formed. These are specified checks on illustrative outputs, rather than results from a model run. Reusing a learned base creates opportunities; the actual output determines whether each opportunity becomes useful. Pretraining establishes learned parameters through

an objective applied across training examples. A parameter is a numerical value that helps define the model’s transformations. A checkpoint saves a particular trained state. You will see the prediction objective and its update signal later. For now, the important boundary is that changing the saved trained state

can change behavior across future requests. Post-training can change that state toward instructions, preferences, or selected capabilities. The museum might compare a base checkpoint with an instruction-tuned version from the same family. The instruction-tuned version has undergone additional training intended to make requests such

as summarize this notice more useful. That is a change to the model under test. Simply placing a summary instruction and the notice into a request conditions that request; it does not by itself perform a lasting parameter update. A historical example makes the difference concrete.

In March 2022, Ouyang and colleagues described InstructGPT. They began with pretrained GPT-3, trained on human demonstrations, then used ranked outputs to train a reward model and further adapt behavior. Their raters preferred adapted outputs on the study’s prompt distribution.

This supports examining adaptation alongside model size; it does not establish performance on our museum task or every later system. Now consider a proposal that keeps the same checkpoint but supplies the museum’s current notice. The application formats the request, adds the task instruction and approved notice,

and sends that context to the model. If a visitor asks whether the sculpture room is open at 4, the supplied notice is relevant evidence. The application must preserve its scope and check the answer. If the notice is missing, a familiar closing-time pattern supplies

no observation about this museum. The application can obtain the notice or ask for the missing information before answering. Other application components change the service again. A timetable tool can supply a scheduled event time. A permission check can restrict a staff-only notice.

Stored conversation state can retain a visitor’s language preference under the application’s rules. The interface can expose a draft to a staff member before it reaches visitors. These components determine which information and actions are available around the model. A test of a bare checkpoint and a test of this

service answer different comparison questions. The team therefore needs 2 kinds of controlled comparison. To study a trained-version change, keep the task, notice, formatting, settings, scoring rule and access conditions as consistent as the experiment permits, and identify the 2 exact checkpoints.

To study the value of supplying the notice, hold the checkpoint fixed and change the defined context condition. If both the trained version and the evidence change together, an observed difference belongs to the combined configuration. The experiment cannot isolate which change produced it.

Generalization concerns performance beyond the exact examples used in development under a stated claim. After checking the sculpture-room notice, the team can reserve a new notice about archive and gallery hours, an exception such as a cancelled event, and a visitor question with missing information. A correct familiar example cannot

establish reliable performance on those cases. The expected decision and the evidence supporting it should be written before inspecting outputs. This gives the team a way to recognize a useful transfer and a specific failure. Here is a checkpoint. An assistant answers a new notice question

correctly after the application supplies that notice. The checkpoint file is unchanged. Has this observation established permanent learning in the model, and has it shown that this model is better than a competitor given no notice? Pause and decide. The observation can support successful use

of supplied context on that case. It has not established a parameter update. The competitor received a different evidence condition, so that result compares complete configurations. To compare the model versions, give them the same defined task and evidence and record the remaining conditions. Further held-out cases are needed

for a broader performance claim. For a separate practice task, a rail service compares 1 model given a current timetable with another using only the passenger’s question. List what would have to remain fixed to test model capability, and what you would change to test the timetable’s contribution.

Pause and make a brief plan before reading the worksheet key. Your plan should identify the exact trained versions, the same task and source packet, output rules, generation settings, access conditions and scoring. A context comparison keeps the model fixed while changing the timetable condition.

The rail task changes the domain, but the reasoning is the same: name the intervention before attributing the result. The museum can now describe a proposal in useful terms: a particular checkpoint and adaptation, for specified tasks, inside an application that supplies evidence, controls access and checks outputs.

Broad learning makes reuse possible. Evaluation establishes how well that reuse works under the conditions that matter. Next, we will follow the notice into the model and see why its sequence units depend on a tokenizer.