Training And Inference, Step By Step

The retailer wants to estimate next week's umbrella sales. It has records from earlier weeks, but next week's sales have not happened yet. Two different jobs are involved: learning a useful relationship from past examples, and using the learned model to estimate a new outcome. Those jobs are called training and inference. This distinction matters because improving a model

and operating a service are different problems. A system can finish training and still be expensive or slow when people use it. And an assistant can respond differently to a new message without changing its learned parameters. In this lesson, we'll follow one predictor from past records to a new forecast, then compare that process with a

language model and a working service. Our route is: learn from past examples, use new inputs, and operate at the required scale. By the end, you should be able to explain what changes during training, what happens during inference, and why the two can need different operating arrangements. Training is a process that adjusts a

model using data and a learning procedure. Inference uses the resulting model to compute an output from an input. Remember the distinction from the previous lesson: data supplies information, parameters are internal values, and instructions request behavior during use. Here, we're following when and how the learned values are adjusted, compared with when they're used.

Start with the sales predictor, rather than a language model. The retailer collects examples from earlier weeks. Each example contains inputs, such as advertising spend and relevant context, together with a known outcome: the number of umbrellas actually sold. The outcome is known because that week is already over. These recorded outcomes give the learning

process something to compare its predictions with. For a small arithmetic illustration, temporarily use advertising spend as the only input. Imagine a deliberately simple model that multiplies dollars spent on advertising by one stored value to estimate units sold. Real sales depend on more than advertising, and a useful model may include weather,

season, and other relevant inputs. This tiny example is a way to see the learning loop, not a claim that advertising alone explains sales. Suppose one past week's advertising spend was one hundred dollars, and actual sales were twenty umbrellas. Start with a stored multiplier of zero point one. One hundred times zero point one

gives a prediction of ten umbrellas. We can now compare the prediction, ten, with the recorded outcome, twenty. The prediction is ten units too low. The learning process needs a measure of error. Such a measure is often called a loss. For this illustration, use the absolute difference: the distance between the predicted and actual counts,

without a positive or negative sign. The absolute difference between ten and twenty is ten. Other learning tasks can use different loss measures. Loss makes a particular kind of error measurable; choosing it is part of designing the learning process. An optimization process is the procedure that changes parameters in pursuit of its learning objective. To illustrate the effect of an adjustment,

change our multiplier from zero point one to zero point two. With the same one hundred dollar input, the model now predicts twenty umbrellas. On this one example, our absolute error falls from ten to zero. The stored value changed; the recorded input and actual outcome did not.

That is an illustration of a parameter update, not a complete optimization algorithm. It does not tell us how a real system should choose its next adjustment. During training, the system repeatedly produces predictions from examples, compares them with known outcomes, and adjusts parameters according to its learning procedure. Module one develops those mechanisms more carefully.

A perfect match on one past week is not proof of a useful predictor. Other weeks may behave differently. Repeating the learning process can improve predictions under the learning conditions, but improvement is not guaranteed on every example or on future weeks. We need evaluation beyond simply noticing that one training error went down.

A relationship in historical records also does not, by itself, prove that increasing advertising will cause the predicted increase in sales. Pause and identify what happened. The prediction changed from ten to twenty because the stored multiplier changed. Was that training, or was it merely using a fixed model with a new input?

That was an illustrative training update. We changed a parameter using a known outcome to assess error. If we had kept the multiplier fixed and only supplied a different input, that would instead have been inference with this model. Now finish the training phase for our example and use the resulting multiplier, zero point two.

The retailer plans one hundred and fifty dollars of advertising for next week. Inference supplies that new input and computes an output with the learned model. One hundred and fifty times zero point two gives an estimate of thirty umbrellas. The parameter stays fixed during this calculation. Thirty is a forecast,

not a record of thirty completed sales. Next week has not happened yet. At the moment of inference, we have the planned input and the model's estimated output. We do not yet have the actual outcome needed to calculate the forecasting error for that week. Suppose next week ends and actual sales turn out to be twenty-four umbrellas.

We can now compare the earlier forecast, thirty, with the observed outcome, twenty-four. The absolute forecasting error is six units. This later result helps us evaluate the system. Depending on the development process, it may eventually become part of future training data. Receiving the actual result is not, on its own, a parameter update.

Think about the timeline. When the retailer first produced the estimate of thirty, was twenty-four already available as the actual outcome for that future week? No. The forecast came first; the actual outcome arrived later. Evaluation could use that later evidence, and a later learning process might use it too.

Keeping those events in order prevents us from treating a future result as information the model already had when it made the prediction. A language model follows a more complex version of the same distinction. During training, a learning process adjusts many parameters using large collections of examples and chosen objectives. During inference, the model processes

the current input and generates output. The scale and details differ from our tiny predictor, but adjusting learned values and using learned values remain different activities. For example, you might tell an assistant, “Keep the answer brief,” and then ask a question. The next response can be shorter because the instruction is present in the current context.

Earlier conversation messages can also be available in that context. Adapting an answer to those messages does not automatically mean the conversation retrained the model. To claim that parameters changed, we need evidence about the learning process, not just a change in the answer. There can also be additional learning after an initial training phase.

This is called post-training. It is often used to improve instruction-following, task behavior, or other selected goals. The exact pipeline varies, so post-training is not the name of one compulsory sequence that every model follows. Post-training can use supervised examples: examples of inputs paired with desired responses or behavior.

It can use preference information, such as comparisons indicating which response is preferred under some criteria. It can also involve reward-driven methods, in which a reward signal guides learning. These describe possible ingredients. We are not assuming that every system uses all of them, or that any of them guarantees a correct answer.

Keep three levels separate. Initial training develops learned capabilities. Subsequent behavioral adaptation can further adjust the model through additional learning. Use-time application controls can supply instructions, restrict available tools, or check outputs without necessarily changing the model's parameters. A product's behavior can depend on all three.

Later lessons and modules develop the learning mechanisms and system controls in detail. Now move from the model's logic to the work of running it. Both training and inference need processing and memory. Processing performs computations; memory makes the information needed for those computations available. The fact that both need these resources does not

mean their best hardware or operating configurations are identical. A training run may continue for a long period, repeatedly processing examples and computing parameter updates. When work is spread across processors, they may need to communicate updates. Picture repeated passes through learning work, with processors, memory, and communication supporting the process. The learning procedure and scale affect what becomes a bottleneck.

Interactive inference has a different immediate obligation: answer users within a useful response time. Requests can arrive unpredictably. One moment there may be a few users; the next, many may ask at once. Latency is the time a request takes to receive its response. Concurrency concerns requests in progress at the same time.

A serving system has to manage both, along with the quality of the results. Not all inference is interactive. Some tasks can be delayed or grouped together, such as processing a set of documents for a report due later. Other requests need timely individual responses. Grouping work may help use resources,

but waiting to form a group can conflict with a response-time requirement. The task's requirements help determine an appropriate operating design; the word inference alone does not settle that choice. This creates an economic distinction as well. Training cost is not the only cost that matters. Developing a model can require substantial work and spending. Once the model is in service,

processing requests creates recurring inference demand. A business has to serve those requests affordably at the required quality and latency, not simply celebrate that training is complete. For an arithmetic illustration, suppose the total serving cost averages two cents per request for a particular setup and workload. At one million requests, two cents multiplied

by one million is twenty thousand dollars. These are invented numbers to show the multiplication, not a quote for any model or provider. Actual cost depends on the workload and operating design. Lowering a cost is only useful if the service still meets the quality and response-time requirements of the task. Consider this diagnostic:

an assistant answers correctly in a demonstration but costs too much when deployed to many users. Training may already be complete. The immediate problem can be inference economics: the cost of repeatedly serving the selected task. A model that is efficient to train can still be expensive to serve. Research and development economics and the economics of

operating the service therefore need separate analysis. Here's the final checkpoint. Name one resource needed in both training and inference. Then name one reason their best configurations can differ. Both need processing and memory; either is a valid shared resource. Training also computes parameter updates and may communicate them across processors.

Interactive inference must manage latency, concurrent requests, and serving behavior. Those differences can lead to different hardware choices, schedules, and operating arrangements, even though both activities use computation. Return to our retailer's timeline. Past records supplied known inputs and outcomes for learning. Training compared predictions with those

outcomes and adjusted a stored value. Inference then used the learned model with a new planned input to produce a forecast. The actual sales arrived later, giving evidence for evaluation and potentially future training. None of those stages should be substituted for another. When someone says a model is learning, ask what is actually happening: are parameters being adjusted,

is a fixed model processing a new context, or is an application controlling the interaction? When someone says a model is affordable, ask whether they mean developing it or serving the required workload. Those questions locate the mechanism and the economic problem. Next, we'll separate prediction, generated content, and actions that change the outside world.