How Next-Token Training Works

The training text says: The exhibition opens on Monday. Suppose Monday is the next target token in a supplied teaching sequence. A model gives that target probability 0.1 in 1 calculation and 0.8 in another. What makes the training process prefer the 2nd prediction?

Follow the observed target, the vocabulary scores, the loss calculation, and the parameter update. Then ask what this learning signal can tell us about a visitor's real question. A basic causal language-model objective uses an observed sequence to provide targets. At each prediction position, the preceding allowed tokens are the input and

the observed next token is the target. For this example we stipulate Monday as 1 token; an actual tokenizer determines the real units. The training target comes from the sequence. No researcher needs to write a separate question and answer for every position in this basic setup.

The model's hidden state reaches an output projection that computes logits, meaning unnormalized scores for vocabulary candidates. Softmax converts these scores into probabilities that sum to 1 across that vocabulary under the defined calculation. This distribution is over possible output tokens. In the attention example, softmax normalized shares

over source positions and combined value vectors. The same mathematical function now has a different set of objects and a different role. The target's probability supplies a simple loss calculation: take its natural logarithm and change the sign. The natural logarithm, written ln, uses

the mathematical constant e as its base. You can inspect the values with a calculator rather than derive logarithms to follow the argument. A target probability close to 1 gives a small loss; a target given very little probability receives a larger loss. With target probability 0.8,

negative ln of 0.8 is approximately 0.223. With target probability 0.1, negative ln of 0.1 is approximately 2.303. The 2nd loss is much larger. We have held the target token fixed while comparing the probabilities assigned to it. These are supplied numerical cases, not measured predictions from a museum model.

The lower loss rewards assigning more probability to this observed target under the objective. This calculation is a simple instance of cross-entropy loss for a single observed target. Training combines losses across eligible positions and examples under its setup. Gradients describe how a small change in a parameter would change that combined loss.

An optimizer uses this information to update parameters. The update is distributed through the learned computation; it does not append a new database row saying that Monday is always the correct answer. Nor does this example supply a parameter-update size.

That requires the actual model computation, derivatives, optimizer and learning rate. A training step aims to reduce the chosen objective, but it need not improve every individual example after a shared update. Different examples can pull shared parameters in different directions. Evaluation on separate data helps

us distinguish fitting the training examples from useful prediction beyond them. A complete training sequence supplies many prediction positions. A program can calculate multiple positions together while causal masks prevent each prediction from accessing prohibited later tokens. For the earlier prediction of opens,

Monday is still beyond its allowed prefix. Having the whole sequence in the training record does not authorize leaking that future token into the prediction. Ordinary generation has a different dependency: each newly selected output token supplies context for the next step. An evaluation measure called perplexity

is related to predictive loss. Under the natural-log convention, exponentiating the average token loss gives perplexity. Its value depends on the data, tokenization and evaluation procedure. Changing the tokenizer changes the units being scored, so raw perplexities across different tokenizers need careful interpretation.

The number also does not describe every downstream skill or the usefulness of a deployed service. The 2020 GPT-3 paper reported Penn Tree Bank perplexity and omitted other language-modeling datasets because of training overlap. That choice illustrates why the evaluation record matters: prediction on previously encountered

text can overstate performance on separate data. For the museum, lower held-out prediction loss is useful evidence about that objective. An opening-hours answer still needs the applicable current schedule and a task-level check. Pause at the 2 supplied target probabilities. Calculate negative natural log loss for 0.8 and 0.1 to 3 decimal places.

Which receives the larger penalty, and why does the smaller loss alone fail to establish the truth of a new opening-hours answer? The losses are approximately 0.223 and 2.303. The 0.1 target probability receives the larger penalty because the observed

target was assigned less probability. The smaller loss supports this prediction objective on the specified example. It has not independently checked a new schedule, selected the applicable date or verified a generated factual claim. Those require evidence and evaluation beyond this calculation.

Try a transfer case with supplied target probabilities 0.5 and 0.25. A calculator gives natural logarithms of approximately negative 0.693 and negative 1.386. Record each negative-log loss and identify the preferred prediction under this objective. Then a colleague proposes comparing average token losses from 2 different tokenizers

as a direct ranking of service reliability. What information is missing? The losses are approximately 0.693 and 1.386, so assigning 0.5 to the same target produces the smaller loss. A comparison across tokenizers needs the tokenization, data and scoring protocol, including how different units are made comparable.

Service reliability also requires task-level tests. Neither a training-loss value nor its perplexity transformation establishes the correctness of every response. Text contains patterns that support many useful tasks, so learning to predict it can build useful representations and behavior. The objective we inspected rewards prediction under the data and training design.

Suitable data, adaptation, application evidence and evaluation help turn that capability into a dependable service. You can now distinguish the observed target, its assigned probability, the loss and the update. Next we will follow a fixed model generating a response, where tokens are selected

without an ordinary inference-time parameter update.