Foundation Models: Tokens, Attention, And Generation · Start Here 3

Follow One Question Into An Answer

A language model processes a question and supplied information, builds a response through successive text choices, and still needs a check that its answer is supported.

Study And Save Progress

Watch The Lesson

3 min 54 sec

Read the video transcript

A visitor asks, Is the museum archive open today? The application supplies a current notice: The archive is closed today. The cafe remains open. We will follow that question into an answer, then check whether the answer helps the visitor.

Keep the archive and cafe separate as the wording moves through the system. The application passes the question and relevant notice to a language model. These words are the information available for this request. The notice has to match the day being asked about.

For this teaching example, it does. No real museum service or model has been run; we are tracing how the parts of a possible service fit together. The system divides text into small pieces called tokens. A token might cover a word, part of a word or punctuation.

Think of cutting a message into pieces that the system can handle consistently. The exact cuts depend on the model's text-processing method. We can follow the idea without choosing the cuts for any particular product. Those pieces are represented using numbers so that the computer can calculate with them.

Order information helps distinguish different arrangements. You do not need to calculate the numbers here. What matters is that the model now has a way to process the wording and its relationships, rather than treating the entire notice as an indivisible object. In a widely used design

called a transformer, a process called attention lets information from different text positions influence what is computed. For our example, a useful answer needs the relationship between archive and closed, while cafe and open belong together. The original transformer paper appeared in June 2017. We are using a simple illustration

of relationships, without claiming to have inspected a model's internal attention for this notice. The model then produces scores for possible next text pieces. A choice process selects a piece, adds it to the growing response and continues. For instance, a response could grow from The to The archive and

then to The archive is closed today. These are illustrative stages; the actual piece boundaries depend on the model. Producing that sentence leaves a practical job: check it. Does archive refer to the requested facility? Does closed match the current notice? Does today apply to the visitor's day?

A fluent sentence about the cafe would send the visitor in the wrong direction. The application or a person must compare the answer with the evidence before relying on it. Try a different notice: The pool is closed this afternoon. The gym remains open.

Someone asks about the pool this afternoon, and a proposed answer says it is open. What part of the notice decides whether that answer is supported? Pause and identify it. The pool's closure sentence is the relevant evidence.

The gym sentence concerns a different facility. A generated answer that borrows open from that sentence fails the support check even if its wording is clear. Use the worksheet to explain the archive example in order: what enters, what happens to the text, how a response grows and what must be checked.

Everyday words are welcome. Pause and write your explanation. The question and current notice enter. Small text pieces receive numerical representations and are processed in relation to other pieces. The response grows through successive choices, then its claim is checked against the notice.

You now have a whole-task picture. The 1st core lesson builds on this picture to examine how a broadly trained model is adapted and used; later calculations offer a closer view of individual steps.

Main Point

A language model processes a question and supplied information, builds a response through successive text choices, and still needs a check that its answer is supported.

Key Points

1. Supply the right information

The visitor asks whether the archive is open today. The current applicable notice says: “The archive is closed today. The cafe remains open.” The question and notice enter the request together. This is a fictional teaching case with a stipulated current notice, rather than an observed model test.

2. Make text available for calculation

A token is a small text piece, such as a word, word part or punctuation. The particular system determines the boundaries. Numerical representations allow the model to calculate with those pieces and their order. You need no numerical method to understand the role: text becomes something the system can process in relation to surrounding text.

3. Use relationships and grow the response

Attention in the transformer design allows information from different positions to contribute to computation. Our illustrated relationship links archive with closed and cafe with open; it is not an observed internal attention map. Separately, the model produces scores for possible next pieces, a choice process selects a piece, and the growing response becomes part of the material used for the following step. The illustrative “The”→“The archive”→“The archive is closed today” sequence shows growth without claiming exact token boundaries.

4. Compare the claim with its support

The proposed archive answer must match the facility, closure and applicable day in the notice. Clear wording about the cafe cannot answer the archive question. For a transfer task, the notice says the pool is closed this afternoon and the gym remains open. A proposed open-pool answer conflicts with the pool sentence.

5. Give a practical explanation

Follow the question and evidence into text processing, then successive output choices, then the support check. You can explain this chain before studying its calculations. Lesson 1 now examines reuse and adaptation of the learned base. Later technical lessons reveal the numerical details; keep this whole-task picture beside them.

6. Learning checks

M14-START03-Q01

Notice: “The pool is closed this afternoon. The gym remains open.” Question: “Is the pool open this afternoon?” Proposed answer: “It is open.” Which part decides support, and is the answer supported?

Compare your reasoning

The sentence stating that the pool is closed this afternoon is the relevant evidence. The proposed answer conflicts with it. The gym sentence concerns a different facility and cannot support an open-pool answer.

M14-START03-P01

Explain the archive request in 4 steps: input; text processing; response growth; support check. Use everyday language.

Write your own explanation before opening the comparison. The separate worksheet provides blank response spaces.

Compare your explanation

The question and applicable archive notice enter together. The system divides text into pieces, represents them numerically and processes relationships. It builds a response through successive choices. The final claim is compared with the archive closure and day in the notice before anyone relies on it.

Preparation and optional refresh

AI Foundations is available for a whole-request refresher: connect a model’s output to the task and the surrounding service. Optional support: Data: From Business Records To Model Inputs is available if you want to follow how a reliable record or notice becomes usable input. Choose a targeted refresh where needed; neither whole course is an added prerequisite.

Availability checked 30 October 2026.

Sources

Sennrich, Rico, Barry Haddow and Alexandra Birch. Neural Machine Translation of Rare Words with Subword Units. ACL, August 2016. The paper describes reusable word fragments in a historical translation method.

Devlin, Jacob, Ming-Wei Chang, Kenton Lee and Kristina Toutanova, Google AI Language. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL-HLT, June 2019; preprint version 2 dated 24 May 2019. The input-representation description and Figure 2 show numerical input components and contextual processing in this historical architecture.

Vaswani, Ashish, and colleagues. Attention Is All You Need. First submitted 12 June 2017; version 7 dated 2 August 2023. Sections 3.2 and 3.4 distinguish combining information from text positions from producing scores for output pieces. The museum links are a teaching illustration, without measured internal attention. All sources accessed 30 October 2026.

Learner Materials

Download Start Here 3 Learner Materials

Use the supplied fictional cases and templates for practice.

Practice

Write your answer, compare it with the model answer and decide what to revisit. These written checkpoints are self-assessed.

Notice: “The pool is closed this afternoon. The gym remains open.” Question: “Is the pool open this afternoon?” Proposed answer: “It is open.” Which part decides support, and is the answer supported?

Show The Model Answer

The sentence stating that the pool is closed this afternoon is the relevant evidence. The proposed answer conflicts with it. The gym sentence concerns a different facility and cannot support an open-pool answer.

Compare your reasoning with the model answer. These written checks are self-assessed.

Explain the archive request in 4 steps: input; text processing; response growth; support check. Use everyday language. Write your own explanation before opening the comparison. The separate worksheet provides blank response spaces.

Show The Model Answer

The question and applicable archive notice enter together. The system divides text into pieces, represents them numerically and processes relationships. It builds a response through successive choices. The final claim is compared with the archive closure and day in the notice before anyone relies on it.

Compare your reasoning with the model answer. These written checks are self-assessed.

Open Practice And Save Your Work