Inside A Transformer: Layers And Connections

A diagram of a language model sometimes shows tokens entering attention and an answer coming out. Follow the package sentence from the last lessons instead. Attention has combined information from allowed positions. What happens to that new vector before it becomes the input to another layer?

We will follow parallel attention heads, the 1st addition and normalization, a further learned transformation, and the 2nd addition and normalization. Leaving out these connections changes the computation we are trying to explain. Use the encoder block in the original Transformer paper, first released in June 2017, as an inspectable historical design.

Its input is a sequence of vectors containing information about the input and its positions. Call the vector at 1 position x. The whole sequence enters the attention calculation, while a separate path carries x around that calculation to an addition. Keeping that path visible will let

us identify exactly what is combined. Multi-head attention performs several attention calculations with different learned projections. Each head computes its own queries, keys and values, combines allowed source-position values, and produces an output. The original paper uses 8 heads. Their outputs are placed together,

then another learned projection combines them into the attention sublayer's output. The model does not simply choose the winning head and discard the others. Different learned projections allow different relationships to affect the representation. That does not give each head a guaranteed human job.

Calling a head the grammar expert or the arithmetic expert requires evidence about its behavior. For our package sentence, draw different computed contributions without inventing a discovered linguistic role. The useful question is which inputs, learned transformations and allowed connections produced the result. Now return to the path around attention.

Add the attention sublayer's output to its own input x, coordinate by coordinate. As a small supplied arithmetic fragment, let x be (2,1) and the attention output be (0.5,negative 0.5). Their sum is (2.5,0.5). This fragment illustrates the addition only; it is not a measured head

output or the complete block calculation. The incoming representation contributes directly alongside the computed change. In this original design, layer normalization follows that addition. It adjusts the coordinates at a position using their mean and variance, a small numerical stabilizer,

and learned scale and offset settings. This changes numerical behavior while preserving the vector's role in the continuing computation. We will label its output y rather than pretend that our addition-only example has already calculated normalization. Later model designs can move normalization before a transformation, so their

diagrams must show their own order. The state y then enters a position-wise feedforward network. Position-wise means that the same learned transformation is applied separately to each sequence position at this stage. In the original paper it contains a learned linear transformation, a nonlinear function,

and another learned linear transformation. Attention mixed information across positions; this network transforms the resulting information within each position. Its weights are learned, while its computed activations depend on the current input. The nonlinear function in the original block is called ReLU. It passes positive values through

and replaces negative values with 0. For a supplied intermediate pair (negative 1,2), its result is (0,2). Without a nonlinear operation between these learned transformations, the pair could be collapsed into 1 transformation with weights and a bias. The nonlinear step lets the composition express a wider set of relationships.

This small pair illustrates that operation, not an observed model feature. A 2nd path carries y around the feedforward network. Add the feedforward output to y, then apply another layer normalization. Notice the reference point: this residual path starts at the feedforward input y.

It does not return to the original block input x. The original encoder block therefore contains 2 separate addition-and-normalization sites, each attached to its own sublayer. The resulting sequence of vectors becomes input to the next block. Repeating blocks repeatedly transforms the representation;

the original encoder stacks 6. Weights can differ between these blocks even though the architectural pattern repeats. A final hidden state has passed through more than an embedding lookup and an attention mixture. The later output projection and vocabulary scores remain separate parts of the complete model.

Pause and trace the original post-normalized encoder block from x. Where do the 2 residual paths begin, and does the feedforward transformation happen before or after the 1st normalization? Explain the order without using a single box labelled attention as your whole answer. The 1st path starts at

x and joins the attention output. Normalization produces y. The feedforward network transforms y, while the 2nd residual path also starts at y. Their sum passes through the 2nd normalization. The feedforward transformation therefore follows the 1st addition and normalization. Both sites belong in

this historical block diagram. Try a different diagram. It shows attention, then feedforward, then 1 final addition of the original input and 1 normalization. A colleague says the missing connections are merely drawing shortcuts. Identify the computation that

this diagram fails to preserve, and say what evidence would justify labelling 1 of its heads the arithmetic expert. The diagram removes the 1st attention addition and normalization and gives the later residual the wrong starting state for the original architecture. Restore attention plus x, normalize to y,

then feedforward plus y and normalize again. A head label needs a defined behavioral test and evidence; its number or a suggestive arrow is insufficient. This diagnosis concerns the original post-normalized design. Another architecture requires its own explicit specification. Architecture also matters when planning a service. Head arrangements and key/value dimensions affect

temporary cache storage; feedforward shapes affect matrix work and hardware use. A model name alone does not provide those quantities. You can now inspect the connections inside a defined block and ask which configuration is actually being served. Next we will use that discipline

to compare model families by which directions information can travel and what each family is trained to produce.