The Transformer

Now we assemble the pieces. A transformer is built from one block, repeated many times. In this lesson, you will see inside that block, and you will understand why this design took over the whole field of A I.

• One block, repeated many times
• Attention plus a thinking layer
• Why this design won

Inside one transformer block

Here is one block. Tokens enter from the bottom, as embeddings. First comes the attention layer, where words look at each other, and share information. Next comes a feed forward layer, where each word is processed on its own, a bit like thinking about what it just learned. Around both layers, there are shortcuts, called residual connections, and a step that keeps numbers stable, called normalisation. The output goes up, to the next block.

1. Input embeddingstokens as meaning + position
2. Attentionwords share information
3. Feed-forwardeach word is processed
4. Outputricher meaning, passed upward

Stacking many blocks

One block is only the start. Models stack many blocks, on top of each other. The first blocks learn simple things, like grammar, and word relationships. Middle blocks learn facts, and meaning. And later blocks deal with high level ideas, and reasoning. GPT three has ninety six of these blocks. Each block refines the understanding of the one below, like a team of editors, passing a document along.

1. Block 1grammar and word relationships
2. Block 2 … 40facts and meaning
3. Block 41 … 96ideas, style and reasoning
4. Output layera chance for every next token

Generating one token

At the very top, the model produces a score for every token in its vocabulary. These scores become probabilities, using a function called softmax. The model picks a token, and adds it to the text. Then the whole process runs again, with one more token. This is how a long answer is written, one token at a time, each one depending on everything before it.

1. Read the texttokens and embeddings
2. Thinkpass through all the blocks
3. Scorea chance for every token
4. Pickadd one token and repeat

Why transformers won

Why did transformers beat older designs? Older models read text one word at a time, in order, which was slow, and they forgot the beginning of long texts. Transformers look at all the words at once, so they can be trained in parallel, on thousands of chips. That made it possible to train on much more data, and to build much larger models. And bigger models, turned out to be much smarter. Scale, made possible by the transformer, is the story of the last few years.

Summary