Follow One Sentence Through a Transformer

You now know all the parts. In this lesson, we put them together, by following a single sentence, from the moment you type it, until the next word appears. It is the best way to see how everything fits, and a wonderful recap of the technical half of this course.

• One sentence, the whole journey
• Every part, in order
• A recap of how it works

Step one: tokens

Our sentence is, the cat sat on the. The tokenizer cuts it into tokens, and gives each one a number. Five tokens, five I Ds. From now on, the model works only with these numbers.

Step two: embeddings

Step two, each token I D is looked up in a big table, and replaced by its embedding, a long list of numbers that captures meaning. On our simplified map, cat sits near other animals, and sat sits near other actions. Position signals are added too, so the model knows the order.

Step three: attention

Step three, attention. Each word looks at the others, to update its meaning. Look at the last word, the. It has to decide what comes next, so it pays attention to the earlier words. It looks strongly at sat, and on, because a seat is likely to come, and also at cat, because the subject matters. Context flows into every word.

Step four: many blocks

Step four, the sentence passes through many blocks. In each one, attention gathers context, and a feed forward layer processes it. Early blocks work out grammar. Later blocks work out facts, and meaning. By the top, the last word's numbers contain a rich summary of everything the model understands, about what should come next.

1. Embeddings inmeaning plus position
2. Blocks 1 to 10grammar and local patterns
3. Blocks 11 to 40facts, meaning, context
4. Top blocka summary: what comes next?

Step five: the next word

Step five, the output. The top layer turns that summary into a chance for every token in the vocabulary. With softmax, they become probabilities. The model picks one, say mat, and adds it to the sentence. Then the whole process starts again, with one more token. Tokens, embeddings, attention, blocks, probabilities, pick. That is the machine.

Summary