Foundation Models

In the past, every task needed its own model. One for translation, one for spam, one for summaries. Today, a single large model can do all of them. In this lesson, you will learn what a foundation model is, and why this idea changed the whole industry.

• Old way: one model per task
• New way: one model for many tasks
• Why this changed everything

From many models to one

Before, teams collected data, and trained a separate model for every single task. It was slow, and expensive. A foundation model takes a different approach. First, train one huge model on a vast amount of general data, so it learns language, and the world. Then, adapt it to many tasks, with a short instruction, or a little extra training. One expensive education, then many jobs.

One model, many skills

The same model can answer questions. It can translate between languages. It can summarise a long report. It can write and fix code. It can brainstorm ideas, and explain difficult topics. Some foundation models can even understand pictures, and sound. These skills were not programmed one by one. They emerged, as the model learned to predict the next word, at huge scale.

The names you will hear

You will hear many names. GPT, from OpenAI. Claude, from Anthropic. Gemini, from Google. Llama, from Meta. Mistral, from a French company of the same name. Some are closed, available only through a service. Some are open, so anyone can download the weights, and run them. We will compare them later, and learn how to choose. For now, remember, they are all foundation models.

Summary