Fine-tuning with LoRA

Updating every weight of a huge model needs a huge computer. But researchers found a shortcut, that changes only a tiny part, and works nearly as well. It is called LoRA, and it has made fine tuning possible for almost anyone. In this lesson, you will understand the idea.

• Updating every weight is expensive
• A clever shortcut
• Fine-tuning for almost anyone

The idea behind LoRA

Here is the idea. Freeze the original model, so none of its billions of weights change. Next to certain layers, add two tiny extra matrices, called adapters. During fine tuning, only these small adapters learn. They act like a thin lens, on top of the frozen model, that changes its behaviour. The adapters are so small, that they may be a thousand times smaller than the model itself.

1. Freeze the modelbillions of weights stay fixed
2. Add tiny adapterssmall extra matrices
3. Train only the adaptersfast and cheap
4. Plug them ina small file changes behaviour

Why LoRA is a big deal

LoRA brings real benefits. Training needs far less memory, so it runs on a single graphics card. It is much faster, and cheaper. The adapter is a small file, a few megabytes, so you can keep many adapters, one for each task, or customer, and swap them in, on the same base model. And the original model is never damaged.

A fine-tuning project, step by step

A typical project has five steps. Collect a few hundred to a few thousand high quality examples, of the behaviour you want. Format them, as instruction and answer pairs. Train a LoRA adapter, while watching the validation score. Test it, against examples it has never seen. And compare it with a well prompted base model. If the improvement is not clear, you may not need the fine tune at all.

1. Collect exampleshundreds to thousands, high quality
2. Format pairsinstruction and ideal answer
3. Train the adapterwatch validation
4. Test and compareagainst a well-prompted base

Summary