Fine-Tuning LLMs

Bright Books wants Scribe to sound like Bright Books. Warm, a little bookish, and never pushy. A prompt gets you part of the way. But what if you need that voice on every reply, with hundreds of rules? That's when people turn to fine tuning. It's also the most overused tool in NLP.

• Scribe needs a distinct voice
• Prompts go part of the way
• Fine-tuning is overused

Prompt, RAG, or fine-tune?

Here's the decision. Prompting needs no data, and it's the cheapest. RAG needs documents, and it gives the model new facts. Fine tuning needs hundreds of good examples and more effort, and it changes style and behaviour, not knowledge. Run it, and read the table. Always try them in that order. Prompts, then RAG, and fine tuning only if you still need it.

options = {
    "Prompting": {"data": 0, "cost": 1, "knows_new_facts": False, "new_style": "some"},
    "RAG": {"data": 100, "cost": 2, "knows_new_facts": True, "new_style": "no"},
    "Fine-tuning": {"data": 500, "cost": 5, "knows_new_facts": False, "new_style": "yes"},
}

for name, o in options.items():
    print(f"{name:<12} examples {o['data']:<4} effort {o['cost']}  "
          f"new facts {str(o['knows_new_facts']):<5} new style {o['new_style']}")

Prompting    examples 0    effort 1  new facts False new style some
RAG          examples 100  effort 2  new facts True  new style no
Fine-tuning  examples 500  effort 5  new facts False new style yes

When fine-tuning is worth it

Fine tuning makes sense for a consistent style, for a strict output format, for domain language, and for cutting cost, by making a small model do the job of a big one. It's a poor choice for adding facts, since they go stale, and for fixing bad prompts. And if you can't write down what good looks like, then you aren't ready.

Summary