Text Augmentation

You've only got ten examples of a rare type of complaint. The model keeps missing them. With images, you can flip and rotate. Text is harder, because changing one word can change the meaning. Still, there are some safe ways to stretch a small dataset.

• Ten examples of a rare complaint
• Text is harder than images
• Safe ways to stretch data

Simple augmentation

Two simple tricks. Swap replaces words with synonyms that we've chosen. Drop removes a few words at random, which teaches the model not to rely on any single word. Run it. Each line is a slightly new example, with the same meaning. Be careful, though. Random swaps can flip meaning. A hand picked list is safe. Replacing good with bad wouldn't be.

import random

SYN = {"parcel": ["package", "delivery"], "late": ["delayed", "overdue"], "book": ["novel", "copy"]}
random.seed(1)

def swap(text):
    return " ".join(random.choice(SYN[w]) if w in SYN else w for w in text.split())

def drop(text, p=.15):
    return " ".join(w for w in text.split() if random.random() > p)

line = "my parcel with the book came very late"
print(swap(line))
print(swap(line))
print(drop(line))

my package with the novel came very overdue
my package with the copy came very overdue
my parcel with the book came very late

Better options today

Large language models make this much easier. Ask one to rewrite an example in five different ways, and to keep the label's meaning. Back translation sends text to French and back, which gives natural paraphrases. Always check a sample. And keep your test set made of real examples, or your score will mean nothing.

Summary