Synthetic Data
Training on fake apples.
Bruised apples are rare, perhaps four in every hundred. Some kinds of damage only happen a few times a year. How can you train a model on something you barely have? You make it. Synthetic data is generated by computer, and it arrives with perfect labels.
- Rare problems are hard to photograph
- Make your own examples
- Perfect labels for free
Drawing training data
The simplest approach is to draw it yourself. Here, a small function makes apple images, with random sizes, colours, and positions, and adds a bruise half of the time. Since we drew the bruise, we know exactly where it is. Run it. That gives us unlimited, perfectly labelled images. Random variation like this is called domain randomisation.
import numpy as np
import cv2
import matplotlib.pyplot as plt
rng = np.random.default_rng(1)
def synth():
img = np.full((96, 96, 3), rng.integers(30, 70), np.uint8)
r = int(rng.integers(22, 34)); x, y = rng.integers(34, 62, 2)
colour = tuple(int(v) for v in (rng.integers(170, 230), rng.integers(20, 60), rng.integers(20, 50)))
cv2.circle(img, (int(x), int(y)), r, colour, -1)
bruised = rng.random() < 0.5
if bruised:
cv2.circle(img, (int(x) + 8, int(y) - 6), int(r * 0.35), (90, 50, 35), -1)
return cv2.GaussianBlur(img, (3, 3), 0), int(bruised)
fig, ax = plt.subplots(2, 6, figsize=(9, 3.2))
labels = []
for a in ax.ravel():
im, b = synth(); labels.append(b)
a.imshow(im); a.axis("off")
plt.tight_layout(); plt.savefig("synthetic.png", dpi=110)
print("Bruised in this batch:", sum(labels), "of 12")Bruised in this batch: 4 of 12
Training on fake data
Can a model learn from this? We generate fifteen hundred fake apples, and train a simple classifier. Run it. Strong accuracy on new fake apples. The real test, though, is real apples. There's always a gap between simulation and reality. So the best recipe is to train on lots of synthetic data, then fine tune on a small real set.
import cv2
import numpy as np
from sklearn.linear_model import LogisticRegression
rng = np.random.default_rng(1)
def synth(n):
X, y = [], []
for _ in range(n):
img = np.full((48, 48), rng.integers(30, 70), np.uint8)
r = int(rng.integers(12, 18)); cx, cy = rng.integers(16, 32, 2)
cv2.circle(img, (int(cx), int(cy)), r, 190, -1)
b = rng.random() < 0.5
if b:
cv2.circle(img, (int(cx) + 4, int(cy) - 3), 5, 90, -1)
X.append(img.ravel() / 255.0); y.append(int(b))
return np.array(X), np.array(y)
Xtr, ytr = synth(1500)
Xte, yte = synth(300)
model = LogisticRegression(max_iter=500).fit(Xtr, ytr)
print("Trained only on fake apples")
print("Accuracy on new fake apples:", round(model.score(Xte, yte), 3))Trained only on fake apples Accuracy on new fake apples: 0.977
Making synthetic data useful
Bridge the gap in a few ways. Randomise lighting, textures, and backgrounds. Paste real objects on to varied backgrounds, which is cheap, and surprisingly effective. Use diffusion models to create realistic rare defects. Always test on real images. Warehouses and self driving companies train on huge amounts of simulated data.
Summary
Synthetic data solves the rare case problem, as long as you test on real images. Next, we'll step fully into the third dimension. How do machines capture the shape of the real world?
Key takeaways- Draw your own examples
- Perfect labels for free
- Mind the sim-to-real gap