Deep Learning Basics for Ops

You do not need to be a deep learning researcher to operate deep learning models. But you do need to understand their lifecycle, because it differs from classic models. Training is longer, files are bigger, hardware matters, and saving and loading must be done carefully. Let's train a tiny network, and follow its journey.

• Longer training, bigger files
• Hardware matters
• Save and load carefully

The training loop

Every neural network trains with the same loop. Take a batch of data. Run it forward through the network, to get predictions. Measure the error, called the loss. Run backward, to compute how each weight contributed. And let the optimiser nudge the weights, to reduce the loss. Repeat for many batches, and many epochs. Operationally, three things matter. The loop can run for hours. Its state, weights and optimiser, must be checkpointed. And its randomness must be seeded, to be reproducible.

1. Forwardpredictions from a batch
2. Losshow wrong were we?
3. Backwardgradients for every weight
4. Optimiser stepnudge the weights

A tiny network in PyTorch

Here is a real network, in PyTorch. We seed the random numbers, prepare the data as tensors, and scale it. The model has two layers, with a non linear step between them. We choose an optimiser, and a loss function. Then the loop. Clear the old gradients, compute the loss, run backward, and step. Five epochs, and the loss falls each time. Then the operational part. We save only the weights, called the state dictionary, create a fresh network of the same shape, and load the weights into it. Finally we check that both networks give identical outputs.

import torch
from torch import nn
from churn import make_data

torch.manual_seed(0)
df = make_data(4000, seed=1)
X = torch.tensor(df.drop(columns="churn").to_numpy(), dtype=torch.float32)
X = (X - X.mean(0)) / X.std(0)
y = torch.tensor(df["churn"].to_numpy(), dtype=torch.float32).unsqueeze(1)

model = nn.Sequential(nn.Linear(4, 16), nn.ReLU(), nn.Linear(16, 1))
optimizer = torch.optim.Adam(model.parameters(), lr=0.05)
loss_fn = nn.BCEWithLogitsLoss()

for epoch in range(1, 6):
    optimizer.zero_grad()
    loss = loss_fn(model(X), y)
    loss.backward()
    optimizer.step()
    print(f"epoch {epoch}  loss {loss.item():.3f}")

torch.save(model.state_dict(), "weights.pt")
reloaded = nn.Sequential(nn.Linear(4, 16), nn.ReLU(), nn.Linear(16, 1))
reloaded.load_state_dict(torch.load("weights.pt"))
reloaded.eval()
with torch.no_grad():
    same = torch.allclose(model(X[:5]), reloaded(X[:5]))
print("reloaded model gives the same outputs:", same)

epoch 1  loss 0.712
epoch 2  loss 0.579
epoch 3  loss 0.489
epoch 4  loss 0.432
epoch 5  loss 0.401
reloaded model gives the same outputs: True

import numpy as np, pandas as pd

def make_data(n, seed, shifted=False):
    rng = np.random.default_rng(seed)
    tenure = rng.integers(1, 72, n)
    charges = rng.normal(65 + (28 if shifted else 0), 18, n).clip(20, 140).round(2)
    calls = rng.poisson(1.5 + (1.5 if shifted else 0), n)
    contract = rng.choice([0, 1, 2], n, p=[.55, .25, .20] if not shifted else [.30, .30, .40])
    logit = -1.0 - 0.04 * tenure + 0.025 * (charges - 65) + 0.45 * calls - 0.8 * contract
    if shifted:
        logit = -0.2 - 0.01 * tenure - 0.01 * (charges - 65) + 0.2 * calls - 0.2 * contract
    churn = (rng.random(n) < 1 / (1 + np.exp(-logit))).astype(int)
    return pd.DataFrame({"tenure": tenure, "charges": charges, "calls": calls, "contract": contract, "churn": churn})

Saving deep learning models safely

A few rules for saving. Save the weights, the state dictionary, plus the code that defines the architecture, rather than pickling a whole object, which is fragile. Record the framework version, with the file. Use safe formats, like safe tensors, for sharing, because loading a pickle can run code. For production serving, export to a portable, optimised format, like O N N X, or TorchScript. And checkpoint during training, with the optimiser state, so that you can resume.

Summary