Fairness

A model can have a great overall score, and still treat some groups of people badly. If it decides who gets a loan, a job interview, or a retention offer, that matters a great deal. Fairness checking is part of operating a model responsibly. In this lesson, you will measure it, with real numbers.

• A good average can hide harm
• Check every important group
• Measure, then decide

Where unfairness comes from

Unfairness usually comes from one of four places. The training data may under represent some groups, so the model learns less about them. The labels may carry old human biases, because the model copies the past. The features may act as stand ins for sensitive facts, like a postcode that hints at ethnicity. And the way the model is used may differ across groups. Removing the sensitive column does not fix this, because other columns can still carry the signal.

Measuring a gap

Here is a simple fairness check. We train a model, and score a test set. Then we split the customers into two groups, new customers, and long term customers. For each group, we print two numbers. The share of customers the model flags as at risk, and the recall, meaning, of those who really leave, how many the model caught. Then we compare the flag rates, as a ratio. A common rule of thumb says the ratio should be above point eight. Here, the model catches far fewer of the long term customers who leave. That is a gap we must investigate, and either fix, or consciously accept, and document.

import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import recall_score
from churn import make_data

train, test = make_data(4000, seed=1), make_data(2000, seed=2)
features = ["tenure", "charges", "calls", "contract"]
model = RandomForestClassifier(n_estimators=100, max_depth=6, random_state=0)
model.fit(train[features], train["churn"])

test["flagged"] = model.predict(test[features])
test["group"] = np.where(test["tenure"] < 12, "new customers", "long-term")

rates = {}
for name, g in test.groupby("group"):
    rates[name] = g["flagged"].mean()
    recall = recall_score(g["churn"], g["flagged"])
    print(f"{name:14s} flagged {g['flagged'].mean():.2f}   recall {recall:.2f}")

ratio = min(rates.values()) / max(rates.values())
print("selection-rate ratio:", round(ratio, 2), "->", "review needed" if ratio < 0.8 else "within the 0.8 rule")

long-term      flagged 0.01   recall 0.05
new customers  flagged 0.09   recall 0.22
selection-rate ratio: 0.13 -> review needed

import numpy as np, pandas as pd

def make_data(n, seed, shifted=False):
    rng = np.random.default_rng(seed)
    tenure = rng.integers(1, 72, n)
    charges = rng.normal(65 + (28 if shifted else 0), 18, n).clip(20, 140).round(2)
    calls = rng.poisson(1.5 + (1.5 if shifted else 0), n)
    contract = rng.choice([0, 1, 2], n, p=[.55, .25, .20] if not shifted else [.30, .30, .40])
    logit = -1.0 - 0.04 * tenure + 0.025 * (charges - 65) + 0.45 * calls - 0.8 * contract
    if shifted:
        logit = -0.2 - 0.01 * tenure - 0.01 * (charges - 65) + 0.2 * calls - 0.2 * contract
    churn = (rng.random(n) < 1 / (1 + np.exp(-logit))).astype(int)
    return pd.DataFrame({"tenure": tenure, "charges": charges, "calls": calls, "contract": contract, "churn": churn})

Common fairness measures

There are several ways to define fair, and they can conflict. Demographic parity asks that each group is selected at the same rate. Equal opportunity asks that, among people who deserve a positive outcome, each group is found at the same rate. Predictive parity asks that a positive prediction is equally reliable for each group. You usually cannot satisfy all at once, so the choice is a decision about values, made with the business, legal, and affected people, not only by the data scientist.

Summary