Privacy

In the nineties, a researcher showed that most Americans could be uniquely identified by only three facts. Zip code, birth date, and sex. She found the medical records of a state governor, in a dataset that was supposed to be anonymous. Removing names doesn't make data anonymous.

• Three facts identify most people
• She found a governor's records
• Removing names isn't enough

How unique are your customers?

We've got no names here. Just city, age, and visits per month. Surely that's safe? We count how many people are the only one with their exact details. Then we use coarser groups, with age bands, and city only. Run it. Over four thousand of our customers are unique with their exact details. With bands, that nearly disappears. This idea's called k anonymity. Everyone should look identical to at least k other people. A smallest group of one still fails that test, so you'd need coarser bands, or you'd hide that group.

import pandas as pd

df = pd.read_csv("customers.csv").drop_duplicates()
df["city"] = df["city"].str.strip().str.title()
df["age_band"] = (df["age"] // 10 * 10)

exact = df.groupby(["city", "age", "visits_per_month"]).size()
bands = df.groupby(["city", "age_band"]).size()

print(f"Unique people, exact details: {(exact == 1).sum():,}")
print(f"Unique people, in bands:      {(bands == 1).sum():,}")
print(f"Smallest group in bands:      {bands.min()}")

Unique people, exact details: 4,170
Unique people, in bands:      1
Smallest group in bands:      1

Handling personal data

Follow a few rules. Collect only what you need. Remove direct identifiers, and hash the keys. Restrict who can see what. Delete data when you've finished with it. Know the law where you work, such as GDPR in Europe. And never share customer data with an outside tool without permission, and that includes AI chat assistants.

Summary