Privacy and Guardrails

Customers paste everything into a chat box. Card numbers, addresses, and health details. If all of that gets sent off to a third party, or stored in your logs, that's a legal problem, and a broken promise. So let's add some guardrails.

• People paste everything
• Cards, addresses, health
• A legal problem

Redacting private data

The first guardrail runs before the model sees anything. Using patterns, we replace emails, phone numbers, and card numbers with labels. Run it. The message still makes sense, but the private details are gone. Remember our regex lesson? For names and addresses, you'd add an NER model. This simple step reduces your risk enormously.

import re

patterns = {
    "EMAIL": r"[\w.]+@[\w.]+\w",
    "PHONE": r"\b\d{3}[- ]\d{4}\b",
    "CARD": r"\b(?:\d[ -]?){13,16}\b",
}

def redact(text):
    for name, p in patterns.items():
        text = re.sub(p, f"[{name}]", text)
    return text

print(redact("I'm sam@mail.com, call 555-0192, card 4111 1111 1111 1111."))

I'm [EMAIL], call [PHONE], card [CARD].

Checking the output

The second guardrail runs after the model replies, and before the customer sees anything. We check for things that it should never do, like promising a discount, or leaking an email. Run it. The normal reply passes. The discount promise gets blocked. In a real system, that would trigger a safe fallback reply, or send the conversation to a person.

import re

def check_output(reply):
    problems = []
    if re.search(r"\b\d+% (off|discount)\b", reply, re.I):
        problems.append("promised a discount")
    if re.search(r"[\w.]+@[\w.]+\w", reply):
        problems.append("contains an email address")
    if len(reply) > 300:
        problems.append("too long")
    return problems

for reply in ["Your order ships tomorrow.", "Good news, you get 100% off!"]:
    print(reply, "->", check_output(reply) or "OK")

Your order ships tomorrow. -> OK
Good news, you get 100% off! -> ['promised a discount']

Bias in language

Language models learn from human writing, along with its bias. We ask a pretrained model to fill in a blank, after the doctor, and after the nurse. Run it. Look at the pronouns that it picks. The model has absorbed a stereotype from its training text. So always test how your system treats different groups, particularly when it's making decisions about people.

from transformers import pipeline

fill = pipeline("fill-mask", model="distilbert-base-uncased", device="cpu")

for text in ["The doctor said that [MASK] was busy.",
             "The nurse said that [MASK] was busy."]:
    top = fill(text)[:2]
    print(text, [(r["token_str"], round(r["score"], 2)) for r in top])

The doctor said that [MASK] was busy. [('he', 0.03), ('she', 0.03)]
The nurse said that [MASK] was busy. [('she', 0.06), ('everyone', 0.04)]

Summary