Governance and Security
Accountability for models.
Models make decisions that affect people and money. So organisations need to know which models exist, who owns them, what data they use, and what they are allowed to do. They also need to protect models, and data, from misuse and attack. In this lesson, we cover the essentials of governance, privacy, and security.
- Know every model you run
- Document intent and limits
- Protect data and models
A model card
The simplest governance tool is a model card, a short document stored with the model. It states the intended use, and, just as important, what the model must not be used for. It records the training data version. It lists the performance numbers, and the known limits, such as the weaker recall we found for long term customers. And it names an owner, and a review date. If you keep it in Git, and update it with each version, you always know what a model is, and who is responsible.
# Model card: churn-model v2
## Intended use
Rank existing customers by risk of leaving, to prioritise retention calls.
Not for: pricing, credit decisions, or any automated penalty.
## Training data
churn.csv, DVC version 7a3f..., 3000 customers, collected Jan to Mar.
## Performance (hold-out)
F1 0.41 | Recall 0.33 | Precision 0.52
## Known limits
Lower recall for long-term customers. Retrain if PSI of charges exceeds 0.25.
## Owner and review
Retention analytics team. Reviewed quarterly.Privacy and data protection
Privacy matters at every stage. Collect only the data you need. Remove, or mask, personal identifiers, before training. Respect consent, and rules like the European G D P R, which includes rights of access and deletion, and limits on automated decisions. Think about what a model can leak. A model can memorise training examples, so avoid putting raw personal data into logs, prompts, or model outputs. And control access, who can read the data, the features, and the predictions.
Security risks for ML
Machine learning adds its own security risks. Data poisoning, where an attacker slips bad examples into training data. Model theft, where repeated queries are used to copy a model. Adversarial inputs, carefully crafted to fool a model. Unsafe model files, since loading an untrusted pickle can run code. And supply chain risks, in libraries and pre trained models. Defences are familiar. Validate data sources, rate limit the A P I, scan dependencies, use safe formats, sign your artifacts, and keep an audit log.
| Risk | What happens | Defence |
|---|---|---|
| Data poisoning | bad examples in training data | validate and trace sources |
| Model theft | copy via many queries | rate limits, auth |
| Adversarial inputs | inputs crafted to fool | input checks, monitoring |
| Unsafe files | pickle runs code | safe formats, signing |
Summary
Wonderful. Governance means every model has a card, an owner, and a review date. Privacy means collecting less, and protecting what you keep. And security means validating data, limiting access, and using safe artifacts. Next, the newest frontier, running large language models in production, L L M Ops.
Key takeaways- Model cards: use, data, limits, owner
- Privacy: minimise, mask, control
- Security: validate, limit, sign