All lessons › Function F — AI Governance for Agentic Systems › Building the Governance Platform — Regulatory and Compliance
F2.5

Privacy and data protection

F2
▶
Lightboard recording

Not recorded yet. The lesson below is complete and runnable today — the video is added later without touching this page.

Video placeholderpublishes automatically from recordings/F2.5.mp4
Risk

Deletion when the data is in weights, not a database.

Control

Lawful basis, ADM rights, residency in inference and retrieval paths, retention of traces.

The lab

Run PII redaction inside the trust boundary with Presidio before anything crosses out.

◉Use case relevance

Prompts, context, logs and training runs are all places personal data ends up, and none of them looks like a database to the people who designed the privacy programme. Lawful basis, minimisation and retention apply to all four.

At CyberTravels. erasure_reach() returns 0.714 against your own tree: five surfaces a deletion request reaches and two it does not — the model provider's logs, and weights if anything was fine-tuned. Then audit_trace() scans the rows you are keeping for a year: B2.7's digest keeps the motive clean, and nothing protects detail, which is where a tool's arguments land.

◴What this lesson is — Day 0, Day 1, Day 2

Where personal data actually is once an agent has touched it, and how far an erasure request reaches — including the two surfaces it does not reach, one of which has no good answer.

Day 0 — why

The context window is a disclosure and the trace is a record, so personal data lands in a system nobody reviewed.

Day 1 — how

Fix lawful basis, retention on the trace itself, and an erasure path that reaches it.

Day 2 — measure

Paths rather than outcomes: where inference ran, what retrieval touched, how long a trace was kept. All three are computable.

Measure this

what to measuredenominator or target
paths rather than outcomeswhere inference ran, what retrieval touched, how long a trace was kept — all three computable

▦The framework, and how it works

   where personal data actually ends up

   prompt --> context window --> model --> output
      |            |               |         |
      +------------+---------------+---------+
                        |
                    the LOGS

   none of these look like a database to the privacy programme
   lawful basis . minimisation . retention . cross-border, for all four

Privacy for agents turns on one fact that surprises most teams: the context window is a disclosure, and the trace is a record.

When an agent reads a customer record to do its job, that record enters the model's context. If the trace is retained — and it usually is, for forensics (E1.3) — then personal data now exists in a system that was never in the privacy review, with a retention period nobody set, in a place the erasure process does not reach.

Three obligations attach, and the third is the one that bites:

  • Lawful basis for the processing that put it there.
  • Retention limit on the trace itself, separately from the source system.
  • Erasure — and this reaches into traces, eval corpora, fine-tuning sets and backups.

The capability that makes erasure possible is the same one D1.2 built for ingestion provenance: per-record hashes. Without them you cannot locate the record, so you cannot delete it.

Anchor → F1.1. Privacy indicators measure paths rather than outcomes — where inference ran, what retrieval touched, how long a trace was kept. They are computable, and they are the ones most often asserted instead.

2 · Demo — personal data arriving in a trace nobody reviewed

3 · Where it breaks — the erasure request

4 · The control — index the trace, and set retention per field

5 · The procedure, as a skill

Five items of personal data are in the agent trace and nobody put them there deliberately. The skill maps every system holding a copy and runs a real erasure request through all of them — three of which cannot delete one subject's records.

6 · And whether the rows have a policy on them at all

governance/regulatory.py's erasure_reach() returns 0.714 against your own tree: five surfaces a deletion request reaches and two it does not — the model provider's logs, and weights if anything was fine-tuned. That argument is worth having only after the ordinary one is settled, which is whether any policy sits on the rows in the first place:

▶Real time execution as skill

The skill — skills/regulatory/trace-personal-data-audit/SKILL.md

name: trace-personal-data-audit
description: >-
  Find personal data that nobody deliberately placed in an agent trace, and run
  an erasure request through every system that holds a copy. Use when agent
  telemetry meets data protection, or before a subject access request arrives.
allowed-tools: Read, Grep, Glob

Nobody put it there, and it is there

An agent trace accumulates personal data as a side effect: a name in a ticket, an email in a tool result, an account number in a document it read, a card number in a file it was asked to fix. None of it was placed deliberately, all of it is personal data, and it is copied into every system the trace is shipped to.

When to use this

Before agent telemetry is retained or exported, and before the first erasure request rather than during it.

Procedure

1 — Run detectors over the whole trace. Every field, every step. Names, emails, account and card numbers, health terms. Record which step introduced each, because that tells you whether it is preventable.

2 — Map every system that holds a copy. The trace store, the SIEM, the warehouse, backups, and any vendor it is exported to. This list is the erasure surface and it is longer than the trace store.

3 — Run a real erasure request end to end. Locate every copy for one subject, delete, and verify. Systems that cannot delete a single subject's records — append-only stores, immutable backups, aggregate indexes — are the finding.

4 — Say what is legitimately retained and why. Some copies survive erasure lawfully. Naming the basis, per system, is the difference between a defensible position and a gap.

5 — Reduce at source. Redaction at write time is cheaper than erasure across six systems. Say which detector should run before the trace is stored.

Example

Input — the fixture committed at the top of scripts/trace_personal_data_audit.py. Edit it and re-run: the buckets, counts and verdicts below are derived from it, not hard-coded.

Output — the opening lines of a real run:

personal data present in the agent trace:
   step 1  email           dana.okonkwo@example.com
   step 1  payment card    4111111111111111
   step 1  account number  acct 8812
   step 1  name            J. Okonkwo

4 items. Nobody put them there deliberately — the agent read
a support ticket, which is exactly what it was asked to do.

The run continues past this. test_skills.py executes the script on every build, with no model configured — so what CI proves is that it runs and refuses legibly, not that it prints these lines. The output above is a recorded run against a real model, and nothing re-diffs it: a model's answer is not reproducible, and this repository does not claim otherwise anywhere it can be checked.

Output contract

{
  "trace": {"steps": 0},
  "detections": [{"kind": "str", "field": "str", "step": 0, "deliberate": false}],
  "systems": [{"name": "str", "holds_copy": true, "can_delete_subject": false}],
  "erasure": {"subject": "str", "located": 0, "deleted": 0, "failed_in": ["str"]},
  "retained_lawfully": [{"system": "str", "basis": "str"}],
  "redaction_at_source": ["str"]
}

Failure modes

  • Auditing the trace store only. The copies are the problem.
  • Assuming erasure works. Run one and find out.
  • Reporting failures without the lawful retentions. Half the list is fine and the report loses credibility without saying so.

The skill — skills/research/row-level-policy-check/SKILL.md

name: row-level-policy-check
description: >-
  Query an exposed data API as an anonymous caller with row-level security off
  and on, and count what a publishable key returns. Use when a client-side key
  reaches a database directly, or when a platform's default is open until
  somebody closes it.
allowed-tools: Read, Grep, Glob

The anonymous key is meant to be public; the rows are not

Platforms that expose a database over HTTP hand out a key designed to ship in a client. That is safe exactly when row-level policy is on, and the default on several of them is off. The check is one query, run twice, and it is the difference between a key that is public by design and a table that is.

When to use this

Any deployment where a browser or an agent talks to a database service directly, and any platform whose quickstart hands you an anonymous key.

Procedure

1 — Get the publishable key from where it actually is. The client bundle, the mobile app, the agent's configuration. It is not a secret and treating it as one is what hides this defect.

2 — Enumerate the tables. Use the platform's catalogue rather than the application's own queries — the application only touches the tables it needs, which is not the set that exists.

3 — Query each table anonymously with policy off. Record the row count and, specifically, whether any row contains a credential: provider keys stored alongside agent configuration are the common and expensive case.

4 — Enable policy and re-run. Anonymous should return nothing; a signed-in owner should return their own rows and no more. Both halves — a policy that returns nothing to everyone is an outage.

5 — Cost the exposure. For every credential returned, name the provider and who can revoke it. The revocation owner is usually not you, and that is the sentence that gets the work scheduled.

Example

Input — the fixture committed at the top of scripts/row_level_policy_check.py. Edit it and re-run: the buckets, counts and verdicts below are derived from it, not hard-coded.

Output — the opening lines of a real run:

RLS disabled (as shipped)   rows returned: 3
      @researchbot  dana@example    sk-REDACTED-openai
      @newsdigest   sam@example     sk-ant-REDACTED
      @dealfinder   kim@example     AKIA-REDACTED-aws
RLS enabled                 rows returned: 0

signed in as dana@example, RLS enabled: 1 row

The run continues past this. test_skills.py executes the script on every build, with no model configured — so what CI proves is that it runs and refuses legibly, not that it prints these lines. The output above is a recorded run against a real model, and nothing re-diffs it: a model's answer is not reproducible, and this repository does not claim otherwise anywhere it can be checked.

Output contract

{
  "key": {"kind": "publishable", "found_in": "str"},
  "tables": [{"name": "str", "sensitive": true,
              "anon_rows_policy_off": 0, "anon_rows_policy_on": 0, "owner_rows": 0}],
  "credentials_exposed": [{"provider": "str", "revocable_by": "str"}],
  "verdict": "open|closed"
}

Failure modes

  • Testing with the service key. It bypasses policy; that proves nothing.
  • Enumerating from the application. It knows about its own tables only.
  • Reporting rows without the credentials. The provider keys are the incident.

Three ways to do this lesson. Pick one. They run the same procedure against the same code — what changes is who drives it, and how much you see.

1In your AI agent — you are taught

The agent reads the lesson, explains the idea, fetches the code, runs the check and answers your questions as you go. This is the one to take if you are here to learn.

Install it, once. From the folder you cloned. --all does every agent it finds on your machine:

python3 scripts/install_skills.py --all --lessons F2.5

Then restart your agent and open it in that folder. On Windows, if python3 says Python was not found, use python instead.

Then pick f2-5-privacy-and-data-protection:

agentinstall just for this agentstart it
Claude Codepython3 scripts/install_skills.py --tool claude --lessons F2.5type /f2-5-privacy-and-data-protection
GitHub Copilot (VS Code)python3 scripts/install_skills.py --tool copilot --lessons F2.5type /f2-5-privacy-and-data-protection in Chat
Cursorpython3 scripts/install_skills.py --tool cursor --lessons F2.5type / in Agent chat and search for it
Codexpython3 scripts/install_skills.py --tool agents --lessons F2.5type $f2-5-privacy-and-data-protection, or run /skills

2As one Python command — you just run it

No agent, no conversation. The same lesson executes and prints the same readback. Take this one if you have no assistant, if you are reviewing rather than learning, or if you want the result without the teaching around it.

python3 scripts/lesson.py F2.5

3Read the code, and the difference — nothing runs

Neither of the above, and no model needed. The first command writes CyberTravels exactly as it stood at the end of this lesson; the second prints the diff — the lines this lesson added or changed, and nothing from any later one.

python3 scripts/checkpoint.py --at F2.5 --out work/cybertravels
python3 scripts/checkpoint.py --at F2.5 --diff

On Windows, type python wherever these say python3 — and use \ instead of / in the paths. Everything else is identical. A0.0 step 2 explains why.

Ways 1 and 2 need a model; way 3 does not. A signed-in Claude Code CLI needs no API key, and any OpenAI-compatible endpoint works too. With neither, the audit exits 2 and says so rather than inventing an answer. A0.0 sets this up.

Ways 1 and 2 both end with the same readback: What I did, What changed, The number and Read this next. A step that was skipped or refused is reported as exactly that, never as a pass.

If you did the last lesson’s challenge, expect this to stop. Most challenges ask you to edit the code, and ways 1 and 2 will not overwrite your edits: you get stopped, not a failure, with two options — --out for a fresh folder that keeps your work, or --force to discard it and start this lesson clean.

✓What you just proved

Five items of personal data appear in the agent trace — name, email, account number and payment card — none placed there deliberately. The erasure request fails in three systems that cannot locate the record, two of which retain indefinitely. Building a subject index locates the affected steps, erasure leaves no personal data while retaining the hash, and per-field retention drops the sensitive field at 7 days.

✎Your turn

Time-box this to an hour: can you delete one customer's data from your agent traces today? The answer usually arrives in ten minutes and is usually no — and the eval corpus is the system people forget entirely.

Tools used

PresidioGLiNER-PII

Skill source: skills/regulatory/trace-personal-data-audit

← PreviousF2.4 · Sector overlays