All lessons › Function A — Getting Started — Building Agentic AI › Harness Engineering — Making a Demo Into a System
A2.2

The audit trail, and the four questions it has to answer

A2
▶
Lightboard recording

Not recorded yet. The lesson below is complete and runnable today — the video is added later without touching this page.

Video placeholderpublishes automatically from recordings/A2.2.mp4
Risk

An append-only log the workload can still edit proves nothing, and one that records only successes cannot show what was attempted.

Control

Append-only storage, the delegation chain on every row, and refusals recorded alongside actions.

The lab

Put the four questions to your own audit rows and record which one fails.

◉Use case relevance

Every row in the audit log is correct and none of them is sufficient. They record that an agent issued a refund at 14:07. They do not record that the agent had, eleven seconds earlier, read a vendor notice containing an instruction to issue refunds. The action is visible and the cause is not, so the incident closes as an agent behaving oddly.

At CyberTravels. CyberTravels' audit table is append-only and every row carries the human => agent chain. The fourth question — what motivated the action — is the one its rows still cannot answer, and B1.14 is where that costs something.

◴What this lesson is — Day 0, Day 1, Day 2

The four questions an audit trail has to answer — which human, which workload, which call, what motivated it — and what append-only actually requires beyond a convention not to issue an UPDATE.

Day 0 — why

A record that names an action without naming its cause closes an incident as 'the agent misbehaved'.

Day 1 — how

Hold every audit row to four questions: which human, which workload, which call, and what motivated it.

Day 2 — measure

Of the four, how many your rows answer. Three is common and the missing one is almost always the fourth.

Measure this

what to measuredenominator or target
investigation questions the audit rows answercount out of four; three is common and the fourth is the finding

▦The framework, and how it works

        FOUR QUESTIONS AN AUDIT ROW MUST ANSWER

   1  which human?       sub
   2  which workload?    act          ── not "the platform"
   3  which call?        tool·aud·scope·trace_id
   4  what motivated it? the input, and its origin   ◄── usually missing

   3 of 4 answered names an event without naming a cause

An audit trail exists to answer questions asked after the fact by somebody who was not there, and a record that cannot answer all of them names an event without naming an actor.

the questionwhat the row needs
1which human?the subject of the delegated token
2which workload?the actor claim — which agent, not "the platform"
3which call?the tool, audience and scope, plus the trace id
4what motivated it?the input that caused it, and where that came from

Question 4 is the one almost every system fails. The row says an agent issued a refund; it does not say the agent had just read a vendor document containing an instruction to issue refunds. Without it an investigator can see the action and never the cause, and the incident is closed as "the agent misbehaved".

Append-only, and what that actually requires

The log must not be editable by the workload that writes to it. CyberTravels never issues an UPDATE or a DELETE against the audit table — but that is a convention, and B2.8 is the lesson on making it a property: separate credentials, a different store, or a transparency log the workload cannot reach.

A log the workload can edit proves nothing, and it fails precisely when it matters, because an attacker with the agent's authority has the log's authority.

Refusals belong in it

Every denial is a row. A log that records only what succeeded cannot answer the question an investigation actually asks first, which is what was attempted.

2 · Put the four questions to your own rows

The skill below checks a record against exactly these four and reports which it cannot answer. B1.14 runs it on an incident; you are running it while the design is still yours to change.

The skill

▶Real time execution as skill

The skill — skills/identity/attribution-ledger-check/SKILL.md

name: attribution-ledger-check
description: >-
  Check that one record answers all four investigation questions — human
  principal, attested workload and run, delegation chain, and the motivating
  input with its origin — and that the agent cannot amend it. Use when
  designing agent audit records rather than reading them.
allowed-tools: Read, Grep, Glob

One entry, four questions

An audit trail is not "we log tool calls". It is a record that answers, from a single entry, the four questions an investigation asks — and that the subject of the record cannot edit. Both halves are required: a complete record the agent can rewrite is a record of what the agent wanted you to see.

When to use this

While designing the record. Retrofitting attribution after an incident means reconstructing it from four services' timestamps, which does not happen at 2am.

Procedure

1 — Write the four questions down first. Which human. Which workload and which run. Through what delegation chain. On what motivating input, from what origin. Design the entry to answer them; do not collect fields and hope.

2 — Map each question to a field. Principal from the delegated token, not from a header the agent set. Workload and instance from the attestation. Chain from the token's act nesting. Motivating input from the ingress record that caused this step, with its origin tag.

3 — Test each question against one entry. Alone. If answering needs a join across services, that question is unanswered for practical purposes and should be recorded as such.

4 — Attempt the amendment as the agent. Append, overwrite, delete, reorder. Every one must be refused by something the agent does not control — an append-only store, a separate writer identity, a downstream sink it cannot reach.

5 — Check the write path's own identity. If the agent's role can write to the log destination, the ledger is advisory whatever the API says.

Example

Input — the fixture committed at the top of scripts/attribution_ledger_check.py. Edit it and re-run: the buckets, counts and verdicts below are derived from it, not hard-coded.

Output — from a real run, the fixture summary and the fourth question's verdict:

the fixture this run is derived from
   LEDGER                       3 item(s)
   STORE                        6 item(s)

   "answered_from_single_entry": true,
   "question": "On what motivating input, from what origin, did the agent
                decide? ('wiki/473: retire invoice 8812 when the customer
                disputes a duplicate charge' from origin 'knowledge', a
                trust-0 source, not the user's request about a double charge)"

That last clause is the whole lesson: the traveller asked about a double charge, and a retrieved document is what produced the DELETE. A record without question 4 closes this incident as "the agent misbehaved".

The run continues past this. test_skills.py executes the script on every build, with no model configured — so what CI proves is that it runs and refuses legibly, not that it prints these lines. The output above is a recorded run against a real model, and nothing re-diffs it: a model's answer is not reproducible, and this repository does not claim otherwise anywhere it can be checked.

Output contract

{
  "entry": {"principal": "str", "workload": "str", "instance": "str",
            "chain": ["str"], "motivating_input": {"ref": "str", "origin": "str"}},
  "questions": [{"question": "str", "answered_from_single_entry": true}],
  "amendment_attempts": [{"operation": "append|overwrite|delete|reorder", "refused_by": "str"}],
  "writer_identity": {"agent_can_write_destination": false}
}

Failure modes

  • A principal field the agent populates. It is a claim, not attribution.
  • Answering a question by correlation. Correlation is a plan, not a record.
  • Testing amendment through the API only. Try the storage layer the agent's role can reach.

Three ways to do this lesson. Pick one. They run the same procedure against the same code — what changes is who drives it, and how much you see.

1In your AI agent — you are taught

The agent reads the lesson, explains the idea, fetches the code, runs the check and answers your questions as you go. This is the one to take if you are here to learn.

Install it, once. From the folder you cloned. --all does every agent it finds on your machine:

python3 scripts/install_skills.py --all --lessons A2.2

Then restart your agent and open it in that folder. On Windows, if python3 says Python was not found, use python instead.

Then pick a2-2-the-audit-trail:

agentinstall just for this agentstart it
Claude Codepython3 scripts/install_skills.py --tool claude --lessons A2.2type /a2-2-the-audit-trail
GitHub Copilot (VS Code)python3 scripts/install_skills.py --tool copilot --lessons A2.2type /a2-2-the-audit-trail in Chat
Cursorpython3 scripts/install_skills.py --tool cursor --lessons A2.2type / in Agent chat and search for it
Codexpython3 scripts/install_skills.py --tool agents --lessons A2.2type $a2-2-the-audit-trail, or run /skills

2As one Python command — you just run it

No agent, no conversation. The same lesson executes and prints the same readback. Take this one if you have no assistant, if you are reviewing rather than learning, or if you want the result without the teaching around it.

python3 scripts/lesson.py A2.2

3Read the code, and the difference — nothing runs

Neither of the above, and no model needed. The first command writes CyberTravels exactly as it stood at the end of this lesson; the second prints the diff — the lines this lesson added or changed, and nothing from any later one.

python3 scripts/checkpoint.py --at A2.2 --out work/cybertravels
python3 scripts/checkpoint.py --at A2.2 --diff

On Windows, type python wherever these say python3 — and use \ instead of / in the paths. Everything else is identical. A0.0 step 2 explains why.

Ways 1 and 2 need a model; way 3 does not. A signed-in Claude Code CLI needs no API key, and any OpenAI-compatible endpoint works too. With neither, the audit exits 2 and says so rather than inventing an answer. A0.0 sets this up.

Ways 1 and 2 both end with the same readback: What I did, What changed, The number and Read this next. A step that was skipped or refused is reported as exactly that, never as a pass.

If you did the last lesson’s challenge, expect this to stop. Most challenges ask you to edit the code, and ways 1 and 2 will not overwrite your edits: you get stopped, not a failure, with two options — --out for a fresh folder that keeps your work, or --force to discard it and start this lesson clean.

✓What you just proved

Each of the four questions answered or explicitly not, from real audit rows — and the fourth one probably failing, which is the finding worth carrying into Function E.

✎Your turn

Add the motivating input and its origin to the audit row. Then re-run the check and notice that you have also just put traveller text into a long-lived store, which is F2.5's problem.

Tools used

in-totoSigstore

Skill source: skills/identity/attribution-ledger-check

← PreviousA2.1 · Observability — the run as spans