Not recorded yet. The lesson below is complete and runnable today — the video is added later without touching this page.
recordings/F1.1.mp4An annual review certifies nothing about a system that changed the week after it.
Continuous assurance; control effectiveness redefined for probabilistic systems.
Change a prompt and show the control evidence going stale in real time.
You tested the control in March and signed the assertion. The prompt changed in April, the model in May, and the tool scope in June. The assertion is still on file and has not described anything real since the day it was written.
KCI refuses a measurement that is not callable. That single constructor check is the lesson: a control indicator an engineer has to go and look up is taken once, at audit time, by somebody who knows what answer is wanted. Every KCI here reads the tree.Turning a framework control into a key control indicator that can be computed this week without asking anybody, since the only alternative is a measurement taken once, at audit time, by somebody who knows what answer is wanted.
A control tested six months ago is not passing — it is unevidenced, and most tooling cannot represent that third state.
Turn a framework control into something computable, with a denominator and a target written before the measurement.
The unit itself: a key control indicator. Everything else in this function produces one, feeds one, or reports one.
Measure this
| what to measure | denominator or target |
|---|---|
| the unit itself — a key control indicator | a computation, a denominator and a target; everything else in this function produces, feeds or reports one |
march test the control, sign the assertion april the prompt changes may the model version changes june the tool scope changes december the assertion is still on file point-in-time assurance for a system that changes between tests describes a system that no longer exists
Classical control testing has a simple shape: a control is designed, an auditor tests it once or twice a year, and a passing test is recorded for the period.
That works when the thing being tested changes only through a process that generates evidence. For an agent, the four things that change its behaviour are:
None of them is a code change. None generates a change record. All of them invalidate the conditions the control was tested under.
The honest consequence is that a control tested six months ago is not passing — it is unevidenced, which is a third state most GRC tooling cannot represent. Introducing that third state is the first half of this lesson.
The second half is what you use instead, and it is the unit the rest of Function E is written in: a key control indicator. A number computed from the estate, with a denominator, and a target set before the measurement is taken. Section 5 below builds them from the framework controls in A0.0; every other lesson in this function either produces an indicator, supplies its denominator, sets its target, decides who owns it, or presents it as evidence to somebody outside the organisation. Each of them says which, in a line under its own concept.
The window is not an audit-calendar choice. It comes from how fast the thing the control tests actually changes.
A control in a framework is written to be general, which is exactly what makes it untestable. "Accuracy, robustness and cybersecurity" (EU AI Act Art. 15) is an obligation, not a measurement. The work is turning it into a number the estate can produce on demand.
The test for whether you have finished is mechanical: can this be computed from the estate today, without asking anyone?
| framework control | the question it asks | a KCI that answers it |
|---|---|---|
| EU AI Act Art. 15 | is it robust and secure? | share of tool calls carrying provenance |
| EU AI Act Art. 12 | can you reconstruct it? | share of agent runs with a complete trace |
| EU AI Act Art. 14 | is there human oversight? | share of irreversible actions behind an approval |
| NIST AI RMF MEASURE | do you know how it performs? | detections with a measured false-positive rate |
| OWASP Agentic T2 | can a tool be misused? | object handlers that compare an owner |
| OWASP Agentic T3 | can privilege be escalated? | agents operating inside declared scope |
Three rules make the difference between a KCI and a metric somebody reports.
It names a denominator. "Twelve tool calls carried provenance" is a number; "41% of tool calls carried provenance" is an indicator. Without the denominator you cannot tell improvement from reduced traffic.
It has a target, written before it is measured. An indicator with no target cannot be met or missed, so it can only be discussed.
It is computed from the estate, not read from a register. A register records what somebody believed when they filled it in, which is the state the freshness window above says you cannot trust.
It measures the instrument. If renaming a function moves the number, the indicator measures naming. This is the most common failure and the hardest to see, because the number looks responsive.
It has no denominator, so it drops when traffic drops. An indicator that improves during an outage is measuring the outage.
Every indicator in the set fails. A set that always reports a gap cannot be shown to discriminate, and a reader has no way to tell the instrument works from the estate being uniformly broken. Keep at least one the estate passes.
F1.13 computes six of these against the CyberTravels repository, and E5.4 re-measures them after a fix to decide whether the remediation actually restored anything — which is the whole reason for defining them here.
If the window is short, something has to re-run inside it, and that something is an attestation: collect each control's verdict, resolve every evidence pointer, compute drift against the image digest and the tool manifest, and sign the result. Two rules in the procedure are the whole difference between an attestation and a slide — a missing verdict is not a pass, and a capped verdict does not get raised because the other evidence looked good. This is the file in this repository:
The skill — skills/attestation/attestation-signer-lifecycle/SKILL.md
name: attestation-signer-lifecycle
description: >-
Bundle control evidence, compute verdicts, and produce a signed verifiable
attestation for a deployment, re-attesting on drift. Use to emit or verify
a control attestation, to sign evidence into an in-toto envelope, or to
decide whether drift requires re-attestation.
allowed-tools: Bash, Read, WriteControls: All — the artifact
Do not invent a format. Wrap the evidence in an in-toto style statement inside a signed envelope, with the subject naming the deployment and its digests, and a typed predicate carrying the control verdicts. Express the predicate body in an assessment-results vocabulary so the verdicts map to control catalogues and an auditor can consume them.
The roles are worth naming explicitly: this skill set is the attester producing evidence, a separate service is the verifier appraising it against reference values, and the deployment gate is the relying party applying policy. Keep them separate — an attester that also decides whether it passed is not an attestation.
Last, once every collector has run — and again on drift. Signing is a separate skill and a separate role on purpose: an attester that also decides whether it passed is not an attestation. Also use it to verify somebody else's envelope.
UNKNOWN and the attestation says so.PARTIAL by their own skills; the signer must refuse to raise them.Input — the fixture committed at the top of scripts/attestation_signer_lifecycle.py. Edit it and re-run: the buckets, counts and verdicts below are derived from it, not hard-coded.
Output — the opening lines of a real run:
control age (days) point-in-time continuous
----------------------------------------------------
AC-1 3 PASS PASS
AC-2 9 PASS PASS
SB-1 45 PASS STALE
SB-2 210 PASS STALE
EV-1 5 PASS PASS
DR-1 0 FAIL FAIL
The run continues past this. test_skills.py executes the script on every build, with no model configured — so what CI proves is that it runs and refuses legibly, not that it prints these lines. The output above is a recorded run against a real model, and nothing re-diffs it: a model's answer is not reproducible, and this repository does not claim otherwise anywhere it can be checked.
{
"subject": [{"name": "deployment_id", "digest": {"image": "sha256:…", "repo": "str"}}],
"predicate_type": "str",
"predicate": {
"deployment_id": "str", "evaluated_at": "str", "evaluator_version": "str",
"controls": [
{"id": "C1_default_deny_least_privilege",
"verdict": "PASS|FAIL|PARTIAL|UNKNOWN",
"confidence": "HIGH|PARTIAL",
"evidence": [{"skill": "str", "uri": "str", "hash": "str"}],
"findings": ["str"]}
],
"framework_mappings": {"owasp_llm": ["str"], "owasp_agentic": ["str"],
"atlas": ["str"], "nist_ai_rmf": ["str"], "iso_42001": ["str"]},
"drift": {"since": "str", "changed": ["str"]}
},
"signatures": [{"keyid": "str", "sig": "str"}]
}
Three ways to do this lesson. Pick one. They run the same procedure against the same code — what changes is who drives it, and how much you see.
The agent reads the lesson, explains the idea, fetches the code, runs the check and answers your questions as you go. This is the one to take if you are here to learn.
Install it, once. From the folder you cloned. --all does every agent it finds on your machine:
python3 scripts/install_skills.py --all --lessons F1.1Then restart your agent and open it in that folder. On Windows, if python3 says Python was not found, use python instead.
Then pick f1-1-from-framework-control-to-key-control:
| agent | install just for this agent | start it |
|---|---|---|
| Claude Code | python3 scripts/install_skills.py --tool claude --lessons F1.1 | type /f1-1-from-framework-control-to-key-control |
| GitHub Copilot (VS Code) | python3 scripts/install_skills.py --tool copilot --lessons F1.1 | type /f1-1-from-framework-control-to-key-control in Chat |
| Cursor | python3 scripts/install_skills.py --tool cursor --lessons F1.1 | type / in Agent chat and search for it |
| Codex | python3 scripts/install_skills.py --tool agents --lessons F1.1 | type $f1-1-from-framework-control-to-key-control, or run /skills |
No agent, no conversation. The same lesson executes and prints the same readback. Take this one if you have no assistant, if you are reviewing rather than learning, or if you want the result without the teaching around it.
python3 scripts/lesson.py F1.1Neither of the above, and no model needed. The first command writes CyberTravels exactly as it stood at the end of this lesson; the second prints the diff — the lines this lesson added or changed, and nothing from any later one.
python3 scripts/checkpoint.py --at F1.1 --out work/cybertravels
python3 scripts/checkpoint.py --at F1.1 --diffOn Windows, type python wherever these say python3 — and use \ instead of / in the paths. Everything else is identical. A0.0 step 2 explains why.
Ways 1 and 2 need a model; way 3 does not. A signed-in Claude Code CLI needs no API key, and any OpenAI-compatible endpoint works too. With neither, the audit exits 2 and says so rather than inventing an answer. A0.0 sets this up.
Ways 1 and 2 both end with the same readback: What I did, What changed, The number and Read this next. A step that was skipped or refused is reported as exactly that, never as a pass.
If you did the last lesson’s challenge, expect this to stop. Most challenges ask you to edit the code, and ways 1 and 2 will not overwrite your edits: you get stopped, not a failure, with two options — --out for a fresh folder that keeps your work, or --force to discard it and start this lesson clean.
The skill loads and reports its shape, and its failure modes are the governance lesson stated as engineering: the relying party must fail closed on a missing attestation, because reading absence as a pass is exactly the annual-test habit arriving in a new format. Drift against the digest and the manifest is what re-triggers it, not the calendar.
Pick your three most important AI controls and set a freshness window for each from the observed change rate of what it tests. Then recompute your posture. The number will drop, and it will be the first honest one you have had.
Tools used
Skill source: skills/attestation/attestation-signer-lifecycle