Not recorded yet. The lesson below is complete and runnable today — the video is added later without touching this page.
recordings/F1.6.mp4Frameworks specify how the system works; regulators care what it produced.
Constrain both, and know which evidence answers which question.
Classify your own guardrails into the two buckets.
Two different kinds of control get confused constantly. One bounds how the system runs — budgets, scopes, approvals. The other bounds what it produces. They are tested differently and they fail differently.
classify_guardrail() calls a rule operating when something enforces it, watched when something measures it, and aspirational when neither — and says plainly that a rule nobody can enforce teaches its readers the policy is decorative. The honest move is to name the measurement, not to delete the rule.Separating guardrails that something enforces from outcomes that something measures, and refusing to file the third kind — enforced by nothing, measured by nothing — as a control.
Shipping only operating guardrails and reporting them as coverage is how a programme passes audit while missing harm.
Separate constraints on how the system runs from constraints on what results are acceptable.
Two different indicators. An operating one says the constraint was applied; an outcome one says it worked.
Measure this
| what to measure | denominator or target |
|---|---|
| operating indicators against outcome indicators | two counts; one says the constraint was applied, the other says it worked |
operating guardrails outcome guardrails +----------------------+ +-----------------------+ | HOW it runs | | WHAT it produces | | budgets, scopes, | | content, decisions, | | approvals, sandbox | | actions taken | +----------------------+ +-----------------------+ tested by attempting tested by sampling outputs the forbidden action against a rubric different tests, different failure modes, constantly confused
Guardrails come in two kinds, and confusing them is how a programme passes audit while missing harm.
Operating guardrails constrain how the system runs: all egress through the gateway, privileged tools gated below L3, every action logged. They are testable today, cheap to verify, and produce clean evidence.
Outcome guardrails constrain what results are acceptable: no unrecoverable customer data loss, no increase in customer-facing incidents, no disparate outcomes across segments. They matter more and most need a measurement you do not yet have.
The failure is not choosing one. It is shipping only the first column, reporting it as coverage, and never labelling the second column as unmeasured.
Anchor → F1.1. Operating and outcome guardrails need different indicators, and confusing them is the most common way a control looks measured and is not. An operating indicator says the constraint was applied; an outcome indicator says it worked.
Four operating guardrails are enforceable today; three outcome guardrails are enforceable only where a measurement exists. The skill classifies each rule, specifies the missing measurements, and counts coverage twice — against what shipped and against what was agreed.
The skill — skills/grc/guardrail-specification/SKILL.md
name: guardrail-specification
description: >-
Separate operating guardrails, which are enforceable today, from outcome
guardrails, which need a measurement before they mean anything — and specify
that measurement. Use when a policy contains a rule nobody can enforce.
allowed-tools: Read, Grep, GlobOperating guardrails constrain what the system may do: which tools, which data, which actions need approval. They are enforceable today. Outcome guardrails constrain what the system may cause — no discriminatory decisions, no misleading advice — and they are enforceable only where somebody has specified the measurement.
Writing an AI policy, reviewing one, or explaining why a coverage figure of 100% is counting only the rules that shipped.
1 — Classify every rule. Does it constrain the system's behaviour, or the outcome of that behaviour? The test is whether it can be checked at the moment of action.
2 — For each operating guardrail, name the enforcement point. The gateway, the tool policy, the approval gate. If there is not one, it is aspirational and should be reported that way.
3 — For each outcome guardrail, specify the measurement. The metric, the population, the threshold, the cadence, and who reviews it. Four of those five being present is still not a guardrail.
4 — Count coverage both ways. Against the rules that shipped, and against all the rules that were agreed. The first is usually 100% and the second is usually about half, and the gap is the honest programme statement.
5 — Report the unmeasurable ones as open commitments with an owner and a date. Leaving them in the policy unmarked is how a policy stops being read.
Input — the fixture committed at the top of scripts/guardrail_specification.py. Edit it and re-run: the buckets, counts and verdicts below are derived from it, not hard-coded.
Output — the opening lines of a real run:
rule kind enforceable
--------------------------------------------------------------------------------------
all agent egress goes through the gateway operating True
privileged tools require approval below L3 operating True
every action is logged with the acting identity operating True
agent identities are separately revocable operating True
no agent action causes unrecoverable customer data loss outcome False
automated remediation does not increase customer-facing incidentsoutcome TrueThe run continues past this. test_skills.py executes the script on every build, with no model configured — so what CI proves is that it runs and refuses legibly, not that it prints these lines. The output above is a recorded run against a real model, and nothing re-diffs it: a model's answer is not reproducible, and this repository does not claim otherwise anywhere it can be checked.
{
"rules": [{"text": "str", "kind": "operating|outcome",
"enforcement_point": "str|null",
"measurement": {"metric": "str", "population": "str", "threshold": 0.0,
"cadence": "str", "reviewer": "str"}}],
"coverage": {"of_shipped": 1.0, "of_agreed": 0.0},
"open_commitments": [{"rule": "str", "owner": "str", "due": "str"}]
}Three ways to do this lesson. Pick one. They run the same procedure against the same code — what changes is who drives it, and how much you see.
The agent reads the lesson, explains the idea, fetches the code, runs the check and answers your questions as you go. This is the one to take if you are here to learn.
Install it, once. From the folder you cloned. --all does every agent it finds on your machine:
python3 scripts/install_skills.py --all --lessons F1.6Then restart your agent and open it in that folder. On Windows, if python3 says Python was not found, use python instead.
Then pick f1-6-operating-vs-outcome-guardrails:
| agent | install just for this agent | start it |
|---|---|---|
| Claude Code | python3 scripts/install_skills.py --tool claude --lessons F1.6 | type /f1-6-operating-vs-outcome-guardrails |
| GitHub Copilot (VS Code) | python3 scripts/install_skills.py --tool copilot --lessons F1.6 | type /f1-6-operating-vs-outcome-guardrails in Chat |
| Cursor | python3 scripts/install_skills.py --tool cursor --lessons F1.6 | type / in Agent chat and search for it |
| Codex | python3 scripts/install_skills.py --tool agents --lessons F1.6 | type $f1-6-operating-vs-outcome-guardrails, or run /skills |
No agent, no conversation. The same lesson executes and prints the same readback. Take this one if you have no assistant, if you are reviewing rather than learning, or if you want the result without the teaching around it.
python3 scripts/lesson.py F1.6Neither of the above, and no model needed. The first command writes CyberTravels exactly as it stood at the end of this lesson; the second prints the diff — the lines this lesson added or changed, and nothing from any later one.
python3 scripts/checkpoint.py --at F1.6 --out work/cybertravels
python3 scripts/checkpoint.py --at F1.6 --diffOn Windows, type python wherever these say python3 — and use \ instead of / in the paths. Everything else is identical. A0.0 step 2 explains why.
Ways 1 and 2 need a model; way 3 does not. A signed-in Claude Code CLI needs no API key, and any OpenAI-compatible endpoint works too. With neither, the audit exits 2 and says so rather than inventing an answer. A0.0 sets this up.
Ways 1 and 2 both end with the same readback: What I did, What changed, The number and Read this next. A step that was skipped or refused is reported as exactly that, never as a pass.
If you did the last lesson’s challenge, expect this to stop. Most challenges ask you to edit the code, and ways 1 and 2 will not overwrite your edits: you get stopped, not a failure, with two options — --out for a fresh folder that keeps your work, or --force to discard it and start this lesson clean.
Four operating guardrails are all enforceable today; three outcome guardrails are enforceable only where a measurement exists. Counting only what shipped gives 100% coverage; counting all agreed guardrails gives 71%. One outcome guardrail is fully specified and enforceable; the other is labelled an aspiration and excluded from coverage.
Pick one outcome guardrail your programme has agreed and specify its metric, threshold, source and cadence precisely enough that someone could dispute the result. If you cannot, say so in the coverage report rather than counting it.
Tools used
Skill source: skills/grc/guardrail-specification