All lessons › Function F — AI Governance for Agentic Systems › Running the Programme — the CISO Office
F3.7

Building the capability

F3
▶
Lightboard recording

Not recorded yet. The lesson below is complete and runnable today — the video is added later without touching this page.

Video placeholderpublishes automatically from recordings/F3.7.mp4
Risk

Hiring for conceptual familiarity instead of practice.

Control

Interview questions that separate the two; internal transition paths.

The lab

Write the interview loop for an agentic security engineer.

◉Use case relevance

You cannot hire this capability at the rate you need it, so most of it has to be built. Role definitions and honest ramp expectations are what stop that becoming an eighteen-month disappointment.

At CyberTravels. build_order() starts with the inventory and puts the red team last, because its findings need somewhere to land — that is D1.11's handoff read as a hiring plan. Call it with demos_first=True and it returns what that produces: striking findings with nowhere to land, and a coverage number nobody computed.

◴What this lesson is — Day 0, Day 1, Day 2

A build order in which each stage's output is the next stage's input, and the comparison with the order that produces demos — including why the red team comes last.

Day 0 — why

Hiring for conceptual familiarity produces a team that can discuss the problem and not measure it.

Day 1 — how

Staff for the ability to turn a control into something computable, and accept an unglamorous first quarter.

Day 2 — measure

What the team can evidence, quarter by quarter. Quarter one produces an inventory and no dashboard, which is the right shape.

Measure this

what to measuredenominator or target
what the team can evidence, quarter by quarterper quarter; quarter one produces an inventory and no dashboard, which is the right shape

▦The framework, and how it works

   you cannot hire this at the rate you need it

   hire            2-3 people who have done it
   convert         AppSec, IAM, detection engineers
   ramp            6-9 months to independent, honestly

   role definitions first, or you interview for a job nobody can describe

Building the capability is a sequencing and hiring question, and the honest version accounts for what you can evidence rather than what you can present.

The curve is deliberately unglamorous early. Quarter one produces an inventory and identities — no dashboard, nothing to demo — and quarter three finally produces numbers anyone outside the team finds interesting.

Programmes that invert it, doing evaluation and dashboards first, report high numbers early and then spend a year discovering they cannot switch anything off (F3.3). The inverted version is easier to fund and produces a capability that fails its first real incident.

The four roles from F3.4 map onto the quarters, and the hire that unblocks the most others is the identity engineer — which is not the one most teams hire first.

Anchor → F1.1. The build curve is unglamorous early precisely because indicators are: quarter one produces the denominator and nothing to demo. The capability being hired for is turning a control into something computable, which is what separates operating a GRC tool from measuring an estate.

2 · Demo — four quarters, measured by evidenced coverage

3 · Where it breaks — the fundable order

4 · The control — hire for the bottleneck, not the demo

5 · The procedure, as a skill

Two orders, the same work. The skill simulates both quarter by quarter: the correct one climbs to full coverage with nothing demoable until Q3, and the inverted one leads on capability for three quarters and finishes lower.

▶Real time execution as skill

The skill — skills/programme/capability-build-order/SKILL.md

name: capability-build-order
description: >-
  Compare a build order that produces control coverage against one that produces
  demos, quarter by quarter, and state what each has at the end of every period.
  Use when planning what to build first with a fixed team.
allowed-tools: Read, Grep, Glob

Nothing demoable until Q3, and 100% coverage at Q4

Two orders, the same work. One builds the controls in dependency order and produces coverage; the other builds the visible things first and produces capabilities. The second looks better for three quarters and finishes lower, and the only way to survive the first is to say in advance that it will look like that.

When to use this

Planning a multi-quarter build with a fixed team, and defending a plan whose early quarters have nothing to show.

Procedure

1 — List the required controls and what each depends on. Coverage is measured against this list, so it has to exist before either order can be scored.

2 — Mark which controls are demoable. Some produce something a stakeholder can see; most do not. This is the honest input to the tension rather than a complaint about it.

3 — Simulate the dependency-respecting order. Coverage per quarter and what is demoable per quarter. Expect a flat, invisible start.

4 — Simulate the demo-first order. It will lead for several quarters on capability and trail on coverage, and it will end lower because later controls block on earlier ones.

5 — Publish both, with the Q1 and Q2 warning explicit. A plan that predicts its own quiet period survives it; one that does not gets re-planned in month four into the other order.

Example

Input — the fixture committed at the top of scripts/capability_build_order.py. Edit it and re-run: the buckets, counts and verdicts below are derived from it, not hard-coded.

Output — the opening lines of a real run:

quarter                         coverage   demoable
------------------------------------------------------
Q1 · inventory + identity            25%          0
Q2 · containment                     50%          0
Q3 · evidence + evaluation           75%          1
Q4 · continuous + stop              100%          3

Q1 and Q2 produce nothing demoable. That is the political problem, and

The run continues past this. test_skills.py executes the script on every build, with no model configured — so what CI proves is that it runs and refuses legibly, not that it prints these lines. The output above is a recorded run against a real model, and nothing re-diffs it: a model's answer is not reproducible, and this repository does not claim otherwise anywhere it can be checked.

Output contract

{
  "required": ["str"],
  "demoable": {"str": false},
  "orders": [{"name": "str",
              "quarters": [{"q": "str", "built": ["str"], "coverage": 0.0, "demoable": 0}],
              "final_coverage": 0.0, "final_capabilities": 0}],
  "warning": "str"
}

Failure modes

  • Building demoable things first. It finishes lower.
  • Not warning about the quiet quarters. The plan gets replaced in month four.
  • Coverage against a list that does not exist yet. Write the required set first.

Three ways to do this lesson. Pick one. They run the same procedure against the same code — what changes is who drives it, and how much you see.

1In your AI agent — you are taught

The agent reads the lesson, explains the idea, fetches the code, runs the check and answers your questions as you go. This is the one to take if you are here to learn.

Install it, once. From the folder you cloned. --all does every agent it finds on your machine:

python3 scripts/install_skills.py --all --lessons F3.7

Then restart your agent and open it in that folder. On Windows, if python3 says Python was not found, use python instead.

Then pick f3-7-building-the-capability:

agentinstall just for this agentstart it
Claude Codepython3 scripts/install_skills.py --tool claude --lessons F3.7type /f3-7-building-the-capability
GitHub Copilot (VS Code)python3 scripts/install_skills.py --tool copilot --lessons F3.7type /f3-7-building-the-capability in Chat
Cursorpython3 scripts/install_skills.py --tool cursor --lessons F3.7type / in Agent chat and search for it
Codexpython3 scripts/install_skills.py --tool agents --lessons F3.7type $f3-7-building-the-capability, or run /skills

2As one Python command — you just run it

No agent, no conversation. The same lesson executes and prints the same readback. Take this one if you have no assistant, if you are reviewing rather than learning, or if you want the result without the teaching around it.

python3 scripts/lesson.py F3.7

3Read the code, and the difference — nothing runs

Neither of the above, and no model needed. The first command writes CyberTravels exactly as it stood at the end of this lesson; the second prints the diff — the lines this lesson added or changed, and nothing from any later one.

python3 scripts/checkpoint.py --at F3.7 --out work/cybertravels
python3 scripts/checkpoint.py --at F3.7 --diff

On Windows, type python wherever these say python3 — and use \ instead of / in the paths. Everything else is identical. A0.0 step 2 explains why.

Ways 1 and 2 need a model; way 3 does not. A signed-in Claude Code CLI needs no API key, and any OpenAI-compatible endpoint works too. With neither, the audit exits 2 and says so rather than inventing an answer. A0.0 sets this up.

Ways 1 and 2 both end with the same readback: What I did, What changed, The number and Read this next. A step that was skipped or refused is reported as exactly that, never as a pass.

If you did the last lesson’s challenge, expect this to stop. Most challenges ask you to edit the code, and ways 1 and 2 will not overwrite your edits: you get stopped, not a failure, with two options — --out for a fresh folder that keeps your work, or --force to discard it and start this lesson clean.

✓What you just proved

The correct order climbs 25% → 50% → 75% → 100% coverage with nothing demoable until Q3. The inverted order reaches 75% after four quarters with 3 capabilities against the correct order's 5, and still cannot halt the fleet. The identity engineer unblocks the most capabilities and is named as the first hire.

✎Your turn

Map your existing team onto the four roles. Most organisations have three of them under other names and are missing the identity one entirely — which is also the one that unblocks everything else.

Skill source: skills/programme/capability-build-order

← PreviousF3.6 · Saying no, and saying yes with conditions