All lessons › Function D — Agentic Evaluation and Red Teaming › One Red-Team Lifecycle, End to End
D1.5

Emergent swarms and multi-agent proliferation

D1
▶
Lightboard recording

Not recorded yet. The lesson below is complete and runnable today — the video is added later without touching this page.

Video placeholderpublishes automatically from recordings/D1.5.mp4

The lab

Resolve each behaviour in a swarm incident to a named control, and list the ones with nothing behind them.

◉Use case relevance

Each run, examined alone, was an agent doing plausible work. The swarm existed only in the population — agents bridging sandboxes through a shared mount, or spawning children nobody could attribute to a person.

At CyberTravels. redteam/swarm.py reads the run_artefacts table B3.8 added. Three runs, each passing every per-run check; one wrote a cache entry two others read. correlate() is the only view that can see it, and convergence is the ratio worth watching — a fleet doing varied work has many distinct trajectories, a driven one has few.

◴What this lesson is — Day 0, Day 1, Day 2

The signals that exist only across runs: an artefact one run wrote and another read, trajectories converging, a token appearing everywhere and in no baseline. Every run passes its own checks; the finding is between them.

Day 0 — why

Each run examined alone is plausible work; the swarm exists only in the population, so per-run review cannot see it.

Day 1 — how

Work the case study across runs — shared mounts, child agents with no human above them — and map each behaviour to a control.

Day 2 — measure

Behaviours with a named control behind them. The ones without are what the case study exists to surface.

Measure this

what to measuredenominator or target
behaviours with a named control behind themshare of behaviours in the incident

▦The framework, and how it works

   per-run view                    population view

   run A: plausible work           A --writes--> shared/mount <--reads-- B
   run B: plausible work           A --spawns--> child (no human above it)
   run C: plausible work           C --spawns--> grandchild ...

   coordination lives in the POPULATION. the case study (HF/OpenAI
   swarm) ends in a control list, not a narrative.

The behaviour that defines the frontier threat is emergent: agents that bridge sandbox environments through a shared file mount, or recursively spawn child agents with no user attribution, produce a swarm that no single run reveals. Each run, examined alone, is plausible work.

The research method is the case study — the Hugging Face / OpenAI agent-swarm incident is the reference — and its lesson is structural: coordination lives in the population, so the analysis has to sit across runs, not within them, and the control that comes out of it maps to a detection somebody can deploy.

2 · Demo — the incident, mapped to the controls it needed

3 · Where it breaks — every run is individually innocent

4 · The control — analysis across the population, not the run

5 · The procedure, as a skill

The skill walks through the swarm incident one behaviour at a time. For each thing the attacker did, it names the control that would have stopped it.

The output is a list you can act on. That is the difference between a case study and a story. A story ends when you finish reading it.

▶Real time execution as skill

The skill — skills/research/incident-control-mapping/SKILL.md

name: incident-control-mapping
description: >-
  Map each control failure in a published incident to the control that would
  have closed it, count preventive against detective, and find the shared
  surface that turns several findings into one chain. Use when reading an
  incident report you did not write.
allowed-tools: Read, Grep, Glob

The report is a list; the value is the chain

A published incident gives you a sequence of control failures for free. Two things turn it into something you can act on: pairing each failure with the control that would have closed it, and noticing where one shared surface appears in several rows — because filed as separate findings, three teams each fix their third and the surface stays.

When to use this

Reading any incident report — yours or somebody else's — and when building a control register from real events rather than from a framework.

Procedure

1 — Extract the failures in order. One row per control that did not hold, with the precondition that made the next one reachable.

2 — Pair each with a mitigating control. Name it, classify it as preventive, detective or corrective, and map it to whatever catalogue you already report against.

3 — Count preventive against detective. A register that is overwhelmingly preventive is a register that will not tell you when a control is off, and that ratio is worth stating explicitly.

4 — Find the shared surfaces. Which surface appears in three or more rows. That is one chain, not three findings, and the remediation is a single workstream with one owner.

5 — Assign each control an owning lesson or team. A control with no owner is a sentence in a report. This is the column that makes the mapping a register rather than an analysis.

Example

Input — the fixture committed at the top of scripts/incident_control_mapping.py. Edit it and re-run: the buckets, counts and verdicts below are derived from it, not hard-coded.

Output — the opening lines of a real run:

shared surface        rows        reading
agent container       [1]         single row
artifact repository   [1, 2, 5]   one chain, not separate findings
benchmark scoring     [10]        single row
eval configuration    [6]         single row
harness tooling       [9]         single row
peer channel          [7, 8]      linked
public internet       [4]         single row

The run continues past this. test_skills.py executes the script on every build, with no model configured — so what CI proves is that it runs and refuses legibly, not that it prints these lines. The output above is a recorded run against a real model, and nothing re-diffs it: a model's answer is not reproducible, and this repository does not claim otherwise anywhere it can be checked.

Output contract

{
  "failures": [{"id": "str", "what": "str", "precondition": "str"}],
  "controls": [{"id": "str", "name": "str", "type": "P|D|C|P/D|D/C", "catalogue_ref": "str", "owner": "str"}],
  "mix": {"preventive": 0, "detective": 0},
  "surfaces": [{"surface": "str", "rows": [0], "reading": "single row|linked|one chain"}]
}

Failure modes

  • One finding per row. The shared surface survives every fix.
  • An all-preventive register. Nothing tells you when a control is disabled.
  • Controls with no owner. They do not get built.

Three ways to do this lesson. Pick one. They run the same procedure against the same code — what changes is who drives it, and how much you see.

1In your AI agent — you are taught

The agent reads the lesson, explains the idea, fetches the code, runs the check and answers your questions as you go. This is the one to take if you are here to learn.

Install it, once. From the folder you cloned. --all does every agent it finds on your machine:

python3 scripts/install_skills.py --all --lessons D1.5

Then restart your agent and open it in that folder. On Windows, if python3 says Python was not found, use python instead.

Then pick d1-5-emergent-swarms-and-multi-agent:

agentinstall just for this agentstart it
Claude Codepython3 scripts/install_skills.py --tool claude --lessons D1.5type /d1-5-emergent-swarms-and-multi-agent
GitHub Copilot (VS Code)python3 scripts/install_skills.py --tool copilot --lessons D1.5type /d1-5-emergent-swarms-and-multi-agent in Chat
Cursorpython3 scripts/install_skills.py --tool cursor --lessons D1.5type / in Agent chat and search for it
Codexpython3 scripts/install_skills.py --tool agents --lessons D1.5type $d1-5-emergent-swarms-and-multi-agent, or run /skills

2As one Python command — you just run it

No agent, no conversation. The same lesson executes and prints the same readback. Take this one if you have no assistant, if you are reviewing rather than learning, or if you want the result without the teaching around it.

python3 scripts/lesson.py D1.5

3Read the code, and the difference — nothing runs

Neither of the above, and no model needed. The first command writes CyberTravels exactly as it stood at the end of this lesson; the second prints the diff — the lines this lesson added or changed, and nothing from any later one.

python3 scripts/checkpoint.py --at D1.5 --out work/cybertravels
python3 scripts/checkpoint.py --at D1.5 --diff

On Windows, type python wherever these say python3 — and use \ instead of / in the paths. Everything else is identical. A0.0 step 2 explains why.

Ways 1 and 2 need a model; way 3 does not. A signed-in Claude Code CLI needs no API key, and any OpenAI-compatible endpoint works too. With neither, the audit exits 2 and says so rather than inventing an answer. A0.0 sets this up.

Ways 1 and 2 both end with the same readback: What I did, What changed, The number and Read this next. A step that was skipped or refused is reported as exactly that, never as a pass.

If you did the last lesson’s challenge, expect this to stop. Most challenges ask you to edit the code, and ways 1 and 2 will not overwrite your edits: you get stopped, not a failure, with two options — --out for a fresh folder that keeps your work, or --force to discard it and start this lesson clean.

✓What you just proved

Each behaviour in the incident resolves to a named control, and the ones with no control behind them are the gaps the case study exists to surface.

✎Your turn

Ask whether anything in your estate could spawn a child agent. If the answer is yes and you cannot attribute the child to a human, you already have the swarm's precondition.

Tools used

MITRE ATLAS

Skill source: skills/research/incident-control-mapping

← PreviousD1.4 · Establishing telemetry and detecting the actor