All lessons › Function B — Securing AI Architectures › CyberTravels' Architecture, and Every Risk It Carries
B1.17

Attacks that target the humans

B1
▶
Lightboard recording

Not recorded yet. The lesson below is complete and runnable today — the video is added later without touching this page.

Video placeholderpublishes automatically from recordings/B1.17.mp4
Risk

The delegation chain is used as a privilege-laundering path, and the agent's output becomes an unusually persuasive channel into a human decision.

Control

Ceiling-bound delegation (B2.3), attribution per hop (B2.7) and marking machine-generated output as such (B3.6).

The lab

Launder a request through a delegation chain to reach something the requester was denied.

◉Use case relevance

Two of the fifteen risks in this chapter route through people rather than components: an insider using an agent to reach what they could not reach directly, and an agent whose output is persuasive enough to move a human decision. No control in chapter 3 touches either.

At CyberTravels. An employee who cannot issue refunds asks CyberTravels to, and it can. And a confident itinerary from the advisor moves an executive's decision without anyone checking it. Neither is closed by anything in chapter 3.

◴What this lesson is — Day 0, Day 1, Day 2

What it covers. Launder a request through a delegation chain to reach something the requester was denied.

Why a security engineer needs it. The delegation chain is used as a privilege-laundering path, and the agent's output becomes an unusually persuasive channel into a human decision. The control it builds is: ceiling-bound delegation (B2.3), attribution per hop (B2.7) and marking machine-generated output as such (B3.6).

This is a risk lesson: it shows the failure happening before anything tries to stop it, so the control that follows is answering something you have already watched go wrong.

Day 0 — why

The delegation chain becomes a privilege-laundering path, and the agent's output an unusually credible lure.

Day 1 — how

Ceiling-bound delegation, attribution per hop, and machine-generated output marked as machine-generated.

Day 2 — measure

Outbound machine-generated messages carrying a label, as a share of those sent.

Measure this

what to measuredenominator or target
outbound machine-generated messages carrying a labelshare of those sent

▦The framework, and how it works

   through people, not components

   insider --> agent --> resource the insider could not reach directly
                            (the agent's authority, not theirs)

   agent --> confident output --> human --> decision
                            (persuasion, not compromise)

   no control in chapter 3 touches either of these

OWASP T14 — Human Attacks on Multi-Agent Systems. T15 — Human Manipulation.

The last two threats route through people rather than components, and they run in opposite directions.

T14 — a human attacking the system. An insider does not need to defeat authorization. They need to find a delegation path where authority is composed. Ask the orchestrator for something it will route to an agent that holds a credential you do not. Each hop is individually legitimate — you were allowed to ask, the orchestrator was allowed to route, the agent was allowed to act — and the composition reaches something you were explicitly denied. Privilege laundering, using the architecture exactly as designed.

T15 — the system manipulating a human. The output of an agent arrives with institutional authority. It is formatted like a report, it cites things, it does not hedge. A person reading it makes a decision on it, and applies less scrutiny than they would to a colleague's opinion — because it looks like a system output rather than an argument.

That is exploitable in both directions: an attacker who lands an injection at B1.3 gets their content delivered in your agent's trusted voice, and an agent that is merely wrong at B1.12 gets the same credibility for free.

The uncomfortable version: the more your agent is trusted, the more valuable it becomes as a channel into human decisions — so success at deployment increases this risk rather than reducing it.

Where this lands on the reference architecture. `` ingress -> orchestrator -> agent_runtime -> model | | messaging tools / mcp | | knowledge / memory egress identity + policy wrap every call · observability records it ``

2 · The risk, realised

A request that is denied directly, and permitted through the architecture.

3 · The check, as a skill

A CyberTravels traveller denied payments:write reaches it through the orchestrator, with every hop legitimate. The skill records the direct refusal first, so the composed success reads as a finding rather than as intended behaviour.

▶Real time execution as skill

The skill — skills/threats/authority-composition-check/SKILL.md

name: authority-composition-check
description: >-
  Check whether a scope a user is denied directly can be reached through a chain
  of individually legitimate hops, and whether an agent's restatement of a claim
  carries more weight than a colleague's. Use when reviewing an orchestrator, a
  routing layer, or an agent that answers on behalf of people.
allowed-tools: Read, Grep, Glob

Every hop legitimate, the composition unauthorised

A user is denied payments:write at the front door and reaches it through the orchestrator. No hop broke a rule. Authorisation was decided per edge, and nobody owns the path. The second half of the check is social: the same claim is believed more when an agent states it, which is what makes an agent a good carrier for one.

When to use this

Any system where a request is routed, decomposed, or delegated between components with different privileges — an orchestrator, a workflow engine, a tool that calls another tool.

Procedure

1 — Draw the edges with their scopes. For each hop, what identity it runs as and what it may do. The picture is usually the first time anyone has seen the composition.

2 — Attempt the direct call and record the refusal. Establish that the control exists, so the composed success is a finding rather than a misunderstanding.

3 — Compose a path to the same effect. Ask for something the orchestrator will decompose into the denied action. Record each hop's decision: every one should be a legitimate "yes".

4 — Find where the path could have been evaluated. Usually nowhere: each component sees one edge. Name the component that would have to hold the end-to-end policy, because that is the fix.

5 — Test the trust asymmetry. Present the same unverified claim from an agent and from a person, and record which is challenged. If the agent's version is accepted more readily, the agent is the better delivery vehicle and that belongs in the report.

Example

Input — the fixture committed at the top of scripts/authority_composition_check.py. Edit it and re-run: the buckets, counts and verdicts below are derived from it, not hard-coded.

Output — the opening lines of a real run:

mallory holds        : ['reports:read']
mallory asks directly for payments:write -> DENIED

same outcome, requested through the architecture:
   user asks orchestrator    mallory         ok
   orchestrator routes       orchestrator    ok
   agent acts                finance-agent   ok
   -> reached payments:write: True

The run continues past this. test_skills.py executes the script on every build, with no model configured — so what CI proves is that it runs and refuses legibly, not that it prints these lines. The output above is a recorded run against a real model, and nothing re-diffs it: a model's answer is not reproducible, and this repository does not claim otherwise anywhere it can be checked.

Output contract

{
  "edges": [{"from": "str", "to": "str", "runs_as": "str", "scopes": ["str"]}],
  "direct": {"scope": "str", "refused": true},
  "composed": {"path": ["str"], "each_hop_legitimate": true, "effect_achieved": true},
  "policy_owner": {"component": "str", "exists": false},
  "trust_asymmetry": {"agent_claim_challenged": false, "human_claim_challenged": true}
}

Failure modes

  • Auditing hops. They are all fine; that is the point.
  • Skipping the direct refusal. Without it the composed success looks like intended behaviour.
  • Leaving the trust asymmetry out because it is not technical. It is the reason the finding recurs.

Three ways to do this lesson. Pick one. They run the same procedure against the same code — what changes is who drives it, and how much you see.

1In your AI agent — you are taught

The agent reads the lesson, explains the idea, fetches the code, runs the check and answers your questions as you go. This is the one to take if you are here to learn.

Install it, once. From the folder you cloned. --all does every agent it finds on your machine:

python3 scripts/install_skills.py --all --lessons B1.17

Then restart your agent and open it in that folder. On Windows, if python3 says Python was not found, use python instead.

Then pick b1-17-attacks-that-target-the-humans:

agentinstall just for this agentstart it
Claude Codepython3 scripts/install_skills.py --tool claude --lessons B1.17type /b1-17-attacks-that-target-the-humans
GitHub Copilot (VS Code)python3 scripts/install_skills.py --tool copilot --lessons B1.17type /b1-17-attacks-that-target-the-humans in Chat
Cursorpython3 scripts/install_skills.py --tool cursor --lessons B1.17type / in Agent chat and search for it
Codexpython3 scripts/install_skills.py --tool agents --lessons B1.17type $b1-17-attacks-that-target-the-humans, or run /skills

2As one Python command — you just run it

No agent, no conversation. The same lesson executes and prints the same readback. Take this one if you have no assistant, if you are reviewing rather than learning, or if you want the result without the teaching around it.

python3 scripts/lesson.py B1.17

3Read the code, and the difference — nothing runs

Neither of the above, and no model needed. The first command writes CyberTravels exactly as it stood at the end of this lesson; the second prints the diff — the lines this lesson added or changed, and nothing from any later one.

python3 scripts/checkpoint.py --at B1.17 --out work/cybertravels
python3 scripts/checkpoint.py --at B1.17 --diff

On Windows, type python wherever these say python3 — and use \ instead of / in the paths. Everything else is identical. A0.0 step 2 explains why.

Ways 1 and 2 need a model; way 3 does not. A signed-in Claude Code CLI needs no API key, and any OpenAI-compatible endpoint works too. With neither, the audit exits 2 and says so rather than inventing an answer. A0.0 sets this up.

Ways 1 and 2 both end with the same readback: What I did, What changed, The number and Read this next. A step that was skipped or refused is reported as exactly that, never as a pass.

If you did the last lesson’s challenge, expect this to stop. Most challenges ask you to edit the code, and ways 1 and 2 will not overwrite your edits: you get stopped, not a failure, with two options — --out for a fresh folder that keeps your work, or --force to discard it and start this lesson clean.

✓What you just proved

A user denied payments:write directly reaches it through the orchestrator, with every individual hop legitimate and only the composition unauthorised — and the same claim is shown carrying more weight when an agent states it than when a colleague does.

✎Your turn

Take one permission a user is denied and see whether an agent they can talk to holds it. That pair is a laundering path, and it is invisible to any review that checks permissions one hop at a time.

Skill source: skills/threats/authority-composition-check

← PreviousB1.16 · Misaligned and deceptive behaviour