Not recorded yet. The lesson below is complete and runnable today — the video is added later without touching this page.
recordings/B1.12.mp4A single fabrication becomes a shared premise, and by the third hop nothing in the system records that it was ever uncertain.
Verification against ground truth before a claim propagates (B3.5).
Let one fabricated fact travel three hops and watch its confidence rise as its provenance disappears.
Agent one is 90% accurate, which sounds fine. Agent two consumes its output as fact, and agent three consumes that. By the third hop the confident wrong answer has been repeated enough times that it reads like corroboration.
What it covers. Let one fabricated fact travel three hops and watch its confidence rise as its provenance disappears.
Why a security engineer needs it. A single fabrication becomes a shared premise, and by the third hop nothing in the system records that it was ever uncertain. The control it builds is: verification against ground truth before a claim propagates (B3.5).
This is a risk lesson: it shows the failure happening before anything tries to stop it, so the control that follows is answering something you have already watched go wrong.
A single fabrication becomes a shared premise, and by the third hop nothing records that it was ever uncertain.
Verify a claim against ground truth before it is allowed to propagate.
Propagated claims carrying a verification result, as a share of claims that reached a second agent.
Measure this
| what to measure | denominator or target |
|---|---|
| propagated claims carrying a verification result | share of claims that reached a second agent |
agent 1 agent 2 agent 3 report
90% right ---> takes as ----> takes as ----> "three sources agree"
fact fact
error compounds: 0.9 -> 0.81 -> 0.73
confidence compounds the other way, because repetition reads as corroborationOWASP T5 — Cascading Hallucination Attacks. LLM09 — Misinformation.
The model component produces confident text. Sometimes the text is wrong. That is a known property, and on its own it is a quality problem rather than a security one.
It becomes a security problem when the architecture has more than one step, because an unverified claim from step one is an input to step two. And inputs are not re-examined — that is the point of a pipeline.
Watch what happens to a single fabrication as it travels:
Hop one. "I could not find a CVE for this dependency, it is probably fine." Hedged, and the hedge is visible.
Hop two. The next agent summarises: "dependency has no known CVEs." The hedge is gone. Nothing lied — summarising removes qualifiers, that is what summarising is.
Hop three. "Dependency verified clean." Now it is a finding, with the confidence of something that was checked, and no field anywhere records that nobody checked anything.
The security consequence is that confidence rises as evidence disappears, which is exactly backwards. And it is not limited to accidents: an attacker who can inject one plausible claim early gets it laundered into an established fact by your own pipeline, which is why this sits in the threat taxonomy rather than in a quality backlog.
Where this lands on the reference architecture. `` ingress -> orchestrator -> agent_runtime -> model | | messaging tools / mcp | | knowledge / memory egress identity + policy wrap every call · observability records it ``
One hedged guess, three hops, and the confidence it acquires on the way.
A hedged sentence about libfoo becomes a confident claim in three hops. The skill tracks both series — confidence and surviving provenance — because either one alone looks ordinary and the pair is the finding.
The skill — skills/threats/confidence-provenance-decay-check/SKILL.md
name: confidence-provenance-decay-check
description: >-
Track a hedged claim across summarisation hops and measure confidence rising
while provenance disappears. Use when output from one model or agent becomes
input to another, in report chains, research pipelines, or any place a summary
is summarised.
allowed-tools: Read, Grep, Glob"I could not find a CVE, it is probably fine" becomes "libfoo is clean" in three hops. Nothing in the chain lied. Each step did what summarising does — dropped the hedge, dropped the caveat, dropped the sentence saying where the claim came from — and the result is a confident statement with no evidence behind it.
Report generation, triage pipelines, research chains, any agent that consumes another agent's output, and any workflow where a human reads only the last artefact.
1 — Find the chains. Every place where generated text is input to a later generation step. Include the human-visible summary at the end; it is a hop.
2 — Instrument the original claim. Record its hedge, its confidence if one is stated, and its provenance — the source it rests on.
3 — Step the chain and record both series. After each hop, the claim's confidence and its surviving provenance fields. Two numbers per hop; the shape is the finding.
4 — Identify the hop where provenance empties. That is where the claim became unfalsifiable, and it is almost always earlier than where the confidence peaked.
5 — Test the carry rule. A chain that propagates confidence and provenance as structured fields, rather than as prose the next step must re-read, does not decay this way. Check whether the interface has those fields at all.
Input — the fixture committed at the top of scripts/confidence_provenance_decay_check.py. Edit it and re-run: the buckets, counts and verdicts below are derived from it, not hard-coded.
Output — the opening lines of a real run:
hop confidence claim
0 0.20 I could not find a CVE for libfoo, it is probably fine
1 0.80 a CVE for libfoo, it is fine
2 0.80 a CVE for libfoo, it is fine
3 0.80 a CVE for libfoo, it is fine
provenance recorded at hop 3: none
confidence at hop 0: 0.20 at hop 3: 0.8The run continues past this. test_skills.py executes the script on every build, with no model configured — so what CI proves is that it runs and refuses legibly, not that it prints these lines. The output above is a recorded run against a real model, and nothing re-diffs it: a model's answer is not reproducible, and this repository does not claim otherwise anywhere it can be checked.
{
"chain": [{"hop": 0, "claim": "str", "confidence": 0.0, "provenance": ["str"]}],
"confidence_delta": 0.0,
"provenance_lost_at": 0,
"interface": {"carries_confidence": false, "carries_provenance": false},
"human_reads_hop": 0
}Three ways to do this lesson. Pick one. They run the same procedure against the same code — what changes is who drives it, and how much you see.
The agent reads the lesson, explains the idea, fetches the code, runs the check and answers your questions as you go. This is the one to take if you are here to learn.
Install it, once. From the folder you cloned. --all does every agent it finds on your machine:
python3 scripts/install_skills.py --all --lessons B1.12Then restart your agent and open it in that folder. On Windows, if python3 says Python was not found, use python instead.
Then pick b1-12-cascading-hallucination:
| agent | install just for this agent | start it |
|---|---|---|
| Claude Code | python3 scripts/install_skills.py --tool claude --lessons B1.12 | type /b1-12-cascading-hallucination |
| GitHub Copilot (VS Code) | python3 scripts/install_skills.py --tool copilot --lessons B1.12 | type /b1-12-cascading-hallucination in Chat |
| Cursor | python3 scripts/install_skills.py --tool cursor --lessons B1.12 | type / in Agent chat and search for it |
| Codex | python3 scripts/install_skills.py --tool agents --lessons B1.12 | type $b1-12-cascading-hallucination, or run /skills |
No agent, no conversation. The same lesson executes and prints the same readback. Take this one if you have no assistant, if you are reviewing rather than learning, or if you want the result without the teaching around it.
python3 scripts/lesson.py B1.12Neither of the above, and no model needed. The first command writes CyberTravels exactly as it stood at the end of this lesson; the second prints the diff — the lines this lesson added or changed, and nothing from any later one.
python3 scripts/checkpoint.py --at B1.12 --out work/cybertravels
python3 scripts/checkpoint.py --at B1.12 --diffOn Windows, type python wherever these say python3 — and use \ instead of / in the paths. Everything else is identical. A0.0 step 2 explains why.
Ways 1 and 2 need a model; way 3 does not. A signed-in Claude Code CLI needs no API key, and any OpenAI-compatible endpoint works too. With neither, the audit exits 2 and says so rather than inventing an answer. A0.0 sets this up.
Ways 1 and 2 both end with the same readback: What I did, What changed, The number and Read this next. A step that was skipped or refused is reported as exactly that, never as a pass.
If you did the last lesson’s challenge, expect this to stop. Most challenges ask you to edit the code, and ways 1 and 2 will not overwrite your edits: you get stopped, not a failure, with two options — --out for a fresh folder that keeps your work, or --force to discard it and start this lesson clean.
A hedged guess at confidence 0.2 becomes a confident claim above 0.8 in three hops, while the provenance field empties — confidence rising at exactly the rate evidence disappears.
Take a finding your pipeline produced and try to walk it back to the step that first asserted it. If you cannot reach a step that checked something, you have found a cascade rather than a finding.
Tools used
Skill source: skills/threats/confidence-provenance-decay-check