All lessons › Function E — The Agentic SOC › Understand — Correlation, Intel and the Hunt
E3.9

Third-party threat intelligence, and the tactics it names

E3
▶
Lightboard recording

Not recorded yet. The lesson below is complete and runnable today — the video is added later without touching this page.

Video placeholderpublishes automatically from recordings/E3.9.mp4
Risk

Unsourced confidence in synthesis loops.

Control

Provenance discipline; refuse claims without a source.

The lab

Build a synthesis loop that must cite or abstain.

◉Use case relevance

Two intel questions, not one: how adversaries are using AI, and who is coming for the AI you run. Most programmes track the first because it is written about, and the second is the one that reaches your estate.

At CyberTravels. intake() splits incoming intelligence by whether anything backs it. Same rule as C2.12, at the other end of the pipe: an unsourced claim cannot carry a severity and cannot become a detection, because a rule whose provenance is a model's fluency is a rule nobody can defend.

◴What this lesson is — Day 0, Day 1, Day 2

Provenance discipline on intelligence intake: a claim with no source cannot carry a severity and cannot become a detection, because a synthesis loop's output is formatted identically whether anything backs it or not.

Day 0 — why

Intelligence that stays a narrative about adversary trends moves nothing, however well written.

Day 1 — how

Convert each indicator into a rule, or record why it cannot be one.

Day 2 — measure

Detections produced per intelligence report. It is the only measure of an intel function that survives scrutiny.

Measure this

what to measuredenominator or target
detections produced per intelligence reportratio; the only measure of an intel function that survives scrutiny

▦The framework, and how it works

   two intel questions, only one of which is well covered

   how adversaries use AI        who is coming for the AI you run
   +---------------------+       +-----------------------------+
   | written about a lot |       | your models, agents, MCP    |
   | mostly capability   |       | servers, eval corpora       |
   +---------------------+       +-----------------------------+
                                       the one that reaches you

Threat intel is judged by exactly one thing: how many detections came out of it. Everything else — feed volume, report quality, briefing frequency — is input, not outcome.

An indicator is actionable when two things are true:

  • it is a type you can match on (a host, a hash, a specific technique with a concrete precondition), and
  • its confidence justifies the false-positive cost of the rule it becomes.

A narrative about adversary trends is not intelligence you can operate. It may be genuinely useful for planning and it should not be counted as detection coverage, because counting it that way makes a programme look covered when it is not.

Anchor → E1.0. Intelligence shortens the detect interval before the incident rather than during it, and only by becoming a rule. That is why the single measure is detections produced: a narrative about adversary trends moves no interval, however well written.

2 · Demo — a feed, converted

3 · Where it breaks — conversion is only the first of three numbers

4 · The control — agent-specific intel is mostly internal

intel sourceconverts to a detectionwhy
your own incidents100%the technique that worked against you
your red team (D1)90%attack-suite results become detections directly
your drift monitor (E1.2)80%baseline changes are leading indicators
vendor advisories50%useful for the supply chain (D1.1)
commercial feed30%generic indicators; little agent-specific content yet
The highest-converting sources are all internal. For agentic threats the intel programme is mostly a feedback loop out of D1 and E1.2, not a purchase.

5 · The procedure, as a skill

Four of seven indicators convert; the two narratives and the low-confidence host are dropped with reasons. The skill then reports the three numbers a renewal conversation needs: converted, alerted, actioned.

▶Real time execution as skill

The skill — skills/detection/threat-intel-to-rules/SKILL.md

name: threat-intel-to-rules
description: >-
  Convert a threat-intel feed into detection rules, drop the indicators that
  cannot be matched or scored, and report the three numbers that say whether the
  feed earns its price. Use when subscribing to intel, or when a feed produces
  alerts nobody actions.
allowed-tools: Read, Grep, Glob

Most of a feed is not matchable, and that is fine if you say so

An intel feed contains indicators, narratives and low-confidence guesses. Only some of it converts into something a detection can match on. The useful output is a small number of rules plus an explicit list of what was dropped and why — and three numbers that tell you whether to renew.

When to use this

On any intel feed, at subscription and at renewal, and when an intel-derived rule fires and nobody knows what to do.

Procedure

1 — Set a confidence floor and a matchable-type list. Hosts, hashes, techniques. Narratives are context, not indicators; write the floor down before reading the feed.

2 — Convert what qualifies. One rule per indicator, with the response attached. An indicator with no response is not ready — the analyst will get the alert and ask what to do.

3 — Drop the rest with reasons. "Narrative, not matchable" and "confidence 0.4, below floor" are both fine. An undocumented drop is indistinguishable from an oversight.

4 — Run the rules against real events. Record which fired and what the response was. A rule that has never fired is not automatically bad; a rule that fires and produces no action is.

5 — Report the three numbers. Indicators converted, alerts produced, alerts actioned. The ratio of the third to the second is what the feed is worth, and it is the number to take to a renewal conversation.

Example

Input — the fixture committed at the top of scripts/threat_intel_to_rules.py. Edit it and re-run: the buckets, counts and verdicts below are derived from it, not hard-coded.

Output — the opening lines of a real run:

indicator                                     kind         conf  verdict
----------------------------------------------------------------------------------------
collect.example.com                           host         0.95  RULE — convertible to a rule
169.254.169.254                               host         0.99  RULE — convertible to a rule
a1b2c3d4e5f6                                  hash         0.72  RULE — convertible to a rule
pastebin.example                              host         0.55  drop — confidence 0.55 below floor 0.7
adversaries increasingly use agentic tooling  narrative    0.40  drop — narrative is not matchable in telemetry
agents reading ~/.aws/credentials             technique    0.88  RULE — convertible to a rule

The run continues past this. test_skills.py executes the script on every build, with no model configured — so what CI proves is that it runs and refuses legibly, not that it prints these lines. The output above is a recorded run against a real model, and nothing re-diffs it: a model's answer is not reproducible, and this repository does not claim otherwise anywhere it can be checked.

Output contract

{
  "feed": {"indicators": 0, "confidence_floor": 0.0, "matchable_types": ["str"]},
  "rules": [{"indicator": "str", "type": "str", "response": "str"}],
  "dropped": [{"indicator": "str", "reason": "str"}],
  "firings": [{"rule": "str", "events": 0}],
  "value": {"converted": 0, "alerts": 0, "actioned": 0}
}

Failure modes

  • Converting narratives. They do not match on anything.
  • Rules with no response. The alert lands and stops.
  • Reporting alerts rather than actions. Alerts are cheap to produce.

Three ways to do this lesson. Pick one. They run the same procedure against the same code — what changes is who drives it, and how much you see.

1In your AI agent — you are taught

The agent reads the lesson, explains the idea, fetches the code, runs the check and answers your questions as you go. This is the one to take if you are here to learn.

Install it, once. From the folder you cloned. --all does every agent it finds on your machine:

python3 scripts/install_skills.py --all --lessons E3.9

Then restart your agent and open it in that folder. On Windows, if python3 says Python was not found, use python instead.

Then pick e3-9-third-party-threat-intelligence:

agentinstall just for this agentstart it
Claude Codepython3 scripts/install_skills.py --tool claude --lessons E3.9type /e3-9-third-party-threat-intelligence
GitHub Copilot (VS Code)python3 scripts/install_skills.py --tool copilot --lessons E3.9type /e3-9-third-party-threat-intelligence in Chat
Cursorpython3 scripts/install_skills.py --tool cursor --lessons E3.9type / in Agent chat and search for it
Codexpython3 scripts/install_skills.py --tool agents --lessons E3.9type $e3-9-third-party-threat-intelligence, or run /skills

2As one Python command — you just run it

No agent, no conversation. The same lesson executes and prints the same readback. Take this one if you have no assistant, if you are reviewing rather than learning, or if you want the result without the teaching around it.

python3 scripts/lesson.py E3.9

3Read the code, and the difference — nothing runs

Neither of the above, and no model needed. The first command writes CyberTravels exactly as it stood at the end of this lesson; the second prints the diff — the lines this lesson added or changed, and nothing from any later one.

python3 scripts/checkpoint.py --at E3.9 --out work/cybertravels
python3 scripts/checkpoint.py --at E3.9 --diff

On Windows, type python wherever these say python3 — and use \ instead of / in the paths. Everything else is identical. A0.0 step 2 explains why.

Ways 1 and 2 need a model; way 3 does not. A signed-in Claude Code CLI needs no API key, and any OpenAI-compatible endpoint works too. With neither, the audit exits 2 and says so rather than inventing an answer. A0.0 sets this up.

Ways 1 and 2 both end with the same readback: What I did, What changed, The number and Read this next. A step that was skipped or refused is reported as exactly that, never as a pass.

If you did the last lesson’s challenge, expect this to stop. Most challenges ask you to edit the code, and ways 1 and 2 will not overwrite your edits: you get stopped, not a failure, with two options — --out for a fresh folder that keeps your work, or --force to discard it and start this lesson clean.

✓What you just proved

Four of seven indicators convert to rules — the two narratives and the low-confidence host are dropped with reasons. The rules fire on three of five events with concrete responses. The three-number summary shows a 57% conversion rate and 67% of alerts actioned, and the source table ranks internal sources highest.

✎Your turn

Compute your own three numbers for last quarter: indicators received, rules deployed, alerts actioned. The ratio between the first and third is the honest value of the programme.

Tools used

MISPOpenCTIGLM-4.6Claude Haiku 4.5

Skill source: skills/detection/threat-intel-to-rules

← PreviousE3.8 · Fleet-level correlation — seeing a swarm