The skill — skills/programme/resilience-readiness-check/SKILL.md

name: resilience-readiness-check
description: >-
  Check the three properties a programme needs for the day a control fails —
  drift is detected, the stop mechanism is tested, and the run is replayable —
  and report which are not ready. Use as a standing readiness check rather than
  after an incident.
allowed-tools: Read, Grep, Glob

Perfection is not available; these three are

A programme cannot prevent every failure and does not need to. What it needs is that when a control fails, somebody notices, somebody can stop it, and somebody can reconstruct what happened. Each of the three is testable today, and each fails quietly if nobody tests it.

When to use this

As a standing quarterly check, before granting unattended autonomy, and as the closing item of a programme review.

Procedure

1 — Detection: compare today against the signed-off baseline. Not "do we have monitoring" — run the comparison and report the drift and any new tool. A monitor that has never been compared against a baseline is untested.

2 — Stop: check the mechanism, the measurement and the date. A named mechanism, a measured time-to-stop in seconds, and when it was last exercised. A test older than a quarter is a claim.

3 — Replay: check the run record has the five inputs — prompts, tool results, model version, seed, retrieved context — and confirm the delegation chain and the resources reached are recorded.

4 — Report each as ready or not, with the missing item named. Three booleans and three sentences. A single readiness score hides which of the three is missing, and they have different owners.

5 — Re-run after every change to any of the three. A model upgrade invalidates the drift baseline; a platform change invalidates the stop test.

Example

Input — the fixture committed at the top of scripts/resilience_readiness_check.py. Edit it and re-run: the buckets, counts and verdicts below are derived from it, not hard-coded.

Output — the opening lines of a real run:

NOTICE   drift 0.500  new tools ['run_shell']  → detected
STOP     12s, tested 41d ago, survives restart True  → ready
RECOVER  replayable True, scope from the chain ['queue', 'repo-core', 'repo-payments']  → ready
programme                 containment ASR  outcome when containment fails
------------------------------------------------------------------------------------------------
prevention only                        0%  undetected — found by a third party, weeks later
                                         elapsed to resolution: 720h
prevention + notice                    0%  detected, cannot halt it — damage continues while you improvise

The run continues past this. test_skills.py executes the script on every build, with no model configured — so what CI proves is that it runs and refuses legibly, not that it prints these lines. The output above is a recorded run against a real model, and nothing re-diffs it: a model's answer is not reproducible, and this repository does not claim otherwise anywhere it can be checked.

Output contract

{
  "detection": {"baseline_at": "str", "drift": 0.0, "new_tools": ["str"], "ready": true},
  "stop": {"mechanism": "str", "measured_seconds": 0, "last_tested": "str", "ready": true},
  "replay": {"inputs_present": ["str"], "missing": ["str"], "chain_recorded": true,
             "resources_recorded": true, "ready": true},
  "verdict": {"ready": false, "missing": ["str"]}
}

Failure modes

• Checking that monitoring exists. Run the comparison.
• An estimated time-to-stop. Measure it.
• A single readiness score. Three properties, three owners.

The skill — skills/research/research-durability-check/SKILL.md

name: research-durability-check
description: >-
  Separate a folklore claim from a research result, and score a finding backlog
  on whether each finding became a control, a detection, or nothing. Use when
  justifying a research function, or when last year's findings cannot be traced
  to anything that changed.
allowed-tools: Read, Grep, Glob

Research is what survives the person who did it

A finding that closed a ticket is a fix. A finding that became a control or a detection is institutional capital, and the difference is measurable — which means a research programme can be defended with a number rather than with enthusiasm.

When to use this

Reviewing a research backlog, defending the function's budget, or deciding what "done" means for a finding.

Procedure

1 — Test each claim for actionability. Three gaps disqualify it: no reproduction, no affected-version statement, no stated condition under which it does not hold. A card failing any of them is folklore, however true it is.

2 — Classify each closed finding by outcome. A control, a detection, an eval case, or nothing — a patch with no accompanying control counts as nothing for this purpose, because the class recurs.

3 — Compute durability. The share of findings that produced a control or a detection. It is usually about half, and the half that produced nothing is usually the more interesting work.

4 — Look at what the lost half had in common. Typically: no owner outside the research team, or a finding whose class had no control to attach to. Both are addressable and neither is about effort.

5 — Define closure to require an outcome. A finding is closed when it has a control, a detection or an explicit accepted-risk record with an owner. Anything else reopens as the same finding next year.

Example

Input — the fixture committed at the top of scripts/research_durability_check.py. Edit it and re-run: the buckets, counts and verdicts below are derived from it, not hard-coded.

Output — the opening lines of a real run:

=== folklore — actionable: False ===
CLAIM     our review agent can be tricked by comments in diffs
SETUP     the review agent
TRIGGER   a comment
OBSERVED  it approved something it shouldn't
RATE      not measured
HOLDS WHEN

The run continues past this. test_skills.py executes the script on every build, with no model configured — so what CI proves is that it runs and refuses legibly, not that it prints these lines. The output above is a recorded run against a real model, and nothing re-diffs it: a model's answer is not reproducible, and this repository does not claim otherwise anywhere it can be checked.

Output contract

{
  "claims": [{"name": "str", "actionable": false, "gaps": ["reproduction", "versions", "conditions"]}],
  "backlog": [{"finding": "str", "outcome": "control|detection|eval|none", "owner": "str|null"}],
  "durability": 0.0,
  "lost": {"count": 0, "common_cause": "str"},
  "closure_definition": "str"
}

Failure modes

• Counting patched as closed. The class recurs and the count flatters.
• Scoring effort. Durability is about outcome, not about difficulty.
• No owner outside research. That is the mechanism by which findings are lost.