Janus, the Roman god of gates, has two faces looking in opposite directions — and so does this pipeline. One face looks downstream, where an OpenShift consultant works: the exact RHEL/OCP builds customers run, their crashes, upgrades, and CVEs. The other looks upstream, where the fix belongs: the kernel, Kubernetes, and KubeVirt communities. Dig deep enough into Linux and the two views meet at the same root cause.
An OpenShift / RHEL / CNV research & investigation pipeline for Claude Code — case types range from kernel crash forensics to upgrade/compatibility analysis, CVE impact assessment, and operator/component behavior investigation. JANUS uses Claude Code itself as the orchestrator: the lead session composes a pipeline of small agent stages per case type, gates the dynamic ones, and hands a ranked-hypothesis report to a human. It lays evidence and hypotheses on the table — the human makes the final call.
Every case feeds both faces: synthesize hands the consultant a
ranked-hypothesis report grounded in the exact builds involved, and when
the root cause turns out to be everyone's problem, upstream-adviser
drafts the contribution proposal to carry it home.
Invoke /janus with a question or an artifact. The lead classifies the
case, shows you the pipeline it intends to run, fans the stages out on
approval, and hands you a ranked-hypothesis report at
cases/<id>/results/synthesis.md.
CVE impact assessment (needs okp-mcp):
/janus Does CVE-2024-1086 affect OpenShift 4.16 worker nodes?
→ { doc-search, source-trace } | synthesize — errata/KB sweep plus the
actual code path, cross-referenced into ranked hypotheses. A
well-supported "not affected, and here is why" is a valid outcome.
Kernel crash forensics (needs drgn):
/janus Analyze the vmcore under cases/2026-07-11-node-panic/artifacts/,
kernel 5.14.0-570.el9. The node panicked during a VM live migration.
→ adds crash-analyze: drgn triage (crashed thread, dmesg, task states),
then up to 5 observe → hypothesize → probe rounds. Every probe and its
output lands in cases/<id>/audit/ — the report's claims point at them.
Upgrade / cross-version compatibility:
/janus What changed between OCP 4.18 and 4.20 that could break VMs
using SCSI-3 persistent reservations over multipath?
→ version-diff investigation across layers (kernel, RHEL userspace,
CNV). When a stage surfaces a KubeVirt PR or an RHEL-NNNNN ticket it
cannot open, the lead launches github-trace / jira-trace follow-ups
at fan-in. For ARO cases, mslearn covers the Azure layer; for ROSA cases,
the AWS MCP servers cover the AWS layer.
Japanese prompts work the same way — the skill triggers on phrases like 「vmcoreを解析」「OOM調査」「アップグレード互換性を調査」「CVEの影響評価」.
Reproducible lab as Infrastructure-as-Code (needs terraform / ansible):
/janus Build the ARO 4.16 lab needed to reproduce this multipath
hypothesis — Terraform + Ansible, ready for review.
→ adds iac-author: it looks every provider argument and version pin up
in the Terraform registry rather than recalling it, checks the sharp-edged
values before writing them (instance-type allowlist, GPU AZ availability,
node disk ≥ 3× model size, image tags that actually exist), and writes
fmt-ed, validate-d, ansible-lint-clean code into cases/<id>/iac/
with documented variables and no credentials. Anything it could not
confirm is left un-defaulted with a TODO(iac-author) at the exact line —
a hole you can see beats a plausible guess that survives review.
Every finding in the report carries Confidence + Basis
(VERIFIED / REASONED / ASSUMED) + a reference a human can open — a
CVE/errata URL, a source permalink, or a drgn audit log. Live-cluster
verification (lab-verify) is only ever proposed: it runs on a
disposable lab, and only after you approve
review-queue/APPROVE_<id>.md.
Authoring that IaC and applying it are deliberately different stages.
iac-author is static and autonomous, because writing a .tf changes no
infrastructure; lab-verify is the only stage that runs it, behind the
approval gate, as explicit shell commands recorded in cases/<id>/audit/.
The ansible MCP's executing tools (ansible_navigator,
ade_setup_environment) are granted to no agent, and the terraform
grants are enumerated rather than wildcarded so that enabling that
server's enterprise tools can never quietly hand an agent apply_run.
scripts/validate.py fails the build if either rule is broken.
How a case flows through the team — the lead session orchestrates, small agent stages investigate in parallel, and the human holds every gate that matters:
flowchart TD
human(["Human"]) -->|"question / artifact"| intake
intake["1 · Intake — classify the case, write case.yaml, caselint it"] --> fanout
fanout["2 · Fan-out — launch stages in parallel"]
subgraph stages["Static stages — each writes cases/<id>/findings/<stage>.md"]
ds["doc-search<br/>okp · rh-docs · rh-security · mslearn · aws"]
st["source-trace<br/>casket (optional)"]
ca["crash-analyze<br/>drgn · gdb"]
ia["iac-author<br/>terraform · ansible<br/>writes IaC, runs nothing"]
end
lv["lab-verify — DYNAMIC<br/>disposable lab, never production"]
fanout --> ds & st & ca & ia
ia -->|"cases/<id>/iac/"| lv
fanout -.->|"only after the human approves<br/>APPROVE_<id>.md"| lv
ds & st & ca & ia --> fanin
lv -.-> fanin
fanin["3 · Fan-in — read findings, spot gaps"]
follow["github-trace · jira-trace<br/>(conditional, read-only)"]
fanin -.->|"a stage surfaced a<br/>PR / Jira key"| follow
follow -.->|"findings/<stage>.md"| fanin
fanin --> syn["synthesize — cross-reference all findings"]
syn --> report[("results/synthesis.md<br/>ranked hypotheses · Confidence + Basis + refs")]
report --> qc["4 · Quality check<br/>chain verify · reportlint · urlcheck · quotecheck · versioncheck · linkcheck · prosecheck (ja) · two judgment gates (C1/C2)"]
qc -.->|"send-back, by sub-code"| syn
qc -->|"handoff"| human2(["Human — final call, writes verdict.md"])
And how JANUS improves itself — two periodic agents sit outside the pipeline and feed a human-gated review queue:
flowchart LR
case["Completed case"] -->|"new failure pattern,<br/>human-approved"| lessons
lessons[("janus-lessons<br/>project-local staging")] -->|"injected into<br/>stage briefs"| next["Next case"]
verdicts[("verdict.md × 10<br/>or weekly")] --> si["self-improver<br/>metrics: hit rate, escalation<br/>precision/recall, calibration"]
lessons -->|"recurs across ≥2 cases"| si
report[("results/synthesis.md<br/>high-confidence upstream defect")] --> ua["upstream-adviser<br/>advisory only — never<br/>opens issues/PRs itself"]
si -->|"IMPROVE_<date>.md"| rq[("review-queue/")]
ua -->|"contribution drafts"| rq
rq -->|"human approves"| plugin["Plugin agent catalogs<br/>failure & reusable patterns,<br/>process changes"]
The pipeline has nine composable stages connected by a universal
findings/*.md format, plus two periodic agents:
{ doc-search, source-trace, crash-analyze, iac-author | [approve] lab-verify }
| synthesize [| localize]
github-traceandjira-tracejoin conditionally when another stage surfaces an upstream PR/issue or a Jira ticket.localizeruns only whenreport_language≠en.- Reusable investigation patterns (drgn triage, CVE tracing, refuting an a-priori hypothesis, goroutine-leak repro, etc.) are inlined into each agent so they travel with the plugin.
| Requirement | Purpose | Required? |
|---|---|---|
| Claude Code | Plugin host (CLI, desktop, or web) | Yes |
| Python 3.9+ | validate.py, selftest.py, integrity scripts (chain.py, urlcheck.py, etc.) |
Yes |
| Bash 4+ | Shell-based tests and hook scripts | Yes |
None of the MCP servers are bundled — paths are machine-specific. Register each one before running an investigation (see MCP dependencies below for setup commands).
| Server | Used by | Required? |
|---|---|---|
| okp-mcp | doc-search (CVE / errata / KB) | Recommended |
MCP server for Red Hat Documentation (rh-docs) |
doc-search (live docs.redhat.com) | Recommended |
MCP server for Red Hat security content (rh-security-mcp) |
doc-search (live CVE / errata) | Recommended |
| casket-mcp | source-trace (versioned source) | Optional |
| drgn-mcp | crash-analyze (vmcore) | For crash cases |
| GitHub MCP | github-trace, upstream-adviser | Optional |
| Atlassian Rovo MCP | jira-trace | Optional |
| mslearn | doc-search (ARO/Azure layer) | For ARO cases |
| aws-knowledge | doc-search (ROSA/AWS layer) | For ROSA cases |
| terraform-mcp-server | iac-author | For lab IaC |
| ansible-dev-tools | iac-author | For lab IaC |
| linux-mcp-server | lab-verify | For live verification |
| slack-mcp | doc-search (team context) | Optional |
Register each server under the name used here — a subagent's tool list is a fixed enumeration, so a differently named server is invisible to the stage. doc-search's optional layers (cloud docs, rh-docs) accept their upstream default names too; see Server names matter.
| Tool | Purpose |
|---|---|
| textlint + ja-technical-writing | Japanese report prose check (gate C2/prose) — fallback when the claude CLI is unavailable |
| ax | Token-aware web fetching |
| mdq | Markdown querying (jq for Markdown) |
This repo is itself a Claude Code plugin marketplace
(.claude-plugin/marketplace.json). Inside a Claude Code session:
/plugin marketplace add nogunix/janus # register straight from GitHub
/plugin install janus@janus # install the plugin (plugin@marketplace)
Or from a local clone (useful when editing the plugin):
git clone https://github.com/nogunix/janus.git ~/janus/plugin marketplace add ~/janus # register the local marketplace
/plugin install janus@janus # install the plugin (plugin@marketplace)
When editing the plugin, run python3 scripts/validate.py before
committing — it checks manifest schema, skill/agent frontmatter,
hook-script paths, and SKILL.md ↔ agents/ stage sync.
Submitted to Anthropic's community marketplace for review; once approved
it will also be installable via /plugin marketplace add anthropics/claude-plugins-community → janus@claude-community. Until
then, use the direct-from-GitHub install above — it tracks main, which
is always the latest plugin version (tagged releases are cut per version).
Restart Claude Code so the skills and agents load, then verify:
/plugin—janusshows as installed and enabled/janusappears in the skill list; the elevenjanus:*agents appear in the Agent tool list
Day-to-day maintenance:
/plugin marketplace update janus # re-read the local clone after edits
/plugin uninstall janus@janus # remove the plugin
/plugin marketplace remove janus # remove the marketplace entry
.claude-plugin/marketplace.json # marketplace listing → plugins/janus
plugins/janus/
.claude-plugin/plugin.json # plugin manifest
skills/janus/SKILL.md # /janus — pipeline driver
skills/janus/references/ # SKILL.md / doc-search detail, read on demand
skills/janus/scripts/gates.py # runs the seven step-7 checks, prints only non-passes
skills/janus/scripts/caselint.py # case.yaml vocabulary check, run at intake
skills/janus/scripts/reportlint.py # report structure + recomputed counts (C2/section, C1/basis)
skills/janus/scripts/sendback.py # send-back ledger + the stopping rule (round cap, repeats)
skills/janus/scripts/findings.py # findings format lint + fan-in digest for the lead
skills/janus/scripts/chain.py # per-case evidence hash ledger (seal/verify/lock)
skills/janus/scripts/urlcheck.py # reference-URL liveness check (backs gate C1/url)
skills/janus/scripts/quotecheck.py # verbatim-quote fidelity check (backs gate C2/quote)
skills/janus/scripts/versioncheck.py # version-provenance check (backs gate C2/version)
skills/janus/scripts/linkcheck.py # evidence links resolve to a real file/anchor (backs C1/link)
skills/janus/scripts/prosecheck.py # ja report prose via claude CLI / textlint (backs gate C2/prose)
skills/janus/scripts/textlintrc.json # ja-technical-writing config for the above
skills/janus/scripts/anchors.py # findings heading → GitHub slug map (evidence links)
skills/deck/ # layer over Anthropic's pptx skill (required): JP fonts, diagrams, PDF/QA
skills/gslides/ # report → Google Slides via the gws CLI
skills/md2pdf/ # Markdown → PDF (pandoc + weasyprint, CJK)
skills/okp-doc-search/ # okp-mcp research know-how (queries, doc_id rules)
skills/ocp-triage-heuristics/ # experiential OCP live-cluster triage reference
hooks/ # secret-safety + evidence-lock (PreToolUse denies) + evidence-chain (PostToolUse auto-seal)
agents/ # 11 agents (patterns inlined into each)
doc-search source-trace github-trace jira-trace crash-analyze
iac-author lab-verify synthesize localize self-improver
upstream-adviser
scripts/validate.py # repo consistency checks (CI-friendly, stdlib-only)
scripts/selftest.py # offline self-tests for chain.py / caselint.py / reportlint.py / urlcheck.py / quotecheck.py / versioncheck.py / linkcheck.py / prosecheck.py / hooks
.github/workflows/ci.yml # runs both on every push / PR
The plugin does not bundle MCP config — server paths are machine-specific.
Register each server yourself (claude mcp add …) before running an
investigation, then confirm claude mcp list shows ✔ Connected — a tool
being advertised isn't the same as the server being reachable.
Bridges to the official Offline Knowledge Portal (OKP) Solr index.
Requires a Red Hat account (registry.redhat.io access + an OKP access key
from https://access.redhat.com/offline/access/). The bridge server itself
is public OSS: https://github.com/rhel-lightspeed/okp-mcp.
podman login registry.redhat.io # needs a Red Hat account
# build the okp-mcp bridge image per github.com/rhel-lightspeed/okp-mcp
podman play kube okp-pod.yaml # manifest below
claude mcp add --transport http okp-mcp http://localhost:8000/mcp --scope userokp-pod.yaml — Solr + bridge in one pod:
apiVersion: v1
kind: Pod
metadata:
name: okp-mcp
spec:
containers:
- name: redhat-okp
image: registry.redhat.io/offline-knowledge-portal/rhokp-rhel9:latest
ports:
- containerPort: 8983
hostPort: 8983
env:
- name: ACCESS_KEY
value: "<your-okp-access-key>"
- name: SOLR_JETTY_HOST
value: "0.0.0.0"
volumeMounts:
- name: redhat-okp-data
mountPath: /opt/solr/server/solr/portal/data
- name: okp-mcp
image: localhost/okp-mcp:latest
ports:
- containerPort: 8000
hostPort: 8000
env:
- name: MCP_SOLR_URL
value: "http://localhost:8983"
volumes:
- name: redhat-okp-data
persistentVolumeClaim:
claimName: redhat-okp-data
restartPolicy: AlwaysRed Hat's hosted MCP server for product documentation (GA 2026-10). doc-search
uses redhat_docs_search / redhat_docs_fetch for what the docs say now —
release notes, deprecations and procedures newer than okp-mcp's snapshot.
It covers docs.redhat.com only (no KCS, CVE or errata) and returns no section
anchors, so it complements okp-mcp rather than replacing it. Needs a Red Hat
account; authentication is Red Hat SSO (OAuth) in the browser.
claude mcp add --scope user --transport http rh-docs https://docs-mcp.api.redhat.com/mcp
# then in Claude Code: /mcp → rh-docs → Authenticate (Red Hat SSO, once)The Docs team's mcp.json snippets name it Red-Hat-documentation;
doc-search grants both spellings (see Server names).
Red Hat's setup page suggests Red Hat Documentation MCP Server, but
claude mcp add rejects names with spaces — use rh-docs.
Red Hat's hosted MCP server for CVE and advisory data. doc-search uses it
first for every CVE / errata question: search with product, severity and
date filters, per-product affected state (Fixed / Not affected / Will not
fix …), CVE → advisory mapping, and the fixed RPMs of an advisory. It is
live (CVEs appear the day they are published) where okp-mcp is a snapshot,
and it supersedes rh-api-mcp's rh_get_errata. It has no KCS solutions,
so okp-mcp stays in the pipeline for workarounds. Same Red Hat SSO as
rh-docs.
claude mcp add --scope user --transport http rh-security-mcp https://security-mcp.api.redhat.com/mcp
# then in Claude Code: /mcp → rh-security-mcp → Authenticate (Red Hat SSO, once)rh-security-mcp is the name Red Hat's setup page suggests, so no alias is
needed.
Used by source-trace to search and read the exact source behind a specific OCP release — component git trees, operator bundles, SRPMs, and layered-product operands. Optional: without it, source-trace is skipped and the report notes a gap.
Its release-aware tools answer what GitHub cannot, because they depend on
what each release shipped: source_for_image (image digest → built
commit and source tree), find_dependency_users (CVE impact on shipped
dependencies), release_diff (what changed between two payload patches),
rpm_source (RHCOS SRPM with Red Hat patches) and check_patch_shipped
(is a fix's diff in each release's shipped source — backports and
non-public build commits included). source-trace fetches a fix's diff
with the GitHub MCP (get_commit / pull_request_read) for the last one.
casket-mcp is part of ocp-source-collector. Full setup (venv, registration, LAN access): mcp/README.md.
# Local (stdio) — run from the ocp-source-collector checkout
claude mcp add casket -- "$CASKET_WORK"/mcp/.venv/bin/python \
"$CASKET_WORK"/mcp/casket_mcp.py
# Or from another machine on the LAN (HTTP, casket-host runs the server)
claude mcp add --transport http casket http://<CASKET_HOST_IP>:8765/mcpPublic remote server, no auth, used by doc-search. Official server docs: https://github.com/MicrosoftDocs/mcp.
claude mcp add --transport http mslearn https://learn.microsoft.com/api/mcpMicrosoft's own snippet names the server microsoft-learn; doc-search
grants both spellings, so either registration works (see
Server names).
The AWS mirror of mslearn, used by doc-search for ROSA (Red Hat OpenShift Service on AWS) and the AWS services beneath it. All optional, from awslabs/mcp; doc-search uses whichever are connected and skips the rest.
- aws-knowledge — hosted, read-only, no auth; cross-searches AWS docs /
blogs / What's New / API references:
claude mcp add --transport http aws-knowledge https://knowledge-mcp.global.api.aws
- aws-docs — read-only, no credentials; runs via
uvx:claude mcp add aws-docs -- uvx awslabs.aws-documentation-mcp-server@latest
- aws-support — needs AWS credentials + a Business/Enterprise support
plan. Only its read-only
describe_*tools are granted to doc-search (the case create / reply / resolve tools are deliberately withheld — JANUS never mutates a support case):claude mcp add aws-support --env AWS_PROFILE=<profile> --env AWS_REGION=us-east-1 \ -- uvx awslabs.aws-support-mcp-server@latest
- aws-mcp — the Agent Toolkit for AWS,
AWS's designated successor to the awslabs servers. Usually installed as
the
aws-coreplugin, which registers it asplugin:aws-core:aws-mcp. Only its no-authaws___search_documentation/aws___read_documentation/aws___retrieve_skillare granted;aws___call_awsandaws___run_scriptare refused byvalidate.py.
A subagent's tool list is a fixed enumeration, so a server registered under a different name than the one JANUS enumerates is invisible to the stage — silently, with no error. doc-search therefore grants each optional layer under two names: JANUS's short one and the one the upstream project's own install snippet produces. A dot in the registered name becomes an underscore in the tool name.
| Layer | Short name | Upstream default |
|---|---|---|
| Microsoft Learn | mslearn |
microsoft-learn |
| AWS docs | aws-docs |
awslabs.aws-documentation-mcp-server |
| AWS Knowledge | aws-knowledge |
aws-knowledge-mcp-server |
| AWS Support | aws-support |
awslabs.aws-support-mcp-server |
| Agent Toolkit for AWS | aws-mcp |
plugin:aws-core:aws-mcp |
| Red Hat Documentation | rh-docs |
Red-Hat-documentation |
Registering under any third name will not work: rename it to one of
the two (claude mcp remove <name> then re-add), except for the plugin-
provided plugin:aws-core:aws-mcp, whose name the user cannot change —
which is why it is enumerated here.
doc-search can supplement official docs with your team's Slack
discussions. It calls the tools of
redhat-community-ai-tools/slack-mcp
(search_messages, get_thread, get_channel_history,
list_joined_channels, …), which runs locally via Podman/Docker
(quay.io/redhat-ai-tools/slack-mcp) or its one-shot setup script.
Register it with SLACK_MCP_READ_ONLY=1 so state-mutating tools
(post/react/join) error out — JANUS only reads. Point it at a workspace
you are authorized to search. Slack hits are supplementary evidence
only — findings attribute them as [slack] #channel, YYYY-MM-DD and
never rest a conclusion on them alone. Without it, doc-search simply
skips the Slack angle.
Used by the conditional jira-trace stage when another stage surfaces a
Jira ticket key (e.g. Red Hat Jira RHEL-NNNNN / OCPBUGS-NNNNN).
This is Atlassian's official remote server (Jira Cloud sites), with
OAuth 2.1 — no API token to manage. Register it under the name
atlassian, then authenticate once from /mcp:
claude mcp add --transport http atlassian -s user https://mcp.atlassian.com/v2/mcpThe safety boundary is the tool grant, not a server flag: jira-trace is
granted only getJiraIssue / searchJiraIssuesUsingJql / search, and
validate.py fails the build if any agent is granted executeWrite or
executeDestructive (JANUS never creates, edits, comments on, or
transitions tickets). Without this server, jira-trace is skipped and
ticket references stay in the report as gaps.
Public OSS: https://github.com/walac/drgn-mcp, an MCP wrapper around the drgn programmable kernel debugger. Run it sandboxed — network-cut, read-only, unprivileged — since upstream has no built-in sandboxing:
git clone https://github.com/walac/drgn-mcp.git
cd drgn-mcp && python3 -m venv .venv && .venv/bin/pip install -e .
claude mcp add drgn -s user -- "$(pwd)/.venv/bin/python" -m drgn_mcp.serverOfficial hosted read-only endpoint of
github/github-mcp-server
— no server to run yourself. The /readonly path serves only read
tools, which is the safety boundary here. Used by github-trace and
upstream-adviser:
claude mcp add --transport http github https://api.githubcopilot.com/mcp/readonlyPublic OSS: https://github.com/rhel-lightspeed/linux-mcp-server,
local or over SSH. Used by lab-verify. Register with LINUX_MCP_TOOLSET=fixed (lowercase) so the
arbitrary-script run_script toolset stays disabled — that's the safety
boundary that keeps lab-verify read-only:
pip install git+https://github.com/rhel-lightspeed/linux-mcp-server.git
claude mcp add linux -s user --env LINUX_MCP_TOOLSET=fixed -- linux-mcp-serverHashiCorp's official terraform-mcp-server, used by iac-author to write HCL against the current registry schema instead of from memory. Its default tool set is read-only registry lookup:
podman run -d -p 8090:8080 --name terraform-mcp \
hashicorp/terraform-mcp-server:latest
claude mcp add --transport http terraform http://localhost:8090/mcpOnly the enumerated registry tools are granted. Enabling the server's
enterprise tools with a Terraform token adds create_run / apply_run,
which apply real infrastructure — JANUS never grants
mcp__terraform__* as a wildcard for exactly that reason.
The ansible-dev-tools MCP server, used by iac-author:
pip install ansible-dev-tools
adt mcp --sse --port 8081 # see the ansible-dev-tools docs for current flags
claude mcp add --transport sse ansible http://localhost:8081/sseOnly the authoring subset is granted — ansible_lint,
ansible_content_best_practices, zen_of_ansible,
create_ansible_projects, define_and_build_execution_env,
adt_check_env, ade_environment_info. Two of this server's tools are
deliberately granted to no agent: ansible_navigator runs playbooks
against real targets, and ade_setup_environment runs the host package
manager. Playbook execution belongs to lab-verify, post-approval, as an
explicit ansible-playbook command in the audit trail.
ax is a token-aware CLI for web
fetching, page discovery, and structured data extraction — used in place
of curl + throwaway parsing scripts.
brew install oven-sh/bun/bun
git clone --depth 1 https://github.com/yusukebe/ax.git /tmp/ax-src
cd /tmp/ax-src && bun install --ignore-scripts
bun build src/index.ts --compile --outfile ~/bin/ax
npx skills add yusukebe/axmdq queries Markdown documents the
way jq queries JSON — extract sections, lists, tables, links, and code
blocks by selector. Useful for filtering findings, reports, and ax --md
output.
brew install mdqNot an MCP server, and not required. For report_language: ja cases
scripts/prosecheck.py checks prose through the claude CLI (Sonnet)
first, which every JANUS session already has; textlint is the fallback
for installs without the CLI or when that call fails. With neither, the
check degrades to a notice. Install textlint if you run prose checks
where the claude CLI is not available:
npm install -g textlint \
textlint-rule-preset-ja-technical-writing \
textlint-filter-rule-node-typesThe config ships with the plugin at
skills/janus/scripts/textlintrc.json and is passed explicitly with
-c, so it never collides with a .textlintrc of your own. prosecheck.py
itself stays stdlib-only, CI needs no npm install, and npx is invoked
with --no-install so nothing is ever downloaded mid-investigation.
Investigation quality is enforced by explicit discipline, not by the model in the seat:
- Model provenance — the claim above is only worth as much as your
ability to check it, so every findings file records
model:— the model that actually ran that stage — and the report's Execution Metadata carries the roll-up. The Model strategy table states what was assigned; the cost and refusal ladders substitute a different model without announcing it, so the table cannot be read backwards. A stage that cannot tell writesunrecordedrather than guessing.validate.pykeeps the assigned side honest: every agent's declared model must match SKILL.md's roster, and the Model strategy table must not contradict it. - Evidence-basis labels — every finding carries
Basis: VERIFIED | REASONED | ASSUMED(tool output observed vs. inferred from reading vs. carried in) alongside its confidence, and a label is only promoted by new evidence. - Named acceptance gates — the lead checks each report against two
judgment gates: C1 GROUNDING (references, public URLs, no
speculation language, basis integrity) and C2 COMPLETENESS &
FIDELITY (completeness, verbatim artifact names, verbatim evidence
quotes). Failures go back to synthesize by sub-code (
C1/basis,C2/quote-absent, …); a HIGH hypothesis needs at least one VERIFIED finding behind it. Three of these checks are mechanical — reference liveness, quote fidelity, and the evidence chain (see Integrity checks below). - Causation gate — crash-analyze may not record a crash cause without "X causes Y because Z" where X and Y are observations from this vmcore; correlation without a mechanism caps at MEDIUM.
- Failure-pattern catalogs — agents carry
symptom → wrong move → correct moveentries seeded from real cases (e.g. a search timeout means "reduce scope", never "report negative"). - Lessons loop — project-specific lessons are banked (with human
approval) in
.claude/skills/janus-lessons/SKILL.md, which plugin updates never overwrite; the lead injects relevant entries into stage briefs, and recurring ones get promoted into the plugin's own catalogs via the self-improver review queue. - Declared fail direction — for each check, what it does when it
proves a defect and what it does when it cannot tell are both
written down, in the Fail direction table
(
skills/janus/references/quality-gates.md). Fail-closed where a defect is provable, fail-open where it is not (an air-gapped install has to stay usable), and across every row: a notice means not checked, never passed.
Stdlib-only scripts turn "trust the report" into "check the report." Seven run at handoff, and a failure sends the report back rather than shipping it; one runs at intake, on the file that configures the work.
Most of them run twice, and the first run is the author's own. Each
stage lints its findings file before reporting complete; synthesize runs
reportlint, linkcheck and quotecheck on the report it just wrote,
and localize adds prosecheck to its translation. The lead's run at
handoff checks the sealed state and catches a stage that skipped its
own — but by then most defects are already gone. A check an author can
run is worth more than a check that only grades them, because the
alternative is a send-back round: the lead re-reads, writes a brief, the
agent re-reads the report, and the case waits on a defect its author
could have seen in one command.
Evidence chain — scripts/chain.py. Each case carries an
append-only hash ledger, cases/<id>/chain.jsonl. Every record holds
the sha256 of one evidence file plus the previous record's hash — the
same linked-hash idea as a blockchain. It makes edits visible, never
impossible: a legitimate revision (a report sent back to synthesize,
an updated finding) appends a new record and the ledger keeps the full
history; an edit that bypasses sealing breaks verification.
$ python3 scripts/chain.py verify cases/<id>
FAIL: TAMPER: results/synthesis.md changed after last seal
Sealing is mostly automatic — a PostToolUse hook
(hooks/evidence-chain.py) seals every write into the evidence set
(case.yaml, findings, report, audit logs, verdict) — and the lead
also seals explicitly before synthesis and at close (covering
shell-written files the hook can't see). Deleting a record to cover
tracks fails too: the broken hash link exposes the gap. The upshot is
that the audit trail behind a claim can't be quietly rewritten after
the fact, and the human verdicts self-improver's metrics stand on stay
ground truth.
The chain detects rewrites; chain.py lock prevents the accident in
the first place. When the lead closes fan-in it drops the write bits on
the fact base (case.yaml, findings/*.md, audit/*), and a
PreToolUse hook (hooks/evidence-lock.py) denies tracked writes to
locked files with an explanation instead of a bare permission error —
so a stage can no longer clobber another stage's findings mid-flight.
chain.py unlock is the lead's explicit escape hatch for a legitimate
revision (unlock → edit → re-seal → lock).
Report structure and arithmetic — scripts/reportlint.py. The
report states things about itself that can be derived from the findings
it was built from: how many HIGH findings there are, how many VERIFIED,
which model ran each stage, how many findings each stage produced. A
derived number is a checkable number, and until this check existed
nothing checked them — a miscount in the one table a reader uses to
size up an investigation was invisible. It recomputes all of them,
along with the template's sections (by the mode case.yaml declares),
empty cells and leftover <placeholders>, and the Objectives
Assessment vocabulary. Backs gate C2/section.
It also decides the half of C1/basis that is arithmetic. SKILL.md says a HIGH hypothesis needs one VERIFIED finding or two REASONED ones from different stages, and that ASSUMED-only evidence cannot rank above LOW. The report cites findings by number and each finding states its own Basis, so both halves follow from the files:
$ python3 scripts/reportlint.py cases/<id>
FAIL: C2/section: Execution Metadata says total findings HIGH: 3, the findings hold 1
FAIL: C2/section: Execution Metadata says doc-search=opus-x, but doc-search.md records model: sonnet-x
FAIL: C1/basis: H1 (line 44) ranks HIGH on F3=REASONED — HIGH needs one VERIFIED
finding, or two REASONED ones from different stages
FAIL: C1/ref: H2 (line 71) cites F9, which no findings file defines
warning: 1 finding(s) the report never cites: F2 [LOW/REASONED] in doc-search.md
That last FAIL is the sharpest one: a hypothesis resting on evidence that does not exist. Like linkcheck it never fails open — every input is a local file. What stays with the human reader is what only a reader can see: whether a filled cell actually says anything, and whether a claim leans on a finding it never cites.
Convergence — scripts/sendback.py. The failure mode that only
appears once both sides are agents: the lead names a defect, synthesize
revises, the lead reads again and finds the next one. Each round is
locally reasonable and the loop has no natural end. JANUS had half a
brake — the same sub-code twice is a NEEDS_HUMAN — and it leaked
twice over. Distinct sub-codes could cycle indefinitely (C1/url, then
C2/quote-mismatch, then C1/basis, each failing exactly once), and
the rule lived in the lead's memory, which a session restart or a
context compaction erases.
Now every send-back is recorded, and the stopping rule is computed from the ledger instead of recalled:
$ python3 scripts/sendback.py record cases/<id> C1/url
round 1 2026-10-19T09:17:40Z C1/url
round 2 2026-10-19T09:21:02Z C2/quote-mismatch, C2/section
round 3 2026-10-19T09:44:55Z C1/url
ESCALATE: C1/url sent back in rounds 1, 3 — the revision did not take
ESCALATE: stop the loop: write review-queue/NEEDS_HUMAN_<id>.md noting both versions
Three verdicts, and the exit code with them: OK (targeted revision),
RECONSIDER at the third round (ask synthesize to reconsider how the
report is built rather than patch more lines — patching is what got
here), and ESCALATE on a repeat or at the cap. Send-backs also name
the class of defect rather than only the quoted line, so the next
round is not the second instance of a problem the agent already
understood. The round count lands in the report's Execution Metadata,
where reportlint.py checks it against the ledger — making
rounds-to-pass a number self-improver can track across cases.
Case configuration — scripts/caselint.py. The only check that
runs before the work, because case.yaml is the one file whose wrong
value is accepted downstream rather than rejected. The case that
motivated it: report_language: jp. The pipeline composes localize on
≠ en, so a Japanese report is produced; gates.py runs prosecheck on
== ja, so the prose gate prints a skip that reads exactly like an
English case. Both halves behaved as written, and one typo turned a
gate off silently. caselint checks every such field — the mode,
status, tracks and report_language vocabularies, an artifact
case with no artifact, a malformed version_scope — and gates.py
now reports an unrecognised language as an ERROR rather than a skip.
Quote fidelity — scripts/quotecheck.py. The telephone-game
failure the lock can't catch: findings survive intact on disk while a
fact mutates as synthesize copies it into the report — "reproduced"
softens into "may reproduce", a version number drifts. The report is a
legitimately new file, so no hash ledger notices. Instead, the report
carries its load-bearing facts as attributed verbatim quotes
(> … / > — findings/<stage>.md), and quotecheck verifies each one
appears word-for-word in the file it cites — backing gate C2/quote:
$ python3 scripts/quotecheck.py cases/<id>/results/synthesis.md
FAIL: report.md:12: quote not found verbatim in findings/doc-search.md: "…"
Reference liveness — scripts/urlcheck.py. Backs gate C1/url by
curl-checking every reference URL in the report. A fabricated citation
(the classic LLM failure) dies as a 404 instead of a footnote nobody
clicked:
$ python3 scripts/urlcheck.py cases/<id>/results/synthesis.md
FAIL: https://access.redhat.com/errata/RHSA-2099:9999/ (404)
The check is deliberately honest about what it can't prove. A portal that 302-redirects a missing path into an SSO login flow (returns 200) is classified gated, not live — existence unconfirmable without authenticating, so it's flagged for a human rather than passed or failed. 401/403/429 fold into the same class. A fully-unreachable network downgrades to a notice and passes, so air-gapped okp-mcp installs stay usable.
Version provenance — scripts/versioncheck.py. The drift the quote
check can't see: a fact observed at one product version reworded into a
claim about another. Backs gate C2/version. The one hard FAIL is a
source location cited with no version anywhere in its Ref (no NVR,
casket path, or commit) — which version was read is unrecoverable. The
rest are warnings the lead judges: a Detail/Ref pair crossed within one
product family (Detail says 4.16, Ref pins 4.18), or — against the
version_scope a case may declare — a finding or report version in that
family but off-scope. Family-anchoring keeps kernel 5.14, image tags,
and RPM releases from drowning the OCP-minor signal:
$ python3 scripts/versioncheck.py cases/<id>
FAIL: findings/source-trace.md F2: source location cited with no version pin
warning: results/synthesis.md: version 4.19 asserted but backed by no finding
Evidence links — scripts/linkcheck.py. The report is meant to be
read by clicking: each claim links straight to the finding behind it, so
a reviewer lands on the evidence instead of grepping for it.
H1 rests on [F3](../findings/crash-analyze.md#f3-sigsegv-in-qemu-kvm)
and the [lab trace](../audit/lab-1.log).That affordance creates a new way to lie, which is why the check exists.
urlcheck.py only sees http(s)://, so a relative link to a finding
that was never written — or to an #anchor no heading produces —
renders as an ordinary blue link and resolves to nothing. Backs gate
C1/link:
$ python3 scripts/linkcheck.py cases/<id>/results/synthesis.md
FAIL: report.md:8: no such anchor in findings/crash-analyze.md: #f9-does-not-exist
FAIL: report.md:9: evidence file does not exist: findings/source-trace.md
Anchors are matched the way GitHub and VS Code generate them (lowercase,
punctuation dropped rather than hyphenated, duplicates suffixed -1),
plus explicit <a id="…">. Links inside fenced code blocks are ignored,
and a target outside the case directory is a FAIL. Unlike urlcheck this
never fails open — local resolution is deterministic and offline, so
a broken link is always a real defect.
Japanese prose quality — scripts/prosecheck.py. The other checks ask
whether the report is true; none ask whether it is readable. For a
report_language: ja case it checks the
ja-technical-writing
rules configured in textlintrc.json — である/ですます mixing, sentences
over 120 characters, 4+ 読点, half-width katakana — and backs gate
C2/prose. The primary backend is the claude CLI (Sonnet), prompted with
those same limits; textlint is
the fallback. Sonnet sees only the first 30,000 characters, and a longer
report gets a warning naming the unchecked rest rather than a silent OK:
$ python3 scripts/prosecheck.py cases/<id>
FAIL: report.md:7:47 [ja-technical-writing/no-mix-dearu-desumasu] "である"調 であるべき箇所に "ですます"調
FAIL: report.md:11:3 [ja-technical-writing/no-hankaku-kana] Disallow to use 半角カタカナ: "ｶﾀﾅ"
Two boundaries make it safe. It never rewrites — no --fix, because
that would mutate prose chain.py has sealed and quotecheck.py
cross-checks, and would break the rule that the lead never patches the
report itself; violations go back to synthesize like any other gate.
And ja-no-weak-phrase is switched off on purpose: it flags hedging,
but in JANUS a LOW-confidence hypothesis is supposed to read as
uncertain. Forcing assertive prose would make the report overclaim —
precisely what the Confidence/Basis labels exist to prevent.
This is the only check that shells out to a non-stdlib tool, so it fails
open in every direction — English case, neither the claude CLI nor
textlint usable, preset missing, no report yet — with a notice and exit 0. A notice means not
checked, never passed; the script refuses to print OK for a run that
did not happen. Quoted evidence, code blocks, tables and headings are
excluded from linting (textlint-filter-rule-node-types), though note
that the preset's own rules already skip those node types — the filter is
a backstop for rules added later, such as prh for terminology, which do
match inside blockquotes.
One call at handoff — scripts/gates.py. The lead runs all seven
through a runner that prints one status per check (PASS / WARN / NOTICE
/ FAIL / ERROR / SKIP) and, for anything but a clean pass, only that
check's non-OK: lines — so a report with forty live URLs costs the
lead one line, not forty. It reads the checks' exit codes and line
prefixes and never changes a verdict; each script stays a standalone CLI.
$ python3 scripts/gates.py cases/<id>
chain PASS OK: chain intact (cases/<id>/chain.jsonl)
reportlint FAIL
FAIL: C2/section: Execution Metadata says evidence basis VERIFIED: 4, the findings hold 3
FAIL: C1/basis: H2 (line 71) ranks HIGH on F7=REASONED — HIGH needs one
VERIFIED finding, or two REASONED ones from different stages
urlcheck PASS OK: 41/41 URLs live
quotecheck FAIL
FAIL: quote not found verbatim in findings/doc-search.md: "…"
versioncheck PASS OK: 6 source Ref(s) version-pinned
linkcheck PASS OK: 18/18 evidence links resolve
prosecheck SKIP
report_language is en
gates: 2 FAIL (reportlint quotecheck), 4 PASS, 1 SKIP
Every check and the lock hook have offline self-tests
(scripts/selftest.py) exercising tamper detection, ledger-edit
detection, lock/deny/unlock, quote-mutation detection, the
gated-vs-dead URL split, version-provenance drift, evidence-link
resolution, case.yaml vocabulary, report-metadata recomputation and the
hypothesis rank rule, and prosecheck's
fail-open paths (including the regression that once printed OK when
textlint was absent); .github/workflows/ci.yml runs them with
validate.py on every push and PR — no npm install needed, because the
prose check degrades to a notice there.
Read-only. Dead-artifact analysis (vmcore) is autonomous-safe; live-target work (lab provisioning, dynamic tracing) requires explicit human approval and a disposable lab, never production. Never spoofs guardrails; the final root-cause call is the human's.
MIT License - see the LICENSE file for details.
See CHANGELOG.md for version history.