[karthickmk2001]

Closes #80

What does this change, and why?

Runtime evidence where fixes are chosen: in the CLI, the SARIF a PR is annotated with, and a CI gate. Docs are in docs/REFERENCE.md → Runtime evidence in the CLI and CI.

Lookup (src/secfoo/runtime/developer.py, RuntimeLookup). Runtime state for each SCA finding comes from two sources, and the stronger state wins:

1. The local store (secfoo runtime ingest, or a self-hosted secfoo serve sharing the store). No network.
2. The server the secfoo cloud login key belongs to, via GET /api/runtime/v1/evidence (Runtime ingest API: sensors use your Secfoo API key #66). Cached under $SECFOO_HOME/runtime-cache/ for offline use; when the server is unreachable, the cache is used and a note says so.

With no key there is no network call at all. No runtime data is the normal state, not an error. --no-runtime skips both sources.

Commands:

• secfoo findings list (new) has a Runtime column, for example "Reached · POST /upload (anonymous) · 312", "Executed · 13 hits", "Loaded · version drift", "Not loaded · 74% coverage · 6h" or "No runtime data". Findings are sorted within each severity: reached, executed, loaded, no data, then not observed. Not-observed findings are never hidden (Tracking: Runtime reachability for SCA (in-app probe + live sensor) #51 rule 4).
• secfoo findings show <fingerprint> (new) shows one finding, its live routes and loaded versions, plus the remediation hint. For a reached SCA finding that's "Fix first: upgrade pyyaml to 5.4; this protects live route(s): POST /upload (anonymous)".
• secfoo findings export: JSONL records and SARIF results for SCA findings carry runtime / properties.secfoo.runtime (state, summary, routes, hits, last seen, version drift). A reached finding's message adds "Reached in production via POST /upload". --runtime-priority raises reached findings to level error.
• secfoo findings check --fail-on reached[:<severity>] (new) is the CI gate. It exits 4, a distinct code next to secfoo run's 1, 2 and 3, when an open finding at that severity or worse is reached on a request route. executed[:<severity>] also counts code that ran with no request context. It's off unless you run it, and passes when there's no runtime data.
• After secfoo run, when runtime data exists for the project, the summary adds one line, e.g. "2 of 5 findings are reached in production: D1 (POST /upload, anonymous), …". It's wrapped so it can never fail a run.

Out of scope here: "Reached via E1 (B1)" needs threat-model entry points (#69, #70), which don't exist yet.

Testing

• pytest tests passes. 1238 passed locally. The only failures are the 4 existing Windows-only POSIX permission and symlink tests that fail on main too; Linux CI is unaffected. The two new CLI tests were then fixed to pass at CI's 80-column width.
• ruff check . adds no new findings.
• Added tests/test_runtime/test_developer.py (19 tests), covering the issue's Done when:
  • every runtime state
  • no sensor ("No runtime data", no errors)
  • no key → no network call, and --no-runtime skipping the fetch even with a key
  • the server fetch with the key, and offline falling back to the cache
  • the stronger of local and server winning
  • list ordering and --no-runtime
  • the show remediation hint
  • the SARIF runtime properties, message and --runtime-priority levels
  • the gate's exit code 4, severity filtering, executed, passing without data, and spec validation
  • the secfoo run summary line, and its silence without data

🤖 Generated with Claude Code

[github-actions]

Leaderboard update

• Closed Runtime evidence at the developer stage: CLI, SARIF and CI gate #80 (difficulty: medium): +6 points

karthickmk2001 now has 67 points (rank #1, Silver). 33 more to reach Gold.

Full leaderboard