Skip to content

Dashboard for custom skills: render any skill's output dynamically #92

Open

Description

@vsn411

Problem: the dashboard only knows the built-in skills. activity_detail() in src/secfoo/web/routes/activities.py picks a dashboard with a chain of skill_id == arch_dashboard.SKILL_ID, == sast_dashboard.SKILL_ID, and so on. Findings lists and finding detail pages exist only for SAST, SCA and secret scanning, each with its own extractor (report/sast.py, report/secret_scanning.py), table and template. Every other skill, built-in (deployment-readiness, prompt-review) or custom (#79), gets a bare page: a per-project run table and severity totals, nothing from the report itself.

That makes custom skills second-class: a team can add a "GraphQL API review" skill (#79), run it, and see the report only as one long Markdown blob on the run page.

Goal: any skill whose report follows the generic output contract gets a full activity dashboard, generated from the report, with no code change. Adding a skill file is enough for findings to show up, filter, persist across runs and count in the overview.

What the generic contract already guarantees (OUTPUT_CONTRACT in skills/renderer.py): a ## Summary with **Overall risk rating:** <severity>, a ## Findings Register table (ID | Finding | Severity | CWE | Verdict), and one ### [SEVERITY] F1: title section per finding. That is enough structure to build everything below.

Build:

  1. Generic findings extractor, src/secfoo/report/generic.py: parse the register rows and each finding section (id, title, severity, CWE, verdict, body) from any report that follows the contract. Tolerate missing CWE and small formatting drift; a report that can't be parsed falls back to today's page with a visible "report did not follow the output contract" note instead of a crash.
  2. Storage: a generic_findings table (project, skill_id, run_id, finding id, fingerprint, severity, verdict, CWE, title, body) with the same open/closed lifecycle as sast_findings: a finding that stops appearing in the newest successful run is closed, one that returns is reopened. Migration for existing databases. Backfill from existing runs on first start so built-in skills without a dashboard today gain history immediately.
  3. Dispatch by output type, not id. Replace the skill_id == chain with a registry mapping an output type to a dashboard builder (architecture, threat-modeling, sast, sca, secret-scanning, generic). A skill's output type comes from its definition (Custom skills: add your own skills without changing code #79 adds output:; built-ins map by id). Custom skills with output: sast land in the SAST dashboard; everything else gets generic.
  4. Generic activity page, templates/activities/generic_dashboard.html: overall risk rating distribution across projects, severity donut (reuse _donut_gradient and severity_rows), open/closed counts, a findings list filterable by severity, verdict and project (reuse web/finding_filters.py), the existing per-project table, and a run trend (findings per run, newest 20 runs). Plus a finding detail route, /activities/{skill_id}/findings/{finding_id}, rendering the finding body with the shared Markdown renderer.
  5. Dynamic everywhere else: custom skills appear in the overview coverage panel, the project page's activity table, top findings (web/top_findings.py) and assessment results, using the generic findings. Skills added or edited while secfoo serve runs show up on the next page load (Custom skills: add your own skills without changing code #79's mtime reload), including a new sidebar entry under the Custom skills group.
  6. Severity scales: the contract lets a skill define its own severity words. Normalise through finding_filters.normalize_severity, and show unknown values as their own bucket rather than dropping them.

Where to look: src/secfoo/web/routes/activities.py (activity_detail), src/secfoo/web/sast_dashboard.py (reference shape for a dashboard builder), src/secfoo/report/sast.py (reference extractor), src/secfoo/report/severity.py, src/secfoo/web/templates/activities/detail.html and sast_dashboard.html, src/secfoo/web/templating.py (SIDEBAR_ACTIVITY_IDS, DISPLAY_SKILL_ORDER), src/secfoo/storage/schema.sql.

Relation to #79: #79 loads custom skills and lists them in the sidebar. This issue makes their output a real dashboard. You can start now against the built-ins that have no dashboard today (deployment-readiness, prompt-review), which exercise the same code path.

Done when: tests cover: extracting findings from a contract-following report and from a slightly malformed one; the lifecycle (open → closed → reopened across three runs); the dispatch registry choosing generic for an unknown skill and sast for output: sast; the generic page rendering for deployment-readiness with filters applied; a finding detail page; and the sidebar plus overview picking up a skill file added after secfoo serve started.

Activity

  1. added
    enhancementNew feature or request
    help wantedExtra attention is needed
    hackathonOpen for the Secfoo hackathon
    difficulty: hardLeaderboard: 10 points when the PR closing this is merged
    on Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    difficulty: hardLeaderboard: 10 points when the PR closing this is mergedenhancementNew feature or requesthackathonOpen for the Secfoo hackathonhelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions