Skip to content

Custom skills: add your own skills without changing code #79

Closed

Description

@vsn411

Problem: the skills Secfoo can run are fixed at build time. src/secfoo/skills/loader.py reads only the bundled src/secfoo/skills/definitions/*.md. A team that wants its own review, such as a GraphQL API review, a PCI DSS scoping pass, or an internal coding-standard check, has to fork Secfoo and change code.

Goal: users add a skill by dropping a Markdown file in a folder. It then shows up in secfoo run --skill, --help, the dashboard and assessments, with no code change and no rebuild.

Most of the pieces already work for any skill: SkillId in cli.py is built from the loaded definitions, a skill without its own contract falls back to the generic OUTPUT_CONTRACT in renderer.py, and severity counting reads the shared ### [SEVERITY] ID: title headings. What's missing is loading from user locations, plus the edges listed below.

Where skills are loaded from, all merged:

  1. Built-ins: src/secfoo/skills/definitions/ (as today).
  2. User folder: $SECFOO_HOME/skills/*.md (default ~/.secfoo/skills/).
  3. Extra folders from config, e.g. a team's shared skills repo checked out locally:
    [skills]
    paths = ["~/work/appsec-skills", "/opt/company/secfoo-skills"]
  4. SECFOO_SKILLS_PATH: extra folders separated by os.pathsep, for CI.
  5. One-off: secfoo run --skill-file ./my-skill.md <target>.

Never load skills from the target being scanned. A cloned repo could ship a .secfoo/skills/ folder full of prompt instructions. Only the locations above count, and a test must prove that a target's folder is ignored.

File format: the same frontmatter as built-ins (id, name, description, version, optional short_name), plus these optional fields:

  • output: either generic (the default) or the id of a built-in skill whose output contract and findings lifecycle to reuse. With output: sast, a custom "SAST for our framework" skill gets the SAST contract, and its findings land in the SAST dashboard. This needs runner.py's dispatch, which today checks skill_id == SAST_SKILL_ID and so on, to key on the skill's output type instead of its id.
  • category: a free-text group label for the UI (e.g. "Compliance").

The skill body holds only the instructions. Secfoo always adds its own safety preamble, target section, depth guidance and output contract around it, so a custom skill can't remove them, and its reports still parse.

Validation and failure handling:

  • id must be lowercase kebab-case and must not match a built-in or another user skill. A clash is an error that names both files. Custom skills can't silently override built-ins.
  • Today a bad definition raises SkillLoadError while cli.py is building SkillId, which would break every command. A malformed user skill must instead be skipped with a one-line warning. Built-ins still fail hard.
  • secfoo serve picks up added or edited skill files without a restart, by checking mtimes when a page lists skills.

CLI:

secfoo skills list                 # id, name, version, source (built-in / user / path), output type
secfoo skills new graphql-review   # scaffold ~/.secfoo/skills/graphql-review.md from a commented template
secfoo skills validate [PATH...]   # parse, check ids and fields, and report problems; exits non-zero on error (for CI on a skills repo)
secfoo skills show <id> [--target PATH]   # print the full rendered prompt, to preview what the agent receives

Traceability: user skills change over time, so record which text produced a report. Add skill_version, skill_source (builtin / user) and skill_sha256 (a hash of the definition file) to runs, with a migration, and show them on the run detail page.

Dashboard:

  • Add a Custom skills group in the sidebar, under the fixed SIDEBAR_ACTIVITY_IDS in web/templating.py, grouped by category when set.
  • Custom skills appear in the run and assessment skill pickers, with a small "Custom" badge.
  • Activity pages for generic skills use the generic run list and severity counts. Skills that reuse a built-in output (output: sast) feed that skill's dashboard.

Docs and examples:

  • docs/SKILLS.md: writing a skill, the frontmatter fields, what the output contract enforces, and sharing skills across a team (a git repo plus [skills] paths).
  • examples/skills/: one or two complete example skills.

Where to look: src/secfoo/skills/loader.py, src/secfoo/skills/renderer.py (SKILL_OUTPUT_CONTRACTS, OUTPUT_CONTRACT), src/secfoo/cli.py (SkillId), src/secfoo/runner.py (findings dispatch), src/secfoo/config.py / settings.py, src/secfoo/web/templating.py, src/secfoo/storage/schema.sql.

Done when: tests cover:

  • loading from each location
  • an id clash (with a built-in, and between two user folders)
  • a malformed user skill being skipped while the CLI keeps working
  • a target repo's .secfoo/skills/ being ignored
  • --skill-file
  • output: sast sending findings to sast_findings
  • runs recording the source and hash
  • the sidebar listing a custom skill

secfoo skills validate runs in CI over examples/skills/.

Activity

  1. added
    enhancementNew feature or request
    help wantedExtra attention is needed
    hackathonOpen for the Secfoo hackathon
    difficulty: mediumLeaderboard: 6 points when the PR closing this is merged
    on Nov 10, 2026
  2. karthickmk2001 commented on Nov 11, 2026

    @karthickmk2001
    Collaborator

    I'd like to take this. PR to follow.

  3. vsn411 commented on Nov 11, 2026

    @vsn411
    CollaboratorAuthor

    Two follow-on issues now build on this one, so this issue can stay scoped to loading, validation, the skills CLI group, run traceability and listing custom skills in the sidebar:

    @karthickmk2001 nothing changes for your PR; the "Dashboard" bullets in the description above still apply as-is. If you want to leave the generic activity page exactly as it is today, #92 will take it from there.

  4. added a commit that references this issue on Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    difficulty: mediumLeaderboard: 6 points when the PR closing this is mergedenhancementNew feature or requesthackathonOpen for the Secfoo hackathonhelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions