A plugin is a directory holding a plugin.toml manifest and a Python module with one
class. The analyzer searches --plugins DIR recursively for manifests, imports each
module by path and instantiates the class. The plugins shipped with the package live
under coretrace_python/bundled/ and are written the same way; they are the reference
for every kind of plugin described here.
coretrace-plugins/
└── sensitive_logging/
├── plugin.toml
└── sensitive_logging.py
coretrace-python-analyzer --check src/ --plugins ./coretrace-pluginsname = "sensitive-logging"
version = "1.0.0"
plugin_api = ">=1,<2"
requires = ["ir.ssa"]
provides = ["vulnerability.sensitive-logging"]
[entrypoint]
module = "sensitive_logging"
class = "SensitiveLoggingPlugin"| Field | Meaning |
|---|---|
name |
Unique plugin name, also used in cache keys. |
version |
The plugin's own version; changing it invalidates cached results. |
plugin_api |
The range of plugin API versions the plugin supports. The current API is version 1. |
requires |
Capabilities the plugin needs; a mismatch with the class's requires is a manifest error. |
provides |
Capabilities the plugin offers, for documentation and conflict detection. |
entrypoint |
The module (without .py, next to the manifest) and the class in it. |
The entrypoint module may be a package, <module>/__init__.py, when a plugin grows past
one file: its submodules are reached through relative imports (from .tables import celery). Two plugins may use the same module name; each is imported under a name derived
from its path.
Every API is named by a canonical symbol: python. followed by the dotted path,
resolved through imports and aliases, so python.os.system covers os.system,
from os import system and from os import system as run. Calling a symbol denotes that
symbol, so a chain such as sqlite3.connect(p).cursor().execute is
python.sqlite3.connect.cursor.execute, and app = Flask(__name__) gives app.route the
symbol python.flask.Flask.route. Builtins are python.builtins.<name>. Symbols are
SymbolId values from coretrace_python.semantic.symbols. A function of the project is
python. followed by its module's import name and its qualified name,
python.app.handlers.main; two files with the same import name, a/app.py and
z/app.py outside any package, define the same symbols, so a model naming
python.app.main names the function of each.
Every plugin is a Plugin subclass (coretrace_python.plugins) with a class-level
name and a requires set of analyses; the base classes below fill in analyze for the
common cases.
Reports every call to one of a set of symbols, whatever name the file gives them.
from typing import ClassVar
from coretrace_python.findings import Severity
from coretrace_python.plugins import SymbolCallDetector
from coretrace_python.semantic.symbols import SymbolId
class SensitiveLoggingPlugin(SymbolCallDetector):
name: ClassVar[str] = "sensitive-logging"
rule_id: ClassVar[str] = "sensitive-logging"
symbols: ClassVar[frozenset[SymbolId]] = frozenset({SymbolId("python.logging.debug")})
severity: ClassVar[Severity] = Severity.LOW
message_template: ClassVar[str] = "call to {symbol} may log sensitive data"The shipped dangerous-eval and weak-crypto plugins are of this kind. accepts(call, symbol, defs)
keeps only some calls; describe(ctx, call, symbol, defs) returns the CallReport (message and
confidence) of one call, the rule's message_template and confidence by default, for a rule that
tells its calls apart, as unsafe-xml does.
Reports the flows of one taint kind that reach a sink and survive refutation. The sources, sinks and sanitizers come from the model plugins, so a detector stays generic and a new framework model makes every detector aware of it.
from typing import ClassVar
from coretrace_python.findings import Severity
from coretrace_python.plugins import TaintDetector
from coretrace_python.taint import TaintKind
class CommandInjectionPlugin(TaintDetector):
name: ClassVar[str] = "command-injection"
rule_id: ClassVar[str] = "command-injection"
kind: ClassVar[TaintKind] = TaintKind.COMMAND
severity: ClassVar[Severity] = Severity.HIGH
title: ClassVar[str] = "Command injection"confidence defaults to high; a hotspot, a flow behind a guard that mentions the value
or an authorization decorator, is reported at medium. The taint kinds every source
carries by default (TaintKind.ALL) are SQL, NOSQL, COMMAND, HTML, PATH,
SSRF, CODE, DESERIALIZATION, REDIRECT and ADVISORY (APIs affected by
advisories). Three kinds are outside ALL, so only a value a model marks with them
reaches their sinks: CREDENTIAL, which credential-named parameters carry so that
ordinary input reaching a database write is not a plaintext credential; LOG, for
values that must not be written to a log as they are; and PII, for personal data
that must not leave the application through a log, a third party or plain storage.
A rule may know more about its own sinks than the refutation does: judge(ctx, function, flow, verdict) returns the rule's verdict on a flow, given the refutation's, and by
default returns it unchanged. ssrf and open-redirect read there what the URL's
constant text proves, a proof they share (coretrace_python.taint.urls) and each
judges by its own criteria. The refutation's
verdict, which other consumers of the flow such as the advisory correlation read, does
not change; a rule reading more analyses declares them in its requires.
A rule that tells cases of its sinks apart by severity too overrides assess(ctx, function, flow, verdict), which returns an Assessment of the verdict and the
severity; by default it pairs judge's verdict with the rule's severity. A hotspot
verdict lowers the confidence of the finding, never its severity. An Assessment may
also carry a title for the finding's message instead of the rule's, and metadata
the finding carries besides the flow's; the rule id never changes, so suppressions and
baselines keep matching. command-injection
reads there the command a flow reaches (coretrace_python.taint.commands): a shell
string or the program is high; an element of an argument list run without a shell is
an option, medium unless a documented option of the program runs a command it chooses.
It sets the injection metadata (command or option, the latter titled Option injection) only when it establishes what the input is.
Declares sources, sinks and sanitizers for a library or a framework. A model plugin reports nothing by itself; the detectors consume what it declares.
from typing import ClassVar
from coretrace_python.plugins import ModelPlugin
from coretrace_python.semantic.symbols import SymbolId
from coretrace_python.taint import EntryPoint, Model, Sanitizer, Sink, Source, TaintKind
class BottleModels(ModelPlugin):
name: ClassVar[str] = "bottle-models"
models: ClassVar[tuple[Model, ...]] = (
EntryPoint(SymbolId("python.bottle.route"), "http"),
Source(SymbolId("python.bottle.request.query"), "http"),
Sink(SymbolId("python.bottle.redirect"), TaintKind.REDIRECT),
Sanitizer(SymbolId("python.bottle.html_escape"), TaintKind.HTML),
)| Model | Declares |
|---|---|
Source(symbol, label, kinds=ALL) |
A call or attribute whose value is attacker-controlled; label names it in messages (http, stdin, argv, http-response). kinds restricts what it can inject. Text input, such as a form field, a header or a URL parameter, carries TEXT_KINDS, every kind but NOSQL: a string cannot hold a query operator. |
Sink(symbol, kinds, positions=(), keywords=(), selects=False, requires_called_result=NONE) |
A call dangerous for kinds. positions limits a kind to the given argument indexes, and keywords to the given keyword names; a restricted kind reaches only the arguments listed there. SQL sinks read the statement only, and the SSRF sinks of HTTP clients read the URL, by position or as url=. A starred argument or **kwargs reaches no restricted kind. selects marks an argument that chooses by name the code that runs (a module, a file, a directory, an attribute): only a check limiting it to constants proves it. requires_called_result lists the kinds that apply only when the call's result is itself called, as for getattr's name: the engine reports them at each invocation of the result, the other kinds at the call. |
Sanitizer(symbol, kinds) |
A call whose result is safe for kinds, including when the call is inside a project function the data goes through. |
EntryPoint(symbol, label, kinds=ALL, inputs=None) |
A decorator or a class base whose functions receive their parameters as label input, such as flask.Flask.route; or a project function or class named by its own symbol (python.app.handlers.main), as an integration reading a deployment declaration names a Lambda handler. inputs limits the parameters that are input to those positions, after self for a method: a handler's event, not its context. |
TypedParameter(symbol, label, kinds=ALL) |
A class whose annotated parameters carry label input, such as Django's HttpRequest. |
NamedParameter(pattern, label, kinds) |
A regular expression on parameter names; matching parameters carry kinds, as password carries CREDENTIAL. |
BasePattern(pattern, label, parameters=(None,), kinds=ALL) |
A regular expression on the canonical symbol of a class's base, for bases generated per project such as a gRPC servicer's (`(?:^ |
RouteRegistrar(symbol, argument, label, kinds=ALL, keyword=None) |
A call registering a view at argument argument (or keyword), such as Django's path; the view receives label input. |
SuffixSink(suffix, kinds, positions=()) |
A sink matched by the end of the symbol, for methods of any class such as objects.raw. |
SafeArgument(symbol, argument, values, position=None, kinds=ALL) |
A call to the sink symbol whose argument (by keyword, or at position) denotes one of values is not a sink for kinds, such as yaml.load with Loader=yaml.SafeLoader. Values are written as the analyzer records arguments: a symbol by its canonical name, a constant as Python writes it (True). Only a value given explicitly makes the call safe. |
TemplateRender(symbol, position=0, keyword="template_name", context=1, request=None) |
A call rendering, with autoescaping, the template named by its argument at position or keyword, such as render_to_string, with the variables of its argument at context or context= and the request at request or request= (None: the call takes none); it makes the filter calls the template applies, fed from that context and, with the request context processor, that request (see the usage guide). What it returns carries no HTML when the name is written in the call and the project's template escapes everything it prints (see the usage guide). A name the analyzer cannot read keeps an advisory a template filter reaches under investigation in the VEX document. |
Validator(symbol, kinds=ALL, argument=0) |
A callable whose truth proves its argument safe for kinds, such as re.fullmatch. A flow guarded by it is refuted when it carries no other kind into the sink, and is a hotspot otherwise: a redirect validator does not make a shell command safe. symbol may be a function of the project, by its project symbol (python.hc.accounts.views._allow_redirect), which then counts wherever it is called, in its own module too. |
RequestObject(symbol, text=()) |
The inputs from symbol, an entry point, a route registrar, a typed parameter or a source, are request objects. A view receives the request first (after self), then URL parameters, which are text. Reading one of the request's text attributes gives text, as Django's GET or aiohttp's query; its body and files keep the request's kinds. An attribute read counts only when every source of the value is such a request. |
AuthorizationGuard(symbol, label) |
A decorator or a condition restricting who reaches the code, such as login_required; a flow behind it is a hotspot. |
Members(symbol, dynamic=None, defined=(), typed=()) |
What the attributes and items of an instance of the class symbol give, for objects that give others by name. typed maps an attribute to the class of what it gives, called or not, since a call denotes its callee: pymongo's get_database gives a Database, as Flask-PyMongo's db property does. dynamic, when set, is the class of every item and of every attribute outside defined and typed: a pymongo client gives a Database by name (client.shop, client["shop"]), a database a Collection. Every path to a collection then denotes the collection class, so a sink on Collection.find covers them all. Without dynamic, an attribute keeps its path and an item its container's symbol, as for any class. |
Two plugins may describe the same symbol. An identical model is registered once; two
Sink models merge their kinds, positions and keywords, so a plugin can add SQL to a sink
another plugin declared for COMMAND; any other difference is a conflict that stops
the analysis with a ModelError naming both plugins. A merged sink selects when either
does, and a kind requires the called result only when every declaration carrying it
does: an unconditional declaration is never weakened, whatever the order.
A plugin may also read models from the project it analyses: project_models(self, root)
returns them, from a file at root for instance. It is called once per directory check,
in every process analysing it, before any module. Its models join the plugin's own and
are part of what cached results depend on. A single-file check reads no project. A
model declared wrongly raises ModelError, which stops the check with its message
(exit status 2). This is how a project declares validators of its own:
class DeclaredValidators(ModelPlugin):
name: ClassVar[str] = "declared-validators"
def project_models(self, root: Path) -> Sequence[Model]:
path = root / "validators.txt"
lines = path.read_text(encoding="utf-8").splitlines() if path.is_file() else []
return tuple(Validator(SymbolId(line)) for line in lines if line)A model plugin may also carry advisories, a tuple of Advisory values
(coretrace_python.dependency) with the package, the vulnerable range and the affected
symbols, as the shipped sample-advisories plugin does. Requirements matching them are
reported, calls to the affected symbols become reachable vulnerabilities and tainted
calls become exploitable ones. An advisory's entry points carry conditions, of kind
argument (on the value, or with present on the argument being passed at all),
host, semantic, keyword_name or sequence, and attacker_arguments, decided as
the usage guide describes. An entry point may also declare suffixes,
matched against the tail of a derived symbol of the project's own modules on dot
boundaries, as a SuffixSink matches, for receivers the engine cannot type (see the
usage guide).
A project plugin may also implement refine(ctx, findings), called once with every
finding of the run after all plugins have reported. It returns the same findings, in
the same order, and may only change their severity and confidence or add metadata:
the rule, the location, the message and the evidence a finding already carries are not
the refiner's to touch, and the engine rejects a refinement that changes or drops any
of them with a RefinementError. A reclassified finding records refined_by and the
original_severity or original_confidence it had, so a report always shows what the
detector said and who changed it.
from dataclasses import replace
from coretrace_python.findings import Confidence
from coretrace_python.plugins import ProjectContext, ProjectPlugin
class InternalOnly(ProjectPlugin):
name = "internal-only"
def analyze_project(self, ctx: ProjectContext):
return ()
def refine(self, ctx: ProjectContext, findings):
return tuple(
replace(f, confidence=Confidence.LOW, metadata={**f.metadata, "exposure": "internal"})
if f.function in INTERNAL else f
for f in findings
)Judges every string literal of the module, or every value of a configuration file, against provider patterns, credential-like names and entropy.
from typing import ClassVar
from coretrace_python.plugins import SecretDetector, SecretPattern
from coretrace_python.plugins.secrets import DEFAULT_CREDENTIAL_NAMES, DEFAULT_PATTERNS
class InternalSecrets(SecretDetector):
name: ClassVar[str] = "internal-secrets"
patterns: ClassVar[tuple[SecretPattern, ...]] = (
*DEFAULT_PATTERNS,
SecretPattern("acme", r"acme_[0-9a-f]{40}"),
)
credential_names: ClassVar[tuple[str, ...]] = (*DEFAULT_CREDENTIAL_NAMES, "acme_key")A pattern whose regex names a secret group ((?P<secret>...)) is judged on that group:
a match whose secret is a placeholder (..., <token>, ${VAR}, fewer than four
characters) is not reported, as with credential names.
entropy_threshold, hex_entropy_threshold and minimum_length tune the high-entropy
rule. Findings carry a redacted preview only.
Runs once per project after every module is analysed, with a ProjectContext: the
module graph, the dependency graph, every advisory the plugins and advisory files
contributed, the policy, the project root, and each module's imports, call graph and
functions. ctx.functions(module) gives every function as the call graph names it
(Admin.get, outer.inner, <module>), with its span, so a finding can be placed in
one, the label of the entry point it is (http for a route, argv for a command)
when the models make it one, and its aliases, the module-level names bound to it by
assigning one name to another (main = actual), transitively and in source order, so
an integration declaring a handler by the name a module binds can tell a name the
engine resolves to no function; a method or a nested function has none. ctx.call_graph(module) gives each function's call sites
(sites(function)) and the symbols it reads, called or not (reads(function): reading
request.form['name'] reads python.flask.request.form), each where it first reads it.
ctx.templates gives the templates found under the project's templates directories:
their names, the filters of Django's own libraries they apply as calls (each a
FilterCall(symbol, span), the function behind the filter and where the template
applies it; a filter a project tag library the template loads replaces is no call),
and where they name a template the analyzer cannot read, or load a library it cannot
read while applying a filter of that library's name (unread).
The shipped vulnerable-dependency, reachable-vulnerability and dependency-policy
plugins are of this kind.
from collections.abc import Sequence
from typing import ClassVar
from coretrace_python.findings import Finding
from coretrace_python.plugins import ProjectContext, ProjectPlugin
class RequirementsPresent(ProjectPlugin):
name: ClassVar[str] = "requirements-present"
def analyze_project(self, ctx: ProjectContext) -> Sequence[Finding]:
return () # inspect ctx.dependencies, ctx.modules, ctx.call_graph(module), ctx.functions(module)Override analyze(self, ctx: PluginContext) and return findings. ctx.module is the
module, ctx.functions its analysable functions and ctx.get(Analysis, function)
any analysis result: SSAAnalysis (coretrace_python.ir.ssa) for the code,
TaintAnalysis (coretrace_python.taint) for the flows, RefutationAnalysis
(coretrace_python.findings.refutation) for their verdicts, DependencyAnalysis
(coretrace_python.dependency) for the requirements. List what you read in requires.
ctx.path(source_id) is the path of a file from the root of the analysis, the one
coretrace_python.findings.context classifies (is_test_code, code_context,
code_origin): judge test code on it, never on the absolute path, which depends on where
the project sits. It is None for a file outside the root.
A Finding (coretrace_python.findings) has a rule_id, a message, a Severity, a
Confidence, a location, the enclosing function and string metadata. Rule ids are
lower-case words joined by dashes. The location (span) is a SourceSpan (file, start
and end line and column) when the line is established, or a FileLocation (file and the
JSON pointer of a value) for a value of a structured file whose line cannot be: a plugin
that reads a finding's line checks which it has, and never invents a line for a
FileLocation (coretrace_python.source.positions places JSON and TOML values).
A plugin describes the rules it reports in its optional rules attribute: each rule id to a
RuleDescription, a title and what the rule detects. The SARIF report gives them to its
consumers (shortDescription and fullDescription), so that GitHub code scanning names and
explains the alert; a rule the plugin does not describe is listed by its id alone.
from collections.abc import Mapping
from coretrace_python.findings import RuleDescription
class SensitiveLoggingPlugin(SymbolCallDetector):
...
rules: ClassVar[Mapping[str, RuleDescription]] = {
"sensitive-logging": RuleDescription(
"Sensitive data logged",
"A logging call at debug level that may write sensitive data to the logs.",
),
}A finding may also point at other places it involves, each a RelatedLocation (a span and a
short message) in its related tuple, as a taint flow reaching its sink inside a project
function points at that sink. The SARIF report lists them as relatedLocations and links to
them from the message.
Load the plugin directory with the engine and check one source or one project:
from pathlib import Path
from coretrace_python import engine
from coretrace_python.source import SourceManager
source = SourceManager().add_source("app.py", "import logging\n\ndef f(x):\n logging.debug(x)\n")
findings = engine.check(source, [Path("coretrace-plugins")])
assert [f.rule_id for f in findings] == ["sensitive-logging"]Pass engine.BUNDLED_PLUGINS too when the plugin relies on the shipped models.