merging to latest maltrail. - #1
Open
efij wants to merge 465 commits into
Open
Conversation
stamparm
force-pushed
the
master
branch
2 times, most recently
from
January 2, 2026 22:31
9ffcb6a to
ca4afa9
Compare
Allowlist-gated via BLACKLIST_ALLOWLIST, falling back to FAIL2BAN_ALLOWLIST; an authenticated session also passes.
Nine of 47 feeds were producing zero indicators. Six of them had been
fetching hosts that no longer exist, on every update, for years:
palevotracker DNS fails
ransomwaretracker dns/ip/url 503 (service retired 2019)
zeustracker monitor/url DNS fails (service retired 2019)
They are removed. trails/static/malware/palevo.txt stays - that is a
curated static list, not the dead feed.
feodotrackerip read 'ipblocklist_recommended.txt', which now returns a
header and nothing else; it is repointed at ipblocklist.csv, which Feodo
Tracker still fills, and takes only currently-online C2s. The CSV also
carries offline history, and an address that stopped serving a botnet
years ago has usually been reassigned since - listing those trades a dead
detection for a live false positive. The malware family now comes from the
feed rather than being hardcoded to emotet.
The reason none of this was noticed is core/update.py:
if not results and not any(_ in url for _ in ("abuse.ch", "cobaltstrike")):
print("[x] something went wrong during remote data retrieval ...")
Zero-yield reporting was exempted for abuse.ch, presumably because a
tracker with no live C2s legitimately returns an empty list. But the
exemption also covered feeds whose service had been RETIRED, and all seven
of the silent ones were abuse.ch URLs. Being told about a feed that is
briefly empty is much cheaper than not being told about one that is
permanently dead, so the exemption is gone and every empty feed is now
reported, plus a summary at the end of the run because a per-feed line
scrolls past in an update this long.
That leaves 41 feeds, 3 yielding nothing: the cybercrime-tracker trio from
issue #19545. Those are not dead - they return HTTP 307 to a Maltrail
user-agent AND to a Chrome one, so it is a challenge gate rather than a
UA check, which is why the links work in a browser and not from a script.
Left in place; they were already being reported, since that host was never
exempted.
Feed yield is unchanged at ~1.91M indicators.
Both sensors dropped every PPPoE frame. Ethernet parsing accepted only ethertype 0x0800 and 0x86dd (plus one 0x8100 VLAN tag), and ethertype 0x8864 - PPPoE session, RFC 2516 - matched neither, so the frame was discarded before any IP parsing. The IP-offset heuristic did not save it: that runs only for an UNKNOWN datalink, and a mirrored Ethernet port is DLT_EN10MB, so there was no fallback at all. This is the whole of the reported symptom. Traffic mirrored from a DSL/fibre uplink is PPPoE-encapsulated end to end, so a SPAN port carrying it produced nothing, while the capture host's own traffic - plain Ethernet/IP on the same interface - was detected normally. That is exactly "detection works from the docker host, not from mirrored clients", and the reporter's own tcpdump output showed "PPPoE [ses 0xf5d7]" on every frame. Handled in both sensors, so this is a shared fix rather than a divergence: 6-byte PPPoE header, 2-byte PPP protocol field, then IP at +8. PPP protocol 0x0021/0x0057 only, so LCP/CHAP/IPCP control traffic stays silent instead of being misparsed as IP, and PPPoE discovery (0x8863) is still ignored. A VLAN tag in front of the PPPoE header works, since the existing 802.1Q skip runs first. Six unit tests including a truncation sweep that never panics, plus a corpus case (plain PPPoE, VLAN+PPPoE, and an LCP frame that must not fire) so the differential harness covers it: both sensors produce the same two events. Full gate green, parity 37/37 and 8/8.
An exit-only list cannot see a host on your own network using Tor. A client connects to a GUARD relay and never to an exit, so outbound Tor use was invisible - which is what the reporter meant by "Tor traffic is never recognized", and the part of that thread that went unresolved. Exits stay in their own feed, because they mean something different: traffic arriving FROM one, rather than one of your hosts reaching out. Source is Onionoo, the Tor Project's own metrics API. That matters because the objection raised on the issue against the suggested community mirrors was Cloudflare: onionoo.torproject.org answers directly from onionoo-backend-03.torproject.org with no CDN interstitial. 10,150 running relays, 10,580 addresses (7,404 IPv4, 3,176 IPv6), 594 kB, ~1.5 s. Rated "bad reputation (tor node)", matching the convention the other infrastructure feeds use (compare bitcoinnodes.py). That is deliberate on two counts. It is low severity, which is the honest rating - a host talking to a Tor relay is informational, not evidence of compromise. And "reputation" is a LOW_PRIORITY_INFO_KEYWORD, which makes the merge order-independent: for an address that is both a relay and an exit, the more specific "tor exit node (suspicious)" wins whichever feed is processed first. Verified against the exact condition in core/update.py in both orders rather than assumed. Separate feed file, so an operator who finds ~10.5k relay addresses noisy turns it off with DISABLED_FEEDS torprojectnodes and keeps the exits. Noted while here, not changed: "tor exit node (suspicious)" matches no group in REMOTE_SEVERITY_REGEX, so exits are unrated for remote severity while relays now rate low. Pre-existing, and re-rating exits is a separate decision.
) Requested so other tooling can ask Maltrail about a single observable instead of downloading and grepping the whole set. There was no way to do it over HTTP: /check_ip is enrichment (ipcat, ASN, country), not a trail lookup, and /trails hands back the entire file. GET /check?q=www.sub.evil.example {"query": "...", "found": true, "trail": "evil.example", "info": "asyncrat (malware)", "reference": "(static)"} Reports WHICH key matched, because that is the part a caller cannot work out for itself: a subdomain of a listed domain matches its parent (the same walk _check_domain_member does, so the answer agrees with what the sensor would decide), and a URL is tried as host/path and then as the bare host, matching how URL trails are stored. Reads through the memory-mapped store via core/trailsbin, so answering costs the server no heap - the table is the same shared mapping the sensor uses, not a copy loaded into the reporting process. Handles are re-opened when trails.csv.bin changes, so a trail update is picked up without a restart; verified by adding a trail to a running server and seeing the lookup change from miss to hit. Unauthenticated, deliberately and consistently with /trails beside it, which already serves the ENTIRE trail set to anyone - that is how a sensor pulls from UPDATE_SERVER - so a single-key lookup discloses strictly less than what is already public on the same port. Event data is the opposite case and stays gated; that distinction is written down at the handler. Bounded input, no wildcards, misses are not errors, and a missing or half-written store answers "unavailable" rather than failing the request. Seven tests including hostile input; test_httpd 38/38.
The endpoint answered from the whole trail store, custom trails included. That bypasses a control this server already applies: ENABLE_MASK_CUSTOM (default on) redacts custom trail names from AUTHENTICATED non-admin users in /events, so treating them as public to callers with no session at all inverts the existing policy. The justification for leaving /check open was that /trails already serves the entire set unauthenticated, so a single-key lookup discloses strictly less. That holds for static and feed trails, which are public data on GitHub. It does not hold for custom trails: those are the operator's own indicators, and ENABLE_MASK_CUSTOM is the server saying so. Custom matches now require a session allowed to see them - the same rule /events uses, so admins and (when no USERS are configured) everyone. A custom-only match is reported as a MISS rather than as a masked hit, because confirming membership is itself the disclosure: an oracle that says "yes, that internal hostname is in your private list" leaks the same thing whether or not it prints the name. Public trails are unaffected and still answerable without a session. Four tests: anonymous is refused and the name never appears in the body, an admin gets it, a uid>=1000 analyst does not, and public trails keep working. test_httpd 42/42.
#15164) Both options took exactly one endpoint, so a deployment could not feed a redundant SIEM or a second collector without running a second sensor. Separated inside the existing option - commas, semicolons or whitespace - rather than the SYSLOG_SERVER_1, _2, ... numbering the request suggested. Every configuration naming a single endpoint keeps working untouched, there are no new option names to document or validate, and it matches how the other list-valued options in maltrail.conf are already written. Implemented in both sensors, so this is not a divergence: core/log.py gains _endpoints() (memoized, since log_event calls it per event) and loops; the Rust sensor parses into a list at load and sends to each. The payload is rendered ONCE per event and sent to each collector, not re-rendered per endpoint - the CEF line and the JSON are identical by construction, so every collector receives byte-identical records. EVERY endpoint is validated at load, not just the first. A typo in the second target is exactly as fatal as one in the first, because a sensor that quietly forwards to one of two configured collectors is the kind of half-working that goes unnoticed for months. LOG_SERVER deliberately unchanged: that is the sensor -> Maltrail server channel, and duplicating events into two servers is a different question from fanning out to two collectors. Verified with real UDP listeners rather than by inspection: two syslog collectors and one logstash collector each received all 5 events from the same replay, identical counts, from BOTH sensors. Config tests cover comma, semicolon and whitespace separation, the single-endpoint case, and a bad endpoint in first and second position. Full gate green.
The image's CMD is `python3 server.py`, but its HEALTHCHECK ran `maltrail-sensor -T`. So a server-only container - the default use of this image - was health-checked against the sensor's requirements and reported unhealthy while serving perfectly. Measured rather than reasoned about: with a normal server configuration (one that does not set DISABLE_CHECK_SUDO, since a server has no reason to), `-T` exits 1 on `capture privileges: no CAP_NET_RAW` alone - a capability a server neither has nor needs. A healthcheck that cannot pass in a correct configuration is worse than none, because it teaches people to ignore container health. The default healthcheck now asks the server whether it is serving, via the unauthenticated /ping endpoint, honouring MALTRAIL_HTTP_PORT. Verified end to end: a server-only container built from this Dockerfile reports `healthy`, where the same container reported `unhealthy` before. The sensor service in docker-compose.yml overrides it with `-T`, which is the right question for that container. Second half of the report: both processes run as uid 10001, and a BIND MOUNT keeps the host directory's ownership, replacing the one the image prepared. A logs directory owned by the host login user leaves the container unable to create the day's log file - and because server.py only opens that file when the first event arrives, the container starts, serves the UI, and silently persists nothing. Named volumes (what the shipped compose uses) are unaffected, which is why this never showed up here. docker/README.md now documents the three ways out: chown the host directory to 10001, rebuild with MALTRAIL_UID/MALTRAIL_GID matching the owner (new build args, verified to produce a container running as that uid), or use a named volume. It also gains a server-only section stating plainly that no capabilities are needed. The reporter's own diagnosis was correct on both counts, including that chowning the directory alone fixed persistence without a restart.
New trail type (CERT), new /check endpoint, new config options, feeds added and removed, and a detection fix in both sensors - not a patch release.
Reported downstream as "13 non-domain entries" in the generated domain
blacklist. The cause is upstream, here: static trails written in a form
the sensor never sees on the wire. They load, occupy a row, and match
nothing - the same failure mode as a dead feed, and just as silent.
* 30 internationalised domains stored as Unicode. DNS carries punycode,
so `ortakoporotör.com` could never match the `xn--ortakoporotr-fjb.com`
that actually arrives. Converted.
* 5 separator lookalikes copied out of reports: U+2010 and en-dashes for
hyphens, U+2024 for a dot. `onlinechatmatrix<U+2024>xyz` was simply
`onlinechatmatrix.xyz`; `xn<en-dash>metaspport-v43e.com` was
`xn--metaspport-v43e.com`, which decodes to `metasụpport.com`.
Each rewritten line keeps its original as an inline comment (the loader
strips those), so the provenance survives review.
One is left alone deliberately: `support¬forum.org`. NOT SIGN is not a
known separator lookalike, so any correction is a guess, and guessing wrong
turns a dead trail into a false positive against an innocent domain. The
checker reports it.
Added sensor/tools/check_trails.py, which finds these. It reports the 134
remaining underscore entries (`properties_76.dzhafarho.ru` and friends):
those are unreachable for a different reason - VALID_DNS_NAME_REGEX rejects
the QUERY before the lookup happens - and fixing that is a change to
matching behaviour, not to data, so it is reported rather than acted on.
Both claims were checked by replaying queries through the sensor rather
than reasoned from the regex: a query with an underscore produces no event,
and a bare TLD trail like `xyz` DOES match `evil.xyz` through the
parent-domain walk. The checker's first version flagged those 55 bare TLDs
from suspicious/domain.txt as broken; they are not, and it no longer does.
Replaying several files produced one set of totals and nothing saying which file they came from, so a run over a directory could not be attributed - which is exactly the case the request names as useful. Each capture now reports itself as it finishes: [i] corpus/dns_queries.pcap: 599 B (6 packet(s), 5 event(s)) in 0.000s [i] corpus/mixed_soup.pcap: 3.6 kB (47 packet(s), 8 event(s)) in 0.000s [i] corpus/http_trails.pcap: 491 B (3 packet(s), 3 event(s)) in 0.000s File, size, packets, events, elapsed - the three things the issue asks for, plus the event count, which is the number an analyst replaying a capture set actually wants per file. Printed as each file completes rather than collected for the end, so a long replay shows progress and a file that stalls is identifiable while it is still running. Suppressed by -q, like every other operational line. Live capture is unaffected: handles carry an empty label there, because there is no file to attribute anything to. Tested through the binary, not the harness: the per-file lines must carry a size and an elapsed time, and their packet counts must RECONCILE with the run total - a per-file breakdown that does not add up is worse than none.
…9080) Brute force against the reporting interface was the one attack on Maltrail that Maltrail could not see. Attempts were already written locally in sshd's shape - "Accepted/Failed password for <user> from <ip> port <port>", which journald and fail2ban parse - but only to the local auth log, so a SIEM could not see them without shell access to the box. They are now also forwarded: CEF to SYSLOG_SERVER (signature "auth", severity 1 for success and 2 for failure) and JSON to LOGSTASH_SERVER, through the same multi-collector path the sensor uses. Two things found while implementing it. The local write forked. It ran `logger` through subprocess.check_output on EVERY attempt and waited for it - one process spawn per login, on exactly the code path an attacker hammers, which turns a brute-force attempt into a fork bomb of the defender's own making. The stdlib syslog module reaches the same auth facility through a socket with no child process. The username was written verbatim. It is whatever the attacker POSTed, so an embedded newline appended a line of their choosing to the audit trail meant to catch them - including a convincing "Accepted password for root from ...". Control characters are now replaced rather than dropped, so the attempt itself stays visible, and the field is bounded at 64 characters so one request cannot flood the log. Verified against a running server with a real UDP collector: success and failure arrive and are distinguishable, and an injected newline comes through as "evil?Accepted password..." on a single line. Three tests plus doctests for the sanitiser; test_httpd 45/45.
Nineteen options were read by the code and mentioned nowhere in
maltrail.conf - the file operators actually edit. An option that exists
only in the source is an option nobody finds, and that is behind a good
share of the support traffic: this week alone I have had to TELL people
about CAPTURE_FANOUT and CHECK_TLS_CERTIFICATES, neither of which a
reader of the configuration could have discovered.
Seventeen were the Rust sensor's own settings - CAPTURE_WORKERS, the
EVENT_THROTTLE_* family, CAPTURE_SNAPLEN, OFFLINE_TIMESTAMPS and the rest
- documented in sensor/docs/ but absent from the config. The worst was
USE_CONDENSED_STORAGE: read by BOTH sensors, defaulting to ON, and
writing a SQLite store, with no mention in the configuration at all.
All are now listed, commented out, with their real defaults and the
reasoning that matters (why CAPTURE_WORKERS is not derived from
PROCESS_COUNT; that a bigger DOMAIN_CACHE_ENTRIES measured slower). No
behaviour changes - every added line is a comment.
The nineteenth, USE_MULTIPROCESSING, is deliberately NOT documented: it
is read only to print a deprecation notice, so advertising it would
invite the thing the notice discourages.
tests/test_options.py keeps it that way, in both directions - an option
the code reads must appear in the file, and an option in the file must be
read by something, so a setting that silently does nothing is caught too.
It also asserts the extraction finds a plausible number of options, since
a scan that matches nothing would let both directions pass vacuously.
Verified by removing an option and watching the test fail, not just by
watching it pass. Its first version was too loose and counted prose
comments as options ("# TLS 1.3 encrypts..." became an option named TLS);
the pattern now follows the file's real convention, no space after the '#'.
) /blacklist and /fail2ban are derived from the CURRENT DAY's events, so a host flagged once stays listed until midnight. The only way off the list was USER_WHITELIST - which is permanent, and also suppresses every FUTURE detection for that host. The reporter said so plainly and it went unanswered for three years: "can I remove my ip from the blocklist, so it can detect new IDS problems in the future?" LOG_DIR/cleared.txt now answers exactly that: 10.13.13.37 # cleared as of this file's mtime 10.13.13.99 2026-08-11 09:30:00 # cleared as of an explicit moment Events BEFORE the mark are ignored by both endpoints; any event AFTER it re-lists the host immediately. Clean the machine, clear it, and it stays watched - which is the difference between this and a whitelist. Costs nothing when unused: the event timestamp is parsed only for an IP that appears in the cleared map, which is empty on virtually every deployment. The cleared file's (mtime, size) is part of both endpoints' cache keys, so an edit takes effect on the next request rather than up to 8 seconds later. That has to be loaded BEFORE the key is built - reading the key first serves the stale answer for one bucket, which looks precisely like the feature not working, and did until I traced it. Verified against a running server through all three states: both hosts listed, the cleared one dropped from /blacklist and /fail2ban, and a later event putting it back. Three tests, each restoring the log in setUp - without that the appending test leaked into its alphabetical neighbours and they failed for a reason unrelated to their assertions.
A bind mount keeps the host directory's ownership, so an image with a
fixed `USER 10001` cannot write `-v ./logs:/var/log/maltrail` unless the
host directory happens to be owned by 10001. Telling operators to run
`sudo chown 10001:10001` first is not a fix.
docker/entrypoint.sh runs as root for a few milliseconds, works out which
uid can write the log and state directories - bind-mount owner, PUID/PGID,
or the image's own user - and drops to it with setpriv before exec'ing
anything. With --user there is no root to adapt with, so it checks the
directories and refuses to start with a message naming the one at fault.
Two more failures found while verifying that, both of which made the
container quietly useless rather than broken:
* cap_add did nothing. Docker puts a requested capability in the
BOUNDING set and leaves the ambient set empty when USER is not root:
`--user 10001 --cap-add NET_RAW` gives CapEff 0, and AF_PACKET fails
with EPERM. The compose sensor could never capture a packet. Only a
process that starts as root can raise an ambient capability, which is
what the entrypoint now is - for the sensor only, by argv.
* the sensor never updated its trails. Without an absolute -c path the
repository root resolved to "", and chdir("") fails with ENOENT, which
Rust reports as "unable to run python3.12: No such file or directory"
- about an interpreter that was there all along. resolve_root() now
returns an absolute path. This hit every Docker and tarball install:
empty trail set, detects nothing, no error that says so.
Also `docker compose -f docker/docker-compose.yml up` failed outright:
compose resolves relative bind-mount paths against the compose file's
directory, so ./maltrail.conf meant docker/maltrail.conf.
docker/tests/entrypoint_test.sh asserts all of it against a real daemon
and runs in CI - a harness image around the entrypoint, not the release
image, so it answers in a minute instead of after a Rust build.
Verified end to end: the reporter's layout (host dir 1000:1000, mode 775,
no --user) runs as 1000:1000 and writes; compose comes up healthy; the
sensor runs as uid 10001 with CapEff 0000000000003000 and logged a live
asyncrat DNS lookup from host traffic.
Every event line carries src_ip, src_port, dst_ip, dst_port and proto. The new dashboard showed three of them: the grid had no protocol column, and the detail panel listed destination ports only, so a UDP flood and a TCP scan looked identical unless you opened the raw-event list. The grid gets a 'proto' column next to 'port' - sortable, and filterable with the proto: token that already existed with nothing in the UI to suggest it. Source ports do NOT get a column: they are a pile of ephemeral numbers and the port column deliberately shows the SERVICE side. They go in that cell's tooltip instead, alongside the destination ports, where they cost no width. The detail panel gains 'source ports' and 'protocols' sections, and the existing 'ports' section is now named 'destination ports' - it never showed anything else. Protocol chips filter the table like the severity and type chips in the header do. In the raw-event list an address with no port renders as '10.0.0.1' rather than '10.0.0.1:'. tests/test_frontend.py checks what a JS syntax check cannot: that the columns in index.html and the cells rendered in main.js are the same list in the same order, that the empty-state colspan matches the column count, and that every sortable data-key is a field a threat object actually has. A column added without its cell shifts every value one place left in a table that still renders and still sorts - that is worth pinning down. Verified by mutation: dropping the header, or renaming a drawer section, fails these tests. Rendered and read back out of headless Chromium against the demo data: the proto column shows UDP, the port cell's tooltip lists ten source ports and destination port 53, and the panel shows destination ports 1, source ports 10, protocols 1.
The new file was written, passing, and not being run: tests/run.sh takes an explicit TESTS list and CI runs that script, so a test not named there covers nothing while looking like it does. Added it, then made the omission impossible: the runner now compares the list against tests/test_*.py and refuses to start if one is missing.
A row shows two tags and collapses the rest into "+N", and the "×" that removes a tag only exists on those two. Tag a threat four times and the last two can be removed from nowhere in the UI - the detail panel had no tag section at all, so there was no second place to try. It has one now: all tags, each removable, with an input to add more. The "+N" chip's tooltip says where to go, and clicking it already opens the panel (verified, rather than assumed). Checked in headless Chromium: four tags added through the panel, the one hidden behind "+N" removed, header count and row both following.
The line chart labelled five fixed gridlines with round(max * g / 4). At a
peak of 2 that is 0, 0.5, 1, 1.5, 2 rounded to "0 1 1 2 2" - two pairs of
gridlines carrying the same number, which is what the report shows. Small
peaks are the normal case on a quiet sensor, so this was the usual sight,
not an edge case.
The two bar charts had the opposite problem: gridlines drawn, never
labelled. The Sources breakdown was five lines and no numbers at all.
Both come from having no axis logic. There is one now: axisTicks() picks a
whole-number step from the 1/2/5x10^k progression so about four gridlines
cover the data, and the top tick becomes the scale - which also keeps the
tallest bar off the ceiling. Whole steps mean no two labels can round to
the same text, because events are counted, never halved.
Measured in headless Chromium, instrumenting fillText on the real charts,
filtered to one single-event threat to reproduce the report exactly:
before events axis ["0","0","1","1","1"] sources axis []
after events axis ["0","1"] sources axis ["0","50","100","150"]
Guarded in tests/test_frontend.py: every count chart must derive its axis
from axisTicks() and label what it draws, and axisTicks() itself is run
under node over 25 peaks from 0 to 1.2M, asserting whole ascending ticks,
no repeated label, and a top at or above the peak. Both checks fail if the
rounded-quarters axis is restored.
`core/update.py` called `str.isascii()`, which is 3.7+. That single call
was the entire reason the project claimed a 3.7 floor - and it did not
degrade, it killed the trail update outright:
[!] trail update failed ('str' object has no attribute 'isascii')
Empty trail set, sensor detecting nothing, on every distribution whose
stock python3 is 3.6: RHEL 8, CentOS 7, openSUSE Leap 15 / SLE 15, Amazon
Linux 2. It is also what sent the Rust sensor looking for a versioned
python3.N, and what the installer was about to start installing a newer
interpreter to work around.
Measured before changing anything: every module in core/, server.py, the
feeds and the tools already PARSE on 3.6.15, and the whole server suite -
222 tests - passes there. `str.isascii()` was the only 3.7+ API in the
codebase. So the floor was one line, not a port.
_is_ascii() now keeps the C method on 3.7+ and uses a precompiled regex on
3.6. The verdict matters beyond speed: it decides whether a trail key is
lowercased or punycoded through encode("idna"), so a wrong answer stores
IDN trails under a key that can never match.
Proof rather than inference: the full offline updater now runs to
completion on 3.6.15 and produces the same 1,601,925-line trail set as
3.12. CI gets a `floor` job that runs the suite AND builds a trail set in
python:3.6-slim - offline, so it exercises core/update.py over the whole
static pile without touching a third-party feed from CI. The runners that
could install 3.6 are retired, hence the container.
tests/test_update.py compares the two implementations against each other
on whatever interpreter the suite runs on, so the 3.6 path stays covered
on 3.13, and fails if the fallback is ever deleted.
The sensor's MIN_PYTHON drops to 3.6 with its tests; 3.5 stays below the
floor. Preferring a versioned python3.N is kept - a box whose python3 is
2.7 or a non-Python shim still needs it.
curl -fsSL https://raw.githubusercontent.com/stamparm/maltrail/master/install.sh | sudo sh Dependencies, a shallow git clone into /opt/maltrail, the prebuilt sensor for this architecture, an unprivileged user, log and state directories, /etc/maltrail.conf, systemd units, setcap, enable --now. Re-running it is the upgrade. --role, --ref, --prefix, --no-service, --dry-run, --uninstall. git rather than a release tarball because the trail lists live IN the repository: a clone brings current detection content with the code, an upgrade is a fetch, and --depth 1 leaves the ~1.8 GB of history behind. Configuration lives in /etc so `git reset --hard` on upgrade cannot eat it, and the units are the repository's own with their paths substituted - one source of truth, not a second copy to drift. tests/install/run.sh runs it in ubuntu:24.04, debian:12, fedora:41, opensuse/leap:15.6 and alpine:3.20 and then checks what is actually there: the server is STARTED as the unprivileged user and asked for /ping, the sensor is asked to validate itself with -T against a real trail set, units are checked for an ExecStart that exists and is executable, the installer is re-run to prove operator configuration survives, and --uninstall is run. 98 assertions, all passing. Two limits are stated in the file rather than discovered later: systemd does not run in a container (units are rendered and checked, enable/start is not), and BSD cannot be tested this way at all because containers share the host's Linux kernel. Four things the harness found, none of which I would have got right by reading the script: * Fedora/RHEL/SUSE ship libpcap as libpcap.so.1, Debian and Ubuntu as libpcap.so.0.8. Same ABI, different name, and the loader's message ("cannot open shared object file") reads as if libpcap were absent. The installer now links the missing name to the file that is there, both directions. * --sensor-bin bypassed the musl check, so it installed a glibc binary on Alpine that cannot exec. Whatever produced the binary, the installer now RUNS it and explains what it found: musl, a too-new glibc, or a missing library - a sensor that cannot start is the worst install outcome, because everything looks done and nothing is ever detected. * a commit SHA is not a branch, so `--branch <sha>` fails - which is every CI checkout, since those are detached HEADs. * a package manager unpacking 46 dependencies is not information, so its output is now only shown when it fails. Also in CI, with the sensor built first so the sensor path is covered. Known, and next: the prebuilt binaries are built on ubuntu-latest and therefore need GLIBC_2.39, so they do not run on Debian 12, Leap 15.6 or RHEL. The harness reports it as a finding and the installer says so plainly and points at building from source, but the release build needs an older glibc baseline. That is the next commit, not a footnote.
The guard was "(custom)" in line with an unanchored global sub, so an event whose info merely contained the literal had the token before it replaced for masked users - on an event with no custom trail. redact_json already keyed on the reference field, so text and JSON disagreed about the same event.
prepare() drops a sidecar with its log, but only for a day someone asks about. Maltrail does not rotate its own logs, so logrotate takes the .log and nothing visits that day again - the sidecar stays forever, at ~2-4x the size of the log it indexed. --rebuild-index could not help either: it iterates the logs that exist. Swept with the rest of the daily maintenance. Measured: 6.6 MB of log built 13.5 MB of sidecar; after rotating two of three days away, 9.0 MB was held for logs that no longer existed.
check.sh regenerates them from the local trails.csv, so they got swept into an unrelated commit. Machine-specific by design - not a committed assertion.
_line_in_scope matched \b(\d+\.\d+\.\d+\.\d+)\b, and \b treats '.' as a boundary, so 10.0.0.5 was found inside the domain 10.0.0.5.evil.com. An analyst scoped to 10.0.0.0/8 was shown events whose src and dst were both outside their networks. check_whitelisted documents fixing the same shape. The retro-hunt regex is left alone: that is a search an analyst typed, where matching a domain that embeds the address is useful and a miss is worse.
It took the address from line.split()[3], assuming the quoted timestamp costs two tokens and the sensor name one. safe_value quotes anything with a space, so SENSOR_NAME 'DMZ firewall' published 'firewall' - and 'dmz 10.0.0.1' published an internal address. A JSON line published '"2026-09-02'. fail2ban bans what it is given, so the blocking quietly stopped working. The BLACKLIST reader beside it was converted to logfmt.fields() for exactly this reason; this one was missed.
…hanged request_line() grew two prebuilt-searcher arguments and the target still called it with one. Nothing builds the fuzz crate, so it rotted silently while fuzz/README.md told contributors to run it. memchr goes in the fuzz manifest too: it is a dependency of maltrail-sensor, which covers sensor/tests/* but not this separate crate. All six targets build now; http had never run. 47k runs, no crash.
_event_precedes_clear matched the text format's opening quoted timestamp, so a JSON line never matched and every event read as not-cleared. A host the operator had already remediated stayed on /fail2ban and /blacklist. Also handles the epoch 'timestamp' a LOGSTASH_SERVER line carries instead of a date string.
_tree_snapshot walked the whole repository - 16,193 files, 15,520 of them sensor/target and the fuzz corpus - so any concurrent writer looked like a file smoke_test() created. With cargo fuzz running it failed 2 runs in 3; a plain cargo build does the same. A coverage-guided corpus is meant to persist between runs, so it stays where cargo-fuzz puts it and the snapshot got precise instead. Pruned by path, not basename: sensor/tests/corpus is tracked and still watched.
It found the fields by locating the end of the quoted timestamp and splitting the rest on spaces. A JSON line has no '" ' so it was skipped entirely - a LOCAL_LOG_FORMAT json LOG_DIR drew an empty map, 0 of 5 events reaching event_country. A quoted SENSOR_NAME shifted every index: for 'my sensor' it read type=UDP, src='sensor"', dst=the source port, trail=DNS. The byte split stays for the ordinary line - a full-day scan runs it per line, and the fallback costs ~9ms on a 200k-line day, paid once because /geo scans incrementally.
The Rust sensor uses threads, so there is no ring buffer, no packet copy and no IPC - PORTING_MAP and COMPATIBILITY both say so. Nothing imported it but its own unit test, whose docstring still claimed a bug there would drop packets between capture and detection. It also had a live bug: worker() skips count += 1 when a block is shorter than 12 bytes, and read_block leaves the slot re-readable, so one short block spins the worker forever at 100% CPU. Verified before deleting. BLOCK_MARKER goes with it. BLOCK_LENGTH stays - CAPTURE_BUFFER is rounded down to whole blocks and src/config.rs mirrors that. PROCESS_COUNT stays too, it still drives the log-throttle bucket.
CHECKS is the only thing that runs a check, and the existing guard named one function - a check added later and forgotten would run nowhere with the suite still green. tests/run.sh already enforces the generic form of this rule for test files; same rule here. Also pins run()'s exit contract: FAIL exits non-zero, WARN does not.
Added in 66b2307 and touched once since, by me. The http target called request_line with one argument where that same commit gave it three, so it had not compiled since the day it landed - four weeks, no corpus, no artifacts, no sign of a single run. A README told contributors to run it. It needs nightly for -Z sanitizer, which is why it could not join CI and why nobody noticed. The property it was for is asserted on stable in every cargo test by tests/fuzz_parsers.rs and tests/fuzz_extended.rs, fixed-seed so a failure reproduces; MT_FUZZ_SEED turns the second into a campaign. Also frees 952 MB of local build tree that regenerated on every run.
maltrail.conf listed six of the eight mutable heuristics, omitting beaconing and dns_tunneling - while three lines above it told the operator to "Mute it with: DISABLED_HEURISTICS dns_tunneling". So the newest heuristic was documented nowhere as mutable. And an unrecognised name was accepted in silence, leaving the heuristic the operator meant to mute still firing - the same failure unknown_keys() already warns about for option NAMES, one level down in the values. Tests guard both directions: the conf list must equal HEURISTIC_NAMES, and the worked example must use names the sensor answers to.
detect_test asserted its detections then deleted the log, so nothing was left to look at. --keep writes it somewhere durable, replays the 42-capture parity corpus into the same dir (JA3, beaconing, DGA, DNS exhaustion, TLS/QUIC SNI, sinkhole, encapsulations), shifts the timestamps so the newest day is today - the fixtures are dated 2023, and the dashboard opens on today - and prints which of the 15 shapes the UI draws differently have an event behind them. --serve then starts the web server on it. 102 events, all 15 shapes covered.
demo.js is 2,500 events of real captured traffic and its value is that it looks real - 557 trails over 13 hours, power-law distributed. Regenerating it from scratch would cover every class and look like a test matrix. So gen_demo_js.py does not regenerate: it keeps the file as the base, finds the shapes the dashboard draws that are missing from it (IPv6, a (custom) origin, ICMP, a second sensor), takes real examples from a --keep run and blends them in - relabelled into the demo's sensor and day, in small clusters, interleaved. 77 added to 2,502; the distribution is unchanged. Fixture names are relabelled on the way in: 'apt test (malware)' is right for a test and wrong for a demo. A condensed SOURCE list is deliberately not synthesised - src_ip is the condensing key in core/log.py, so the sensor cannot emit one.
feeds/statics.py fetched maltrail-static-trails.txt from it on every update. That URL is now 404, so the feed returned zero trails every run - and core/update.py's own comment says a feed empty on every update 'is dead and should be removed, not tolerated'. The static trails come from STATIC_TRAILS_URL (stamparm/trails) since the 3.2 split; this was a second, stale source for the same thing. README advertised a derived blacklist published there (404), and four whitelist entries cited aux issues. Three of those four keep their maltrail commit reference; the citations themselves pointed at a deleted repository. 43 feeds -> 42. Whitelist entries unchanged (3604), only comments removed.
The coverage list only checked RENDERING shapes - icons, glyphs, condensed cells - so every heuristic added since the 2024 capture was still absent, periodic beaconing among them. It is detection CLASSES that matter; 25 are covered now, including JA3 as a trail type. Two needed traffic that did not exist: DNS exhaustion (1000 distinct subdomains - the corpus pcap asserts nothing and never crosses the threshold) and DNS tunnelling. The obvious label generator for the latter reaches only eight distinct characters, giving 2.98 bits against a threshold of 3.00, so not one query counted as carrying and the heuristic stayed silent. --thin halves the base's distinct trails: 557 -> 295, since old bulk buries a beaconing cluster of 14. It protects one carrier of every distinct info and trail type first - feed infos like 'ipinfo (suspicious)' have no donor in a sensor run, so a dropped last carrier could not be re-added. Distinct info strings go UP, 47 -> 64. README documents how to run it.
aggregateRows() rejects a row whose field 7 is not a trail type - that keeps a shifted or truncated line out of the aggregate. The test was /^[A-Z]+$/, letters only, and the sensor emits TRAIL::JA3 and TRAIL::JA4. Both carry a digit, so every TLS-fingerprint detection was discarded with no error and no count: absent from the table, the type breakdown and the charts. Three occurrences, all widened to /^[A-Z0-9]+$/. On the demo data the threat count goes 372 -> 376, and searching 'ja3' returns results instead of nothing. Found by building the public demo site and searching it. A test now checks every TRAIL:: / TRAIL. name in the sources against the character class in main.js, and that the guard still rejects things that are not types. Also: main.js no longer overwrites the version with 'redesign prototype' when one has been substituted - only when the span is empty, which is the case it was for (a file opened raw, where the parser drops <!VERSION!>).
Fixing the Reference after being broke by ed997d0 commit
The sticky header was rgba(...)+backdrop-filter:blur(10px). A blurred layer is re-composited whenever what is behind it moves, and a full-width sticky header means every frame of every scroll. It is now opaque #0d131f - the colour that rgba(14,20,32,.82) already resolved to over the page ground, so it looks the same. The modal overlay and the drawer scrim gave up their blurs for this reason already; both carry a comment saying so. This was the last one. mtpulse animated box-shadow, which cannot be composited, so .period .dot and every .worstasn repainted on every frame for as long as the page was open - and .worstasn is created PER ROW, so a busy day meant one continuous repaint per malicious ASN. It animates opacity now; the static glow carries the look. Confirmed on Firefox/Ubuntu by hand: reported 'slightly snappier' with the header blur off. Headless Chromium could not reproduce it - software raster and no visible window exercise neither GPU compositing nor CSS animation, so 250 rows measured an identical 16.7ms median with and without the blur.
I relabelled the fixtures' RFC 5737 / RFC 3849 addresses to 'realistic' ones and published the result. 2a03:2880:f12d:83:face:b00c::1 is Meta Platforms Ireland and it went out labelled 'cobalt strike beacon (malware)' 22 times; 185.220.101.47 is a live Tor exit node labelled 'wannacry (malware)'. Three more were real allocations. Documentation ranges look synthetic because they are, which is the entire reason they exist. Addresses are no longer relabelled - only names are, and both invented domains are NXDOMAIN. The IPv6 fixture moves from dead::beef (unallocated space) to 2001:db8::beef (RFC 3849). Synthesised events also drew their source from the base capture's whole mix, so 80 of 181 showed a public address in the SOURCE column - as though the deployment were monitoring somebody else's network. They now use the busiest LAN hosts of the base capture. 0 of 141 have a public source. The tool refuses to write if any synthesised event carries a real address in src or trail. dst is exempt: for a DNS detection that is the resolver being queried, which the base capture itself shows as 8.8.8.8, and naming a resolver accuses nobody. Copies of base rows under a second sensor name are exempt too - they are real capture data, not inventions.
The base capture was taken ISP-side: the monitored hosts were public 2.200.x.x subscribers and 10.1.20.50 was the resolver they queried. So 64% of events had a public SOURCE, Google's own 172.217.40.x and 173.194.170.x sat inside the network we claim to monitor, and 8.8.8.8 was the source of a malware detection. It reads as though we monitor the internet and our LAN box is the victim. Which end is ours is not the same for every event, and core/geo.py:event_country already owns that decision, so this mirrors it rather than inventing a second rule: a resolver answering us (src port 53) and the inbound heuristics (PATH web scanning, PORT infection, a scan whose trail IS the source) put our host at the DESTINATION; everything else is outbound and our host is the source. 364 addresses on our side moved to RFC 1918 keeping their /24 grouping, so the 'a few busy subnets' shape survives. 183 on the external side moved to RFC 5737. The two busiest resolvers became 10.1.20.50/.51 next to the internal resolver the capture already had. Public resolvers stay real - a resolver is not an accusation. No real-world allocation is left anywhere. blend() rewrote the source unconditionally, which put a LAN host in the attacker's seat for the inbound heuristics and gave each event of one web scan a different scanner - not what the heuristic detects. It follows the direction now, and the 11 events it had already mis-shaped are one external scanner against one LAN web server. demoGeo() geolocated the first public SOURCE. That is right only for an inbound attack, and it appeared to work solely because the capture was ISP-side; with the hosts correctly private it would have mapped nothing and the published map would have rendered blank. It mirrors event_country too, so DNS is honestly unmapped - 343 mapped, 1370 local/unmapped, which is what the real server shows. dead::beef is unallocated space rather than documentation, so the IPv6 fixture moves to 2001:db8::beef. Three tests, each watched failing first: a public address in the monitored position, a real allocation as the accused trail, and demoGeo reverted to source-only.
The demo had no server to ask, so it faked geolocation: murmur3(ip) indexed into a 20-country list. That put 8.8.8.8 in Sweden and 8.8.4.4 in IRAN, and 1.1.1.1 in Ukraine. The ASN was faked the same way - 'AS' + hash, holder 'Example Networks NNN' - so a real address also got a made-up network. Users notice exactly this and stop trusting everything else on the page. We ship the real table. data/ip2cc*.csv.gz is RIR delegation data and is what the server geolocates with, so the generator now computes a country per address and emits getDemoGeo(). The dashboard reads it. An address with no country - RFC 1918, RFC 5737 - is absent and gets no flag, which is what the server does; it no longer sits on 'locating...' forever either. The invented ASN is gone rather than replaced, because no ASN table ships with the demo. That only works if the addresses are real, and my last pass had moved all 183 external endpoints into documentation space, which geolocates to nowhere. The rule I should have applied is narrower: a real address may appear only with the pairing that was really observed, and an invented pairing must use a fictional address. So the capture's own external endpoints are real again - they are genuine observations - while the 177 events I synthesised keep documentation addresses. Our own hosts stay RFC 1918. 186 addresses now carry a real country across 15 countries, and nothing is fabricated. Checked afterwards: of the 177 invented pairings exactly one touches a real address, 9.9.9.9 as a resolver, which is what Quad9 is. Three tests, each watched failing: 8.8.8.8 restored to Sweden, a real address dropped from the table, and the hashed country list put back.
The demo shows a Log in button that opens a password prompt posting to /login, which does not exist in a static build - checkAuth() only hides that button on the server path, so the demo never ran it. The day picker's heat grid is built from /counts, so the demo was fabricating a density per day by hashing the date: decorative event counts for days that hold no events, in a picker whose day cannot be navigated to anyway. Both the picker and pickDay() had been bypassing the disabled-control check specifically for demo builds; they now honour it, and fetchCounts() invents nothing. Paging, sort and search still work - those need no server. Four tests, each watched failing first.
demoCSV() rebases the demo's events onto today by replacing a literal "2024-01-11". Nothing tied that literal to what demo.js actually holds, so regenerating the demo on any other day would make the replace match nothing and silently freeze every timestamp years in the past. It now reads the days out of the data and shifts them all by the same number of days, so a multi-day demo keeps its span. Times of day are untouched, which keeps the hour histogram intact. Also drops a comment that justified the rebase by the day picker agreeing with the rows - a demo no longer opens the picker.
…, #19622) #19622. 'potential iot-malware download' scored HIGH because its info contains 'malware', level with a proven C2 callback. 307e0e8 then demoted every '(heuristic)' + '(suspicious)' event to LOW, which was the other extreme. It is now capped: a sensor guess is LOW by default, MEDIUM when it names something concrete (a dropper URL, one host spraying 445), never HIGH. Only a feed hit earns HIGH. The same heuristic also fired on traffic that was none of our business. It checked the destination alone, so an inbound or transit request carrying an architecture-tagged path was filed as our infection - a false positive with a loud severity. It now requires the source to be ours as well: RFC 1918, or the learned local prefix for a site that numbers its own hosts publicly. #19621. Four counting heuristics emitted a bare info while every other heuristic carried a class marker, so they rendered with no class icon and skipped the severity rule above. port scanning, udp scanning, web scanning and infection now say '(suspicious)' too. Severity is unchanged by design: scanning stays LOW, infection stays MEDIUM - the demotion this could have caused is what the MEDIUM keywords prevent. CONDENSE_ON_INFO_KEYWORDS matches on substrings, so condensing is unaffected, and pre-existing logs still rank the same. demo.js updated to match, and the severity table now covers both spellings. Every test watched failing first, including the direction guard: with it removed, the two-outsiders case reports the event.
Five projects that never run Maltrail fetched a derived domain list from stamparm/aux: NextDNS, NoTracking, pfBlockerNG-devel, MobSF and MobileAudit. That repository is gone, so the URL 404s and all five now pull nothing. They cannot use trails.csv instead - it carries IPs, URL paths and regex trails they would try to resolve as hostnames. core.assemble --domains-out writes what they need: the malware-category trails, names only, sorted and de-duplicated. Sorting is safe here even though the aggregate must not be sorted - that rule protects label attribution, and this output has no labels. It refuses to write an empty list, which downstream would apply as 'block nothing' without noticing. 966,481 domains from the current content, 18 MB. Nothing outside [a-z0-9_.-] survives the filter: no IP, no URL path, and none of the regex trails, one of which loose in a resolver blocklist would block whatever it matched. stamparm/trails publishes it beside the aggregate on the same 4x/day schedule, uncompressed and under its historical name, since all five hardcode a plain text URL.
1rpc.io and public.1rpc.io are 1RPC's public RPC endpoint, listed as revstealer indicators because the stealer reached a blockchain node through them. 1rpc.io is a top-30k domain, so the trails gate has been failing on it. Dropped from stamparm/trails and whitelisted here. The second half matters more than the one domain. update_trails() drops whitelisted trails when it builds trails.csv, but the projects that aggregate maltrail-malware-domains.txt never run it - so whitelisting a false positive fixed our own sensors and left every downstream resolver still blocking it. malware_domains() now applies the same check_whitelisted() predicate, which is where a whitelist entry has to take effect for a redistributed list to be safe. The predicate is injectable so the tests do not depend on the live file, with one test asserting the default really is the shipped whitelist.
The feeds never adopted the '(malware)' / '(malicious)' / '(suspicious)' convention. Static trail files get it free from their filename and every heuristic ends in it, but 28 of the 42 feeds emit a bare word - so a urlhaus or openphish row rendered with no class icon at all. Those strings are not new and not wrong: urlhaus has read 'malware' since it was added in 2018, openphish 'phishing' since 2015, unchanged but for parsing fixes. People match on them in their own tooling, so classOf() infers the class for the icon rather than rewriting eleven-year-old output. Severity is deliberately untouched - severityOf() reads the info separately and every one of these keeps the rank it had, 'known attacker' LOW and 'phishing' MEDIUM included. A test pins that, because the tempting version of this fix was to relabel the feeds, which would have moved them. The info cell also stripped whichever parenthetical came last whenever a class was found. With a class now inferrable without one, that would have rendered 'bad reputation (tor node)' as 'bad reputation' and lost which kind of node it was, so it strips only a real class marker. Three tests, each watched failing first.
Severity was computed twice - REMOTE_SEVERITY_REGEX for the JSON severity
field, CEF priority and alerting, severityOf() for the dashboard - and nothing
compared them. They disagreed on 10 of 27 representative verdicts.
The regex could not agree even in principle, because severity_for()/severity_of()
saw only the info. Whether a verdict was corroborated lives in the REFERENCE:
'(heuristic)' is the sensor's own guess, '(static)' is a feed hit. Both now match
against '<info> <reference>', which cannot break a custom regex - the search is
unanchored, so a longer subject only ever matches more.
With that, the shipped regex says what the dashboard already did:
* iot-malware moves from the high group to medium. It was HIGH here while the
dashboard said MEDIUM, and tests/test_alert.py asserted HIGH - two green
tests, opposite answers, one event.
* '(suspicious) (heuristic)' joins the low group, which is the rule 307e0e8
added to the dashboard and never brought here. 'long domain (suspicious)'
read LOW on screen and paged anyone on ALERT_SEVERITY=medium; the detect
test alone fires it 160 times.
* 'potential infection' joins medium, so the rule above cannot demote a host
spraying 445 across the network.
tests/test_severity_parity.py is the guard. The heuristic verdicts are
discovered from sensor/src rather than listed, so a new one cannot arrive on one
side only - 21 found today - and it asserts the property underneath both rules:
nothing the sensor concluded on its own outranks something a feed listed.
Every test watched failing first, including both historical drifts re-created.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.