Fleet usage ledger: a versioned record of who uses dol, how, and at which commit

[thorwhalen]

Intent record (maintainer, 2026-08-21). Several upcoming programs — the transform-vocabulary migration, the cruft audit, and any change to the wrap_kvs surface — all need the same substrate: a reliable, versioned answer to who uses dol, how, and at which commit.

What exists today

A local-only inventory (gitignored misc/data/, regenerable by script) already records an import-level census of all 85 direct dependents: file:line import records, per-dependent short commit / branch / dirty state, a symbol fan-out index (which dol names each package imports), a test-gate order, and a uniform test runner. A separate one-off side scan holds the only call-site kwarg data we have (12 wrap_kvs call sites across 5 repos). Numbers are from the 2026-08-03 scan — rescan before relying on them.

The ask: grow the snapshot into a ledger

1. Pin manifest — full commit hashes (currently short); a per-dependent baseline test result (status, exact command, duration) captured at scan time alongside the dol commit; and a restore helper that puts the fleet back at the pinned state. Purpose: back-compat tooling can be exercised against a frozen fleet, and a red suite under a dol candidate can be distinguished from a pre-existing failure.
2. Deeper granularity (AST, not regex) — subclass detection (who actually subclasses Store/KvReader/KvPersister, vs merely importing them) and a call-site kwarg census (who passes key_of_id/obj_of_data/postget/preset/…). The current regex scan stops at import statements.
3. Zero-usage report — computed from the ledger (public exports vs detected usages), as the authoritative input to the cruft audit. Two caveats the report must carry: several packages do from dol import * (usage invisible to an import scan), and ~42 names are imported from dol submodules without being in __init__ — a compat surface any cleanup must respect.
4. Successive scans — timestamp + schema version per scan; a ledger of scans over time, not a single overwritten snapshot.

Privacy discipline stays as-is: outputs live in gitignored local storage, no absolute paths.

What this unblocks

• The vocabulary-migration program (frozen-fleet back-compat testing, kwarg census to scope the break).
• The cruft audit (authoritative zero-usage list).
• The dependents test-gate for any wrap_kvs-surface work (P2 of the flat-model roadmap, misc/docs/dol_roadmap.md).

[thorwhalen]

Built (2026-08-21). The census is now a ledger. It lives where the v1 inventory did — gitignored misc/data/ — as a small package, fleet_ledger, with a README.md next to it. The three v1 scripts still work: legacy-export re-emits dol_ecosystem_usages.json in the v1 shape from the newest scan, so run_dependent_tests.py and gen_dependents_md.py now read current data instead of a frozen snapshot.

python -m fleet_ledger scan          # ~25s -> one immutable scan record
python -m fleet_ledger baseline      # ~3min at --jobs 4 -> every dependent's suite result
python -m fleet_ledger pin check | restore --apply | undo --apply
python -m fleet_ledger diff A B      # which tests newly fail between two baselines, by node id
python -m fleet_ledger history | report | zero-usage | legacy-export

The four asks

1. Pin manifest. Full 40-char SHAs per dependent, plus branch, upstream, ahead/behind, dirty-file count and commit date, in a pins.json kept separate from the (0.6 MB) full record so the restore helper never has to load it. pin restore detaches every clean repo at its pinned commit; it refuses dirty repos, never fetches, never forces, and writes its undo record before the first checkout, so an interrupted run is still undoable with pin undo. Defaults to a dry run.

2. Baselines. Per dependent: status, the exact command (stored, not assumed), duration, pass/fail/error counts, and the set of failing test node ids — so comparing two runs is a set difference that localises a regression rather than just flagging one. Deliberately not -x: fail-fast gives you the first failure and hides the shape of the suite. diff refuses to compare two baselines captured with different commands, which caught a real trap during development (switching -rf → -rfE made three repos' collection errors appear for the first time, reading exactly like three new regressions).

3. AST census. Replaces the regex scan:

• Subclassing, not importing — a class statement whose base resolves to the dol symbol, through any binding form (from dol import X, X as Y, dol.X, dol.trans.X, from dol import base → base.X), then closed over each repo's own class graph so a class three levels removed from KvReader still counts (marked indirect).
• Call-site kwargs — every keyword passed at every dol call site, with the kind of each value (lambda / named function / attribute / call), which is what decides whether a mechanical rewrite is possible. Also whether the call is a decorator, and whether it decorates a class.
• Star imports are no longer a blind spot — free names in a star-importing module are resolved against the star'd module's own public surface (from dol.errors import * scores against dol.errors, not __init__), and yield to any local binding. Such records carry via: "star" so the lower confidence survives into the aggregates.

4. Successive scans. ledger/scans/<utc-timestamp>/, schema_version on every record, an index.json, and mark-superseded for flagging a scan as untrustworthy without deleting it — a ledger that deletes its own history isn't one.

Every output is scrubbed of absolute local paths at write time, and write_json refuses to persist a string that still looks like one, so a privacy leak fails loudly instead of landing on disk.

Refreshed headline numbers (2026-08-21 scan, superseding 2026-08-03)

	2026-08-03	now
direct dependents	85	89 (121 transitive, of 217 registry packages)
__init__ exports	147	148
zero fleet usage	86	82 — of which 58 public >1 year
submodule-only compat names	~42	47
star-importing packages	(unquantified)	5

New, because v1 couldn't measure it:

• Core-class subclassing: 29 dependents subclass Store/KvReader/KvPersister/Collection, across 141 classes. 3 more import one of those and never subclass it. Widening to any dol base: 40 packages, 183 classes.
• Files is subclassed by 9 packages across 16 classes, and Collection by 5 across 35. Worth knowing before P2 decides what the flat engine's product is.
• wrap_kvs: 43 packages reference it, 40 actually call it, across 112 call sites — 29 of those as a class decorator (the population Track B's is-a change would move).
• Kwarg census: obj_of_data 28 pkgs / 59 sites · key_of_id 16/27 · id_of_key 16/25 · data_of_obj 14/21 · postget 8/12 · preset 3/3.

Two findings that change downstream scope

#92 is narrower and more urgent than "three vocabularies". dol/trans.py already accepts key_encoder/key_decoder/value_encoder/value_decoder as aliases onto the four X_of_Y names — the successor spelling isn't prospective, it shipped. And the fleet is already split across both: 34 packages / 132 call sites on X_of_Y versus 9 packages / 28 sites on encoder-decoder, with 5 packages using both. Meanwhile the other two vocabularies are nearly free to settle: kv_wrap is called by 2 packages (5 calls, 4 as class decorators, none passing outcoming_*/ingoing_* by name), and key_codec/value_codec have zero fleet call sites. So the work is converging two live spellings of the same four arguments — not reconciling three peers.

#94's candidate pool is 58, not 82. The other 24 zero-usage exports are the surface shipped in the last twelve months (the content module, autoviv, path_set_writeback/path_del_writeback, PathContext, FirstArgIsMapping, confirm_overwrite, unravel_key) — nothing has had time to adopt them, so their absence of usage is not evidence about them. The report now splits on that automatically. It also splits out one incidental export: a bare import os at the top of __init__.py leaks dol.os onto the export surface, which is a hygiene fix rather than a cruft decision.

The zero-usage report carries three generated caveats, not two. The third is scope: "zero usage" means zero usage in this local fleet, and dol is on PyPI. The fleet census bounds the migration cost we control; it does not license a silent removal.

A hazard the ledger now records

import dol currently resolves to a PyPI wheel in site-packages, not the local source tree. Today the two are byte-identical (both 0.3.63), so this scan's baselines are valid — but a test gate that measures the published package instead of the candidate is worse than no test gate. Every scan now records what import dol actually resolved to, and the report says so in its header. Fix before gating a candidate.

Verification

The census was hand-checked against a sample of dependents before being trusted, and that found two real defects, both fixed and both now frozen as regression tests (fleet_ledger/test_astscan.py, 13 cases):

1. A class rooted both directly and through a local base reported only its direct base — a 3× undercount of Collection (12 classes reported, 35 actual).
2. A submodule star import (from dol.errors import *) was being matched against __init__'s exports instead of the submodule's.

Spot-checks confirmed against source: alias resolution reports the dol name rather than the local one; indirect chains are real; and the encoder/decoder call sites (6 key_decoder, 1 key_encoder) are exactly where the ledger says they are, with the argument kinds it claims.

Baseline snapshot

69 of 89 dependents have a suite. At the pinned state: 62 PASS, 6 FAIL, 1 NO_TESTS, 158s wall clock at --jobs 4 (446s of suite time). The 6 red are pre-existing — mostly collection errors from absent optional drivers, plus one genuine test failure. That is the point: none of them should ever again be read as damage from a dol candidate.

Open, for you

• Should the ledger tool stay gitignored? Its outputs must — they name private packages. The code needn't, and being gitignored means it can be lost with the directory. I left it under misc/data/ for consistency with the v1 scripts rather than move it unilaterally.
• The fleet's py2store legacy imports and the "no test suite" list (20 packages) are recorded per scan but nothing consumes them yet.

[thorwhalen]

Correction to the baseline figure above, and a defect worth knowing about — it was in the v1 gate too.

Reviewing the generated health section, one line didn't survive scrutiny: a package I had just verified by hand as having tests was listed under "no test suite". Chasing it found that 5 of the 20 so-listed dependents do have suites — including two high in the gate order. They had been silently excluded from every test-gate run, v1's included.

The cause is the has_tests heuristic inherited from scan_dol_usages.py: a top-level tests/ directory, or a test_*.py anywhere. That misses a tests/ package inside the source package (the common i2mint layout), and it misses *_test.py naming — which is what pytest's own default python_files accepts and what several of these repos use.

The fix isn't a better heuristic. Only pytest knows what pytest collects, so the gate no longer asks: baseline now runs every dependent, and pytest's "no tests collected" exit code 5 is recorded as NO_TESTS. A repo with nothing to collect exits in well under a second, so running all 89 instead of a filtered 69 costs almost nothing — and nothing can be dropped by a guess. The filename check survives only as a labelled hint in the health section, next to the authoritative line from the baseline.

Worth stating plainly: had this not been caught, the ledger would have shipped a gate that quietly skipped real suites while reporting itself complete — the exact failure mode a ledger is supposed to prevent. It was found by reading the generated report against source, not by the tests.

Revised baseline — all 89 dependents, not a filtered 69: 65 PASS, 6 FAIL, 18 NO_TESTS, 171s wall clock at --jobs 4, 88 of 89 at their pin (one dependent had a dirty tree and was correctly left alone). The three recovered suites contribute real coverage that the gate had never been running: 31 tests, 11 tests and 1 test. The FAIL set is unchanged — the same 6 pre-existing reds.

Worth one more note: even after widening the filename check, the hint and the authoritative answer still disagree on 3 packages in both directions — some repos have test-shaped filenames pytest won't collect, others collect plenty with no test files at all (via --doctest-modules). Which is the argument for not having a heuristic in the decision path at all.

The superseded scan is marked as such in the ledger index rather than deleted, with the reason attached.