feat(check): name the registries that declined to answer - #141
Open
justin13888 wants to merge 9 commits into
Open
feat(check): name the registries that declined to answer#141justin13888 wants to merge 9 commits into
justin13888 wants to merge 9 commits into
Conversation
`ManifestCheck::registry_unreachable` was one boolean per manifest, so a
`--fail-on` gate could refuse to certify a run but never say which registry
had declined. A manifest is not the routing unit either: `route_item` sends a
`deno.json` to npm and to JSR, and a `Cargo.toml` to crates.io and to every
alternate registry its dependencies name, so a per-manifest — or even a
per-ecosystem — answer collapses routes that failed independently.
`fetch_all` now records the route behind each failed lookup, keyed by the
cache key that already distinguishes those routes, and returns them sorted so
`buffer_unordered`'s arrival order cannot reach the message. `ManifestCheck`
gains `unreachable_registries`; the boolean stays, derived from the collection
at the one site that builds both, because this crate is published and the
field is part of its surface.
The refusal now names them: "the Go registry did not answer", "the Go and npm
registries did not answer". A root that is not the ecosystem's own default is
named beside it — `npm (jsr.io)` — reduced to host and port, because a
configured root may carry credentials (`.npmrc` interpolates `${VAR}`) and the
refusal is printed on stderr, which CI captures as job output.
The 404 carve-out is untouched: the predicate that decides what counts as a
registry declining to answer is character-identical, and an empty collection
pushes no reason at all.
Refs #112
Two end-to-end tests the old boolean could not express. `checker.rs` drives a `deno.json` against two servers: npm answers the packument, JSR answers 500. Both routes are `Ecosystem::Npm`, so this is the case that separates a per-registry answer from a per-ecosystem one — the unreachable set names JSR alone, by JSR's own host, while the npm import is evaluated in full. `cli_gate.rs` runs the real binary with npm pointed at one loopback registry and Go at another, and asserts exit 2, that the refusal names the Go proxy, and — the assertion the issue is actually about — that it does not name npm, which answered every request. The 404/410 carve-out tests are unchanged apart from the pinned wording they assert against. Refs #112
The exit-code section said a run that could not reach a registry exits 2, but not what it prints. It now shows the one- and two-registry sentences, says that a non-default root is named beside its ecosystem and reduced to host and port so a credential in a configured root cannot reach CI output, and repeats that a 404 or 410 is an answer rather than an outage. Refs #112
Four defects raised against the previous commit before review. `authority_of` split the path off before stripping userinfo, so a password containing `/` — a base64 token contains one about 40% of the time at 32 characters — survived as the "host": `https://ci:AbC/dEf@nexus.internal/npm` reduced to `ci:AbC`, and the gate printed a username and a token prefix on stderr, where CI captures it as job output. The same held for `?` and `#`. This is a redaction, so it now fails towards saying less: the authority candidate is taken first, and an `@` surviving outside it means the string cannot be split into userinfo and host with any confidence, so the root is dropped and the ecosystem named alone. `\` joins the separators because WHATWG resolves it as `/` for special schemes, so reading it as part of the authority named a host the run never contacted. `unanswered_registries` deduplicated labels by adjacency under a sort keyed on the raw root, but several roots reduce to one label and need not be adjacent: three Rust registries at `http://nexus.corp/a`, `http://other.host/x` and `https://nexus.corp/b` sort in exactly that order, and the sentence named `nexus.corp` twice. Labels are now sorted before they are deduplicated, which also makes the printed order the order a reader sees. `ManifestCheck::unreachable_registries` promised deduplication per registry but deduplicated per route, so two alternate-registry aliases naming one index URL reached a library consumer as two identical entries. The CLI masked it. `label()` now says what an authority-only name cannot distinguish — two paths on one host — and `name_unanswered` no longer claims to have avoided a nesting it in fact produces. Refs #112
…' into feat/112-per-registry-unreachable
`authority_of` asked whether an `@` survived past the authority as a match
guard on the `None` arm of the userinfo split, so the question was put only
when the authority candidate held no `@` of its own. A root whose userinfo and
whose path both contain one took the `Some` arm instead and reduced to the text
between the embedded `@` and the first delimiter — a fragment of the token, and
a host the run never contacted:
https://ci:AbC@dEf/ghi@nexus.internal/npm -> dEf
Such a root makes every request to it fail, so the refusal printed it on every
gated run, into CI job output. Seven shapes leaked, across `/`, `?`, `#` and
`\\` as the delimiter.
Hoist the check above the split so it applies unconditionally. Whenever
`authority_of` now returns `Some`, every `@` in the string lies inside the
authority, so what survives is a host and port and never userinfo. Verified over
a 40-input corpus: the seven leaking rows become `None` and no other row moves,
including every row this branch already asserted on.
The seven are added as regression cases, and `authority_of`'s doc comment and
the README line describing the refusal are corrected — the README now also says
that the line degrades to the bare ecosystem name rather than guessing.
…edentials `label()`'s doc explains at length that a root may carry credentials and must never be printed whole, but a consumer reading the field it is handed got no signal at all — its doc said only where the root came from. The argument that a second consumer must not have to reimplement the redaction needs the field itself to say so, and to name `label()` as the only safe rendering.
…roduces it `unreachable_registries_sort_by_ecosystem_then_root` re-applied `sort_by(unreachable_sort_key)` and `dedup()` in its own body over a hand-built vector, so it exercised the key and not `fetch_all`'s use of it. The one integration test that reached the production site yielded a single entry, so it passed either way: deleting both lines from `fetch_all` left the whole suite green, because the CLI re-sorts and re-dedups downstream and that half is covered. The library's published ordering — which an embedding consumer reads straight off the value — was untested. Add an integration test in which both of a `deno.json`'s registries decline, and assert the returned `unreachable_registries` order directly. Both routes are `Ecosystem::Npm`, so only the root can order them. The outcomes are gathered in a `HashMap`, whose iteration order is seeded per map, so one check would agree by luck half the time; the check is repeated concurrently through fresh `Checker`s, which costs one round of retry backoff and drives an unsorted `fetch_all`'s chance of passing to 1 in 256. Measured: 30/30 runs pass as written, 30/30 fail with the sort deleted from `fetch_all`. The unit test keeps its coverage of the key and the dedup, with its doc comment corrected to claim only that.
…' into feat/112-per-registry-unreachable
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
ManifestCheck::registry_unreachablewas one boolean per manifest, set at a single sitein
Checker::fetch_allwhen any lookup failed for a reason other thanNotFound. A--fail-ongate could refuse to certify a run, but could only say "the registry did notanswer" — never which one. In a polyglot monorepo where
proxy.golang.orgtimed outwhile npm, PyPI and crates.io answered every request, the refusal named nothing anyone
could go and check.
Per-manifest is not the routing unit either.
Checker::route_itemproduces threedistinct routes inside
Ecosystem::Npmalone — the default registry, JSR, and aper-scope private registry from
.npmrc— and aCargo.tomlreaches crates.io plusevery alternate registry its dependencies name. Each already has its own cache key.
So
fetch_allnow records the route behind each failed lookup, keyed by that cachekey, and
ManifestCheckreports them:dependable_fetch::UnreachableRegistry { ecosystem, root }, built withUnreachableRegistry::new, withlabel()rendering the name a message prints.ManifestCheck::unreachable_registries: Vec<UnreachableRegistry>, sortedby ecosystem then root and deduplicated per registry.
ManifestCheck::registry_unreachablestays, derived from that collection at theone site that builds both, so the two cannot disagree. Additive, not breaking.
ManifestReport, re-sorts, andnames them.
The refusal reads:
error: cannot honour --fail-on: the Go registry did not answererror: cannot honour --fail-on: the Go and npm registries did not answererror: cannot honour --fail-on: the Go, JVM and npm registries did not answerA root that is not the ecosystem's own default is named beside it —
npm (jsr.io)—which is what tells JSR apart from npm inside one
deno.json. The root is reduced tohost and port before it is printed:
.npmrcinterpolates${VAR}into itsregistrylines and
.dependable.tomlroots are hand-written, sohttps://ci:secret@nexus.internal/reporenders as
nexus.internalrather than putting a credential into CI job output.Unchanged by construction: the predicate that decides what counts as a registry
declining to answer is character-identical (
!matches!(e, FetchError::NotFound(_))), anempty collection pushes no reason at all, and the
FailOn::AnyandFailOn::Noneearlyexits are untouched. The existing 404/410 carve-out tests pass without edits.
Closes #112
Decisions taken
1. Deliverable boundary — how far the routing key goes
ManifestCheckreports WHICHregistries did not answer, and have the gate's refusal name them. The
partial-certification opt-in the issue also proposes is filed, not built.
ManifestReport::ecosystemwithouttouching
dependable-fetch— it is cheaper, and per-ecosystem granularity really isderivable today, but it adds no routing key anywhere and the issue's own words are that
ErrorOrigin::Unanswered"already carries the per-dependency half; what is missing isthe routing key alongside it". It is also wrong at the margin: a
deno.jsonfailingonly against JSR would report "npm did not answer", which is the same class of
imprecision the issue was filed about, one level down.
DID answer — that changes what a gate PROMISES, not its granularity.
gate_is_answerable's doc comment is an argued position that a gate whose inputs aremissing must fail, and extending it needs that argument made, not a flag added. It also
carries an unresolved semantic: which results are certified when one
Cargo.tomlroutes to two registries and only one answered.
check --format json— no integrity data appears inANY output document today, so this would be the first, and it would set a precedent
about what belongs in the check document as a side effect of a granularity change.
ManifestCheckand derive the message fromManifestReport::ecosystemingate_is_answerable.2. What the routing key IS
a bare string. The message prints the ecosystem name, and appends the root only when it
differs from that ecosystem's default registry, reusing the exact discrimination
default_cache_keyalready makes for deciding whether a cache key needs scoping.Ecosystemalone — it is provably not the routing unit.route_itemproduces three distinct routes within
Ecosystem::Npmalone: the default registry,JSR, and any per-scope private registry from
.npmrc. The issue names exactly thosecases as why per-manifest is the wrong unit, and per-ecosystem reproduces the same
collapse one level down.
RegistryFetcher::registry_root()has atrait default returning
None, so a third-party fetcher yields no label and the typeis
(Ecosystem, Option<String>)in practice anyway. Structuring it that way from thestart is honest about the fallback rather than patching around it.
cache_keyverbatim — it is already at the failure site and costsnothing, but
default_cache_keyrenders a non-default root as{osv_name}-{16 hex digits}, so a corporate Maven mirror would printmaven-9f3c1a2b4d5e6f70. It also freezes an internal cache-namespacing convention as apublic contract.
ecosystemfield and drop the root from themessage.
3. How
ManifestCheck::registry_unreachablechangesCONSTRUCTION in one place so the two cannot disagree. The commit is
feat:, notfeat!:.dependable-fetchis a published crate and the repository CANNOT establish from withinwhether a version carrying
registry_unreachablehas shipped (the field originates infix: stabilization pass — twelve correctness fixes across core, fetch, cli and report #99, still open). A
!commit resting on an unverifiable assumption, in a stack fourpull requests deep, is the wrong risk — especially as feat(cli): implement --ecosystem rather than leaving it removed #105 documents that this stack has
already mis-signalled one CLI break through a
fix:commit.backed by a private field —
ManifestCheckis plain data with every field public and noprivate state anywhere; introducing one private field breaks that shape for a struct
callers pattern-match on.
feat!:commit, once thecrate's published surface can be established.
4. What the refusal message may print
dropping userinfo, path, query and fragment.
https://ci:secret@nexus.internal/repo/npmrenders asnexus.internal..dependable.tomland
.npmrc,expand_envinterpolates${VAR}into.npmrccontent, and the refusalgoes to stderr, which CI captures as job output. A user may legitimately write
https://ci:${NPM_TOKEN}@nexus.internal/repo, and printing it would put a credential ina log. The repository is otherwise careful here.
motivated the routing key, and cannot distinguish two npm registries in one run, which
is the JSR case the issue cites.
rootunmodified, or drop it from the message and keep it in thedata only.
5. Where the label is rendered (taken inside the manifest, not in the record above)
UnreachableRegistry::label()and the privateauthority_oflive independable-fetch/src/check.rs, beside the type. The CLI callslabel().runner.rs— the credential reduction is the security-relevant part of this change, and a second consumer of
UnreachableRegistry(an IDE,the TUI) would have to reimplement it correctly to avoid printing a token. One
implementation, tested where the type lives, is the safer shape.
labelandauthority_ofintorunner.rsand reduce the publictype to its two fields.
6. Silent degradation to the bare ecosystem name
alone and say nothing about the host.
https://nexus.internal/repository/@scope/npm—an ordinary Nexus npm-proxy path — therefore reads
npm, indistinguishable from afetcher that names no root at all.
npm (registry root withheld)— it tells an attacker reading a public CI log that a private registry is configured,
which is itself a disclosure, and it tells the operator nothing they can act on.
@— every additional heuristic is another way to be wrong in the direction that prints a
secret. The failure mode must stay "say less".
Nonebranch oflabel().7. The redundant boolean's lifetime
registry_unreachableunderived-but-derived, without#[deprecated].It is derived at the single construction site so the two cannot disagree.
#[deprecated]now — the crate's published surface cannot beestablished from within the repository, so a deprecation warning would fire for
consumers who may never have seen the field, and the retype it points at is not
scheduled.
#[deprecated(note = "use unreachable_registries")]once thepublished surface is known.
8. Nested comma lists in the refusal
the vulnerability scan did not complete, the Go, JVM and npm registries did not answer and 2 dependencies could not be evaluated.answering one question, and splitting it would make the exit-2 reason arrive in two
places.
9. Label ordering is ASCII, so
JVMandPHPprecedenpmacross machines and runs.
another thing to keep in step with
display_name.Also settled inside the manifest
UnreachableRegistry::newis public.#[non_exhaustive]otherwise makes the typeunconstructible outside
dependable-fetch, and the CLI's own unit tests must build one.ScanIntegritylosesCopy(it keptClone, which is all the code used) because itsregistry_unreachablefield is now aVec. It is private to a binary-only crate, sothis is not a semver event.
UnreachableRegistry::new(ecosystem, None)if the routelookup ever misses. Every task contributes a route so it cannot, but a lost entry must
degrade to a less precise report, never to a certified run.
Not done
version. No
urldependency —authority_ofis hand-rolled over&strand testedagainst userinfo, ports, IPv6 literals, missing schemes and empty input.
--fail-on anyexemption,OSV and
vulnerability_scan_failedare all untouched.so nothing declines to answer. Pre-existing, documented on the field, deliberately not
changed here.
NpmFetcher::resolveroutes an@scopepackage to a per-scope registry thatregistry_root()does not name, so a scoped failure is reported under the fetcher'sdefault root. Documented as a limitation on the type; filed separately.
Follow-ups filed
registry_root()cannot name a per-scope npm registry, so a reported root canname a host that was never asked. Shares a root cause with fix(fetch): the disk cache key misses npm per-scope registries and JSR #126, which reports the same
defect as a disk-cache leak; fix(fetch): registry_root() cannot name a per-scope npm registry, so a reported root can name a host that was never asked #143 says how the two relate and which fix closes both.
Findings raised against this diff before it was reviewed by anyone else
The first commit was audited adversarially before the branch was finished. Four defects
were found and are fixed in
fix(check): stop a registry root's credentials reaching the refusal:authority_ofsplit the path off beforestripping userinfo, so a password containing
/survived as the "host":https://ci:AbC/dEf@nexus.internal/npmreduced toci:AbC, and the gate printed ausername and a token prefix on stderr. A standard-alphabet base64 token contains
/about 40% of the time at 32 characters, and such a root makes every request fail, so
the label prints on every run rather than rarely.
?and#took the same path. Itnow fails towards saying less: an
@surviving outside the authority candidate meansthe string cannot be split with confidence, so the root is dropped and the ecosystem
named alone.
\joins the separators, because WHATWG resolves it as/for specialschemes and reading it as authority named a host that was never contacted.
under a sort keyed on the raw root, but several roots reduce to one label and need not
be adjacent —
http://nexus.corp/a,http://other.host/x,https://nexus.corp/bsort in that order. Labels are now sorted before deduplication.
its own doc: two alternate-registry aliases naming one index URL reached a library
consumer as two identical entries. The CLI masked it.
authority-only label distinguishes and about a nesting
name_unansweredin factproduces. Both now say what is true.
Found and not fixed here — needs its own issue
FetchError::Http'sDisplayappends" for url ({url})", and aurl::Urlserialisesits
user:password@userinfo. So a run whose registry root carries credentials printsthem in the per-dependency error cell —
error: http error: error sending request for url (https://ci:secret@nexus.internal/express)— rendered bycrates/dependable/src/output/table.rsandoutput/github.rs. That is pre-existing anduntouched by this change, and both files are outside this change's manifest, so the
README wording here was narrowed to describe the refusal line only rather than the run.
This is a real credential leak into CI output and should be filed and fixed
separately.
Findings raised in a second review round, and fixed here
High —
authority_ofstill leaked a credential fragment. The check for an@surviving past the authority was a match guard on the
Nonearm of the userinfosplit, so it was consulted only when the authority candidate held no
@of its own. Aroot whose userinfo and whose path both contain one took the
Somearm instead, andrsplit_oncereturned the text between the embedded@and the first delimiter —credential material, and a host the run never contacted. Such a root makes every
request to it fail, so the refusal printed it on every gated run. Seven shapes leaked;
hoisting the check above the split closes all seven and moves no other row:
https://user:p@ss/word@nexus.internal/xsshttps://ci:AbC@dEf/ghi@nexus.internal/npmdEfhttps://ci:p@s?s@nexus.internal/xshttps://ci:p@s#s@nexus.internal/xshttps://ci:p@s\s@nexus.internal/xshttps://ci:tok@en/@nexus.internal/npmenhttps://user@host.example:pw/x@real.internal/yhost.example:pwConfirmed against a 40-input corpus covering every row this branch already asserts on:
exactly those seven rows change, and the other 33 are byte-identical. All seven are now
regression cases. Whenever
authority_ofreturnsSome, every@in the string liesinside the authority, so what survives is a host and port and never userinfo.
authority_of's doc comment and the README line are corrected to claim only that —the README now also states that the line degrades to the bare ecosystem name rather
than guessing (decision 6).
Low — the public
rootfield carried credentials with no warning.label()'s docexplains at length that a root must never be printed whole, but a consumer reading the
field it is handed got no signal. Decision 5 rests on the argument that a second
consumer must not have to reimplement the redaction; the field now says so, and names
label()as the only safe rendering.Low —
fetch_all's sort and dedup were untested at the production site.unreachable_registries_sort_by_ecosystem_then_rootre-applied both in its own bodyover a hand-built vector, and the one integration test reaching
fetch_allyielded asingle entry, so deleting both lines from
fetch_allleft the whole suite green — theCLI re-sorts downstream, and that half is covered. The library's published ordering,
which an embedding consumer reads straight off the value, was untested. A new
integration test has both of a
deno.json's registries decline and asserts thereturned order directly; both routes are
Ecosystem::Npm, so only the root can orderthem. Because the outcomes are gathered in a
HashMapwhose iteration order is seededper map, the check is repeated concurrently through fresh
Checkers, which costs oneround of retry backoff. Measured: 30/30 runs pass as written, 30/30 fail with the
sort deleted from
fetch_all. The unit test keeps its coverage of the key and thededup, with its doc comment corrected to claim only that.
Validation
cargo test -p dependable-fetch --libcargo test -p dependable-fetch --test checkercargo test -p dependable --test cli_gatecargo test -p dependable --bin dependablemise run testmise run fmt:checkmise run lintclippy --workspace --all-targets -D warnings, cleanconvco check origin/feat/111-advance-forced-versions..HEADNo failure was observed in this branch, so nothing needed classifying as caused,
pre-existing, flaky or unavailable.
The refusal was also read off the built binary rather than only off the tests, against
two loopback servers answering 500:
Base
Stacked on #122 (
feat/111-advance-forced-versions), the tip ofmaster ← #99 ← #106 ← #108 ← #116 ← #122 ← this.