Add waitable queues and the placement probe, and reshape the topology model - #56
Add waitable queues and the placement probe, and reshape the topology model#56MikeGrier wants to merge 358 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Confirmed correctness issues in new code paths (notably recv_timeout potentially waiting forever and test hook state leaking on panic) should be fixed before approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This pull request expands the workspace’s Windows observability and measurement tooling (new probes + CI jobs), introduces new crates (windows-waitable-queues, windows-placement-probe), and hardens topology trust semantics by adding an explicit provenance marker to windows-topology-sys.
Changes:
- Add
windows-waitable-queues(bounded, waitable HANDLE-backed queues) and wire it into workspace + release tracking. - Add
windows-placement-probeand expandwindows-platform-probes(new probe binaries + tests), then run probes/spikes in CI for ongoing fleet measurement. - Add
Provenancetowindows-topology-sysso serialized / hand-built topologies cannot silently pass as “measured”.
File summaries
| File | Description |
|---|---|
| tools/run-numa-spikes.ps1 | Builds/runs standalone NUMA “spike” instruments in a scratch crate and captures transcripts. |
| .github/workflows/ci.yml | Runs additional probes (topology, doorbell cost, request cost) and adds a “numa-spikes” observational job with artifact upload. |
| release-please-config.json | Adds windows-waitable-queues to release-please configuration. |
| .release-please-manifest.json | Adds crates/windows-waitable-queues version for release tracking. |
| Cargo.toml | Adds windows-placement-probe and windows-waitable-queues to workspace members. |
| Cargo.lock | Records new workspace crates and their dependency edges. |
| .gitignore | Ignores placement-probe-v*.json backup records to avoid accidental commits. |
| PLANS.md | Adds planning rows / trackers for the new crate and related checklists. |
| crates/windows-waitable-queues/PLANS.md | Adds per-component plans pointer back to the root checklist. |
| crates/windows-waitable-queues/Cargo.toml | Declares the new publishable windows-waitable-queues crate and its windows-sys feature set. |
| crates/windows-waitable-queues/src/race_hooks.rs | Adds deterministic race-window hooks for tests to drive correctness-critical interleavings. |
| crates/windows-waitable-queues/src/options.rs | Adds an Options builder for construction-time queue switches (disposal policy, high-water tracking). |
| crates/windows-waitable-queues/src/metrics.rs | Adds queue metrics (refusals + optional high-water). |
| crates/windows-waitable-queues/src/metrics/tests.rs | Tests metric arithmetic and concurrency behavior in isolation. |
| crates/windows-waitable-queues/src/disposal.rs | Adds teardown policy plumbing for handing undrained items to a caller-provided sink. |
| crates/windows-waitable-queues/src/disposal/tests.rs | Tests disposal/teardown behavior including panicking sinks. |
| crates/windows-waitable-queues/src/capacity.rs | Centralizes bounded-capacity validation shared across queue shapes. |
| crates/windows-waitable-queues/src/blocking.rs | Adds shared blocking recv / recv_timeout loop that parks on a Win32 event handle. |
| crates/windows-topology-sys/src/topology.rs | Adds Topology::provenance and stamps Measured only in discover(), with serde downgrade-on-load. |
| crates/windows-topology-sys/src/topology/tests.rs | Updates/extends tests to pin provenance downgrade semantics and safe defaults. |
| crates/windows-topology-sys/src/provenance.rs | Introduces Provenance type plus serde downgrade helper. |
| crates/windows-topology-sys/src/provenance/tests.rs | Tests provenance ordering, defaults, downgrade rules, and rendering. |
| crates/windows-topology-sys/src/lib.rs | Exposes Provenance from the crate. |
| crates/windows-topology-sys/DESIGN-NOTES.md | Documents the provenance decision (D-12) and its rationale/semantics. |
| crates/windows-ioring-sys/design-sessions/spikes/README.md | Documents why certain spikes are checked in “ready but unrun” and how to interpret/run them. |
| crates/windows-ioring-sys/CHECKLIST.md | Adds/updates queued repairs from the NUMA-sharding measurement (M20). |
| crates/windows-platform-probes/Cargo.toml | Adds new probe binaries and depends on shipping crates for measurement fidelity. |
| crates/windows-platform-probes/src/lib.rs | Exposes new probe modules. |
| crates/windows-platform-probes/src/tests.rs | Adds topology probe consistency tests against real host counters. |
| crates/windows-platform-probes/src/request_cost.rs | Adds request construction timing probe logic. |
| crates/windows-platform-probes/src/bin/topology.rs | New probe-topology binary emitting both human output and a single JSON line for mining. |
| crates/windows-platform-probes/src/bin/doorbell_cost.rs | New probe-doorbell-cost binary measuring doorbell relative costs. |
| crates/windows-platform-probes/src/bin/request_cost.rs | New probe-request-cost binary reporting request construction costs and ratios. |
| crates/windows-platform-probes/src/bin/queue_contention.rs | New probe-queue-contention binary exploring queue tail-claim contention scaling. |
| crates/windows-placement-probe/Cargo.toml | Introduces windows-placement-probe crate and build identity stamping (publish=false for now). |
| crates/windows-placement-probe/README.md | Documents purpose, privacy posture, usage, and provenance expectations for runners. |
| crates/windows-placement-probe/DESIGN-NOTES.md | Records publication/provenance constraints and why crates.io must not become the primary path yet. |
| crates/windows-placement-probe/build.rs | Stamps commit/dirty/source into the binary (CI vs local vs unknown). |
| crates/windows-placement-probe/src/lib.rs | Defines the crate’s module surface and documentation for the measurement tool. |
| crates/windows-placement-probe/src/build_identity.rs | Implements build identity model + trust ordering and “official build” predicate. |
| crates/windows-placement-probe/src/build_identity/tests.rs | Tests build identity trust semantics and build-script stamping shape. |
| crates/windows-placement-probe/src/submission.rs | Renders the “paste-able” submission payload and includes a truncation-detecting checksum. |
| crates/windows-placement-probe/src/peer_index_cache/tests.rs | Tests correctness/honesty of memory placement reporting (esp. on single-node hosts). |
| crates/windows-placement-probe/src/machine/tests.rs | Tests registry-based machine description collection policy and safety invariants. |
| crates/windows-placement-probe/schema/v1.txt | Schema golden for v1 submission records. |
| crates/windows-placement-probe/schema/v2.txt | Schema golden for v2 submission records (adds memory_node). |
| CHECKLIST.md | Records completed tooling item(s) related to sabotage harness promotion. |
| COMPLETED-PLANS.md | Archives the completed topology provenance checklist entry. |
| COMPLETED-CHECKLIST.md | Archives the completed topology provenance checklist in detail. |
Review details
- Files reviewed: 81/95 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
🟡 Changes recommended
There are a few correctness/documentation issues in the new queue crate (broken rustdoc links and a test hook that can leak state on panic) that should be fixed before merging.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
crates/windows-waitable-queues/src/race_hooks.rs:78
- Hook::with does not clear/restore the thread-local hook if
bodypanics (e.g., a failing assertion inside the test closure). That can leak the hook into later tests on the same thread and cause cascading/irreproducible failures. Use a drop guard to restore the previous hook even during unwinding.
- Files reviewed: 81/95 changed files
- Comments generated: 2
- Review effort level: Lite
There was a problem hiding this comment.
🔵 Needs a closer look
There are a few correctness/behavior issues in the new queue code/docs (panic cleanup in race_hooks::Hook::with, potential tight-loop at sub-millisecond timeouts in recv_timeout, and Metrics rustdoc inconsistencies) that should be addressed before approval.
Review details
Suppressed comments (4)
Previously missed (2) — in code that hasn't changed since the last review.
crates/windows-waitable-queues/src/metrics.rs:13
- The module docs say
Metricsholds three counters including "Doorbell rings", but this type currently only storesrefusedandhigh_water(doorbell rings are tracked indoorbell::Doorbell). This makes the rustdoc misleading for readers looking for where to get the ring count.
This issue also appears on line 20 of the same file.
crates/windows-waitable-queues/src/blocking.rs:140
- When
remainingis non-zero but < 1ms,remaining.as_millis()truncates to 0, so this computesmillis = 0and can spin in a tight loop until the deadline. Clamping to at least 1ms avoids the busy-wait at the end of the timeout budget.
crates/windows-waitable-queues/src/metrics.rs:20
- The docs below still describe a 3-counter design (including a "Rings" counter) and explain why "two of the three are free". Since ring counting lives in
doorbell::Doorbellrather thanMetrics, this section should be updated to describe only the counters actually accumulated here.
crates/windows-waitable-queues/src/race_hooks.rs:77 Hook::withclaims to install a hook only for the duration ofbody, but ifbodypanics (or unwinds via?through a panic), the hook is never cleared because the cleanup happens afterbody()returns. This can leak the hook into subsequent operations on the same thread and make later tests flaky or misleading.
- Files reviewed: 81/95 changed files
- Comments generated: 0 new
- Review effort level: Lite
CI's rustdoc job has failed on all three runs of pull request #56. `main` is green, so the branch broke it -- most likely the `mpsc` -> `slotwise_mpsc` rename, which moved every item these links named. Twelve sites across three crates. The job denies `broken_intra_doc_links`, `private_intra_doc_links` and `invalid_rust_codeblocks`, so eleven of these were errors rather than warnings. `MIN_CAPACITY` did not exist anywhere in the workspace: the public docs for `slotwise_mpsc::bounded` referred readers to a constant that was never written. The rule it was supposed to explain lives in the private `BOUNDS`, so the sentence now states the rule itself -- a capacity of one cannot distinguish "published" from "free" under the sequence protocol. The links into private items are delinked rather than repointed, because a public page cannot link to a page rustdoc does not generate. Where the private name is still useful to a maintainer it stays as inline code. Also corrects SH-2.4, which described these as pre-existing warnings to clear before publication. They were neither pre-existing nor warnings, and nothing was waiting on the release -- the release was waiting on them. Completed item: SH-2.4: Clear the eight rustdoc warnings in windows-waitable-queues before it is published Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
There are correctness issues in new test infrastructure and CI tooling (panic-unsafe hook scoping and missing runtime failure handling in the NUMA spike runner) that should be fixed before merging.
Review details
Suppressed comments (2)
Previously missed (1) — in code that hasn't changed since the last review.
crates/windows-waitable-queues/src/options.rs:49
- The rustdoc example comment says “The peak, not the depth right now.”, but the next line asserts
rx.len() == 1(current depth). This reads as a contradiction for users learning the API. Adjust the comment so it matches what’s being asserted.
crates/windows-waitable-queues/src/race_hooks.rs:78
Hook::withdoes not clear the thread-local hook ifbodypanics, which can leak the hook into later code on the same thread (e.g., if a test usescatch_unwind/ asserts a panic) and make failures non-deterministic. Make the hook clearing panic-safe via an RAII guard that resets the slot inDrop.
- Files reviewed: 82/96 changed files
- Comments generated: 0 new
- Review effort level: Lite
Three findings from the Copilot review on pull request #56. **`recv_timeout` could become unbounded.** The blocking loop clamped an oversized remaining duration to `u32::MAX`, which is the same value as `INFINITE` -- imported three lines above the clamp in the same file. A timeout longer than about 49.7 days therefore waited forever instead of timing out, and the loop that was supposed to re-check the deadline never regained control to do it. The existing comment reasoned carefully about clamping versus truncating and was right about everything except the one value it chose. The reviewer's suggested fix, clamping to `u32::MAX - 1`, is incomplete, and a boundary test written for it is what showed that: a duration of exactly `u32::MAX` milliseconds *converts* successfully, so the fallback never fires and `INFINITE` is returned by the conversion rather than by the clamp. Both guards are needed, and they cover different inputs. The clamp is now derived from `INFINITE` rather than written as a number, and the arithmetic is extracted so it can be tested -- the failing case takes 49 days to observe through the public API and so could never be a test of the loop. **A hook survived a panic.** `Hook::with` removed the installed hook with a statement after the body, which an unwind skips. A test that installs a hook and then fails an assertion -- the ordinary way for a test to fail -- left it installed to fire inside whatever ran next on that thread, in the facility this crate's central correctness argument rests on. Now removed by a guard, using `try_with` so teardown cannot replace an unwind already in progress. **Eighteen broken documentation links, from a report of two.** `../../` from `src/*.rs` resolves to `crates/`, which holds no `DESIGN-NOTES.md`; the crate's own notes are one level up. Sweeping the workspace found sixteen more of the same, in `spsc`, `traits`, `reserving_mpsc` and `slotwise_mpsc`. The sweep also caught its own repair damaging a correct link: `windows-file-watcher/src/contract.rs` used `../../../DESIGN-NOTES.md` to reach the *workspace* notes deliberately, and a blanket prefix replacement shortened it. Reverted, and every relative markdown link in every Rust source is now checked to resolve -- 36 of them, none broken. The review's fourth point, that `race_hooks` puts a thread-local lookup on production hot paths, does not hold: `mod race_hooks` and all four call sites are already `#[cfg(test)]`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
There are a few correctness/docs issues (notably build-stamp invalidation and misleading docs) that should be fixed before merging.
Review details
Suppressed comments (3)
Previously missed (3) — in code that hasn't changed since the last review.
crates/windows-waitable-queues/src/options.rs:49
- The doc example comment says
rx.len()is "The peak", butlen()reports the current depth (peak ishigh_water()). This is misleading for API consumers reading the docs.
crates/windows-placement-probe/build.rs:33 - Watching only
.git/HEADis usually insufficient to keep the stamped commit fresh: on most repos HEAD contains a stableref: refs/heads/<branch>line and does not change when new commits are made. This can leavePLACEMENT_PROBE_COMMIT_OUTstale after committing without editing files.
crates/windows-platform-probes/src/request_cost.rs:157 - These Win32 parameters are currently magic numbers, which makes it hard to tell what kind of open is being timed (and easy to accidentally change semantics later). Prefer the named
windows-sysconstants so the request is self-documenting.
- Files reviewed: 84/98 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🔵 Needs a closer look
Release tracking for windows-waitable-queues is added, but the publish automation/tag triggers are not updated to actually publish it, and there are also concrete doc/identity-stamping issues identified in the new code.
Review details
Suppressed comments (3)
Previously missed (2) — in code that hasn't changed since the last review.
crates/windows-waitable-queues/src/metrics.rs:14
- The module docs say this type tracks three counters (refusals, doorbell rings, peak depth), but
Metricsonly storesrefusedandhigh_water. Either the docs are stale or a counter is missing; as written it misleads readers about where doorbell ring counts live.
This issue also appears on line 20 of the same file.
crates/windows-placement-probe/build.rs:33
build.rsprintsrerun-if-*directives, which disables Cargo's default "rerun the build script when any package file changes" behavior. As a result, local edits tosrc/(which make the tree dirty) will not cause the stamp to be recomputed, sodirty/commitcan go stale across rebuilds.
crates/windows-waitable-queues/src/metrics.rs:24
- Follow-on to the docs above: the rest of this section describes rings as a counter maintained here ("two of the three are free..."). With doorbell ring counts not stored in
Metrics, this section should be updated so it only explains the two countersMetricsactually owns.
- Files reviewed: 84/98 changed files
- Comments generated: 0 new
- Review effort level: Lite
Five findings from the Copilot reviews on pull request #56. These arrived only in review *bodies* rather than as inline threads, so the thread query used last time could not see them. **The build stamp did not update on commit.** `build.rs` watched `../../.git/HEAD`, and on a branch that file holds `ref: refs/heads/<branch>` -- a line that does not change when you commit. Since a `rerun-if-changed` directive replaces cargo's default of watching the whole package, the script never re-ran and the binary kept whatever commit it was first built with. Measured rather than reasoned about: `.git/HEAD` had not been touched in 21 hours while fourteen commits landed, and a freshly rebuilt binary reported `b6f23ec9f3bc` against a `HEAD` of `0b96b14746bb` -- six behind. CI hides this completely, because there the commit arrives through `PLACEMENT_PROBE_COMMIT`, so the stamp was wrong only on local builds, which are exactly the ones whose commit is their only traceability. The whole submission record rests on this field. Now resolves what `HEAD` points at and watches that too, handling a detached `HEAD` (which changes on its own), a packed ref (where `packed-refs` changes instead), and a `.git` file redirecting to a worktree. Paths are emitted only when they exist, since a missing path makes cargo re-run the script on every build. **`recv_timeout` busy-waited below a millisecond.** A remainder under 1 ms truncates to zero, and a zero wait returns at once, so the loop re-armed and re-waited without sleeping. Arming clears the doorbell, which is a `ResetEvent` syscall, so this was a syscall storm rather than merely a hot loop. Clamped to a millisecond: overshooting a blocking deadline by less than a timer tick is the right trade, and sub-millisecond precision is not available from a blocking wait at any price. Also: the `Options` doctest labelled `rx.len()` as "the peak" when it is the current depth; the `metrics` module described a ring counter that lives on `Doorbell` without saying so; and `request_cost` passed `0x8000_0000`, `1` and `3` to an open request instead of `GENERIC_READ`, `FILE_SHARE_READ` and `OPEN_EXISTING`, which this repository's conventions forbid. The three constants were checked against `windows-sys` rather than assumed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Working through the five findings raised in the last four review bodies. These arrived as suppressed comments in the review summaries rather than as inline threads, so they were invisible to a All five were checked against the code before acting, and all five were real. Fixed in dd2e9c5 and 81cff89. 1. Build stamp invalidation (
|
There was a problem hiding this comment.
🔵 Needs a closer look
tools/run-numa-spikes.ps1 currently builds scratch spike crates without a lockfile and does not surface non-zero cargo run exits, which can cause silent probe failures and CI flakiness over time.
Review details
- Files reviewed: 83/98 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🟡 Changes recommended
Processor-group handling, provenance stamping, probe reporting, and release plumbing contain correctness and reliability gaps.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 83/98 changed files
- Comments generated: 18
- Review effort level: Balanced
There was a problem hiding this comment.
🔵 Needs a closer look
The long-path probe leaks process state and temporary trees, while deserialization preserves topology anomalies contrary to the restored-data contract.
Review details
- Files reviewed: 91/247 changed files
- Comments generated: 0 new
- Review effort level: Balanced
… from a document Two findings from Copilot review 5118406119, a headline-only round. Both real. The long-path probe leaked process state and a temporary tree. `measure` called SetCurrentDirectoryW and never moved back, and built a 40-deep tree under %TEMP% that nothing removed. Neither is excused by the probe binaries exiting straight afterwards: this is a library function, so a test or any other caller keeps running in a process whose current directory now points into a temp directory. Seven stale trees were sitting on this machine, which is how the leak was confirmed rather than argued. The tree is the sharper half. It is deliberately longer than MAX_PATH, which is exactly what stops Explorer and `del` from removing it -- litter from a probe about long paths is litter that is awkward to clear by hand. Both are now released by an `Apparatus` guard, so the early returns on every apparatus failure clean up too. Restoring the directory before removing the tree is load-bearing rather than tidy: a process's current directory holds a handle on it, so removal while parked inside silently fails. Sabotage confirmed exactly that -- deleting only the restore also broke the tree test. Writing the tests found a third thing: `measure` cannot be called concurrently. It borrows the current directory, which is one per process, so a unique root per call would not fix it. The three tests collided until they took a lock, and `measure` now documents the constraint instead of leaving the next caller to discover it. Deserialization asserted enumeration anomalies into a restored topology. `enumeration_anomalies` was `serde(default)` while its own first paragraph has always said the list is "empty for a hand-built or deserialized topology, which asked nothing" -- a contract contradicted by its own attribute. An anomaly is a fact about an enumeration and a deserialized topology performed none, which is the same reasoning that already made `coherence` skip_deserializing. It now does too, and is still written out, so a dump still carries the diagnosis. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| observation.apparatus_error = Some(error); | ||
| return observation; | ||
| } | ||
| let current_dir_len = root.as_os_str().len(); |
There was a problem hiding this comment.
Fixed in 231750b. You were right, and the workspace already had the answer: wtf-string exists for exactly this, so resolved_len is now counted with Wtf16String::from_os_str(...).len() -- the string held in the encoding Windows uses, so its len is the number under test rather than a conversion of one.
The relative part is built from ASCII constants and so cannot differ today; it is measured the same way anyway, because a unit that is only correct while the input happens to be ASCII is one waiting to be wrong. The field now documents the unit, and a test pins that the two counts agree for ASCII and diverge for U+00E9 / U+4E2D -- the direction that matters, since bytes over-count and would report a refusal as expected when it was not.
| let all = [ | ||
| Provenance::Synthetic, | ||
| Provenance::Restored, | ||
| Provenance::Measured, | ||
| ]; |
There was a problem hiding this comment.
Fixed in 231750b. The comment promised something the code could not keep -- an array is not exhaustive over an enum -- so the list now comes from a match, and the compiler refuses to build until a new variant is added to it.
Verified by adding a fourth Provenance variant: the build fails at this test's own line with non-exhaustive patterns: Provenance::Inferred not covered, which is the guarantee the comment claimed.
…enance sweep exhaustive Three findings from the unresolved PR #56 review threads. The long-path probe measured against MAX_PATH in the wrong unit. `OsStr::len` counts Rust's platform encoding, which is WTF-8; MAX_PATH counts UTF-16 code units. The two agree for ASCII and diverge for anything else, so a %TEMP% with a non-ASCII character made `resolved_len` too large and could report an attempt on the wrong side of the ceiling -- in a probe whose entire output is which side of that ceiling a path landed on. Now counted with `wtf_string::Wtf16String`, which is this workspace's own answer to exactly this question: it holds the string in the encoding Windows uses, so its `len` is the number under test rather than a conversion of one. The relative part is built from ASCII constants and so cannot currently differ, and is measured the same way regardless -- a unit that is only correct while the input happens to be ASCII is one waiting to be wrong. The Provenance pairwise test claimed to be exhaustive and was not. Its comment promised that "a variant added later cannot quietly acquire an upgrade path" while the list was a hand-written array that a new variant would leave untouched. It now comes from a `match`, so the compiler refuses to build until the variant is added -- verified by adding one and confirming the test's own line fails to compile. The topology crate's PLANS.md still advertised M6 as in progress with every M6 item checked, directing a reader at finished work. M6 is archived to COMPLETED-CHECKLIST.md and recorded in COMPLETED-PLANS.md. The `Deferred, and why` section stays in CHECKLIST.md: it is live context about two things left out on purpose, not completed work, and it rode along in the first move. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
SH-4.13 groups three unresolved threads with one root: ProcessorSet cannot represent every u8 processor id while Processor::id is public and constructible, so a deserialized or hand-built topology panics in machine_processors() and in granularity's insert. Per-site validation would close the reported path and leave the next open, so the decision is about ProcessorSet's representable range. SH-4.14 records that probe-long-path-aware builds without its manifest on windows-gnu, making the aware/unaware pair measure one configuration while claiming two. Gated on CI building a GNU target, since a fix would otherwise ship unexercised. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
The topology checklist retains prohibited non-action history, and the new architecture-dependent representability API lacks direct boundary coverage.
Review details
- Files reviewed: 91/248 changed files
- Comments generated: 0 new
- Review effort level: Balanced
There was a problem hiding this comment.
🔵 Needs a closer look
Release planning contradicts the permanent non-publication decision, and an active checklist contains non-actionable historical rationale.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
crates/windows-placement-probe/Cargo.toml:28
- The permanent
publish = falsedecision has not been propagated to the active workspace plan.PLANS.md:25still calls this a publishable tool and says M5+ will publish it to crates.io, whileCHECKLIST-placement-tool.md:460-499records the reversal. Update the plan row so future work does not act on the superseded distribution contract.
- Files reviewed: 91/248 changed files
- Comments generated: 0 new
- Review effort level: Balanced
…mespace-ops # Conflicts: # crates/windows-file-watcher/COMPLETED-CHECKLIST.md # crates/windows-file-watcher/DESIGN-NOTES.md # crates/windows-file-watcher/src/directory/tests.rs # crates/windows-file-watcher/src/queue.rs # crates/windows-file-watcher/src/queue/tests.rs # crates/windows-file-watcher/tests/reopen_by_id_cannot_be_watched.rs
There was a problem hiding this comment.
🟡 Changes recommended
Sensitive partial probe records remain trackable, allocator-dependent tests assert behavior Rust does not guarantee, and documentation still contradicts implemented contracts.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 84/228 changed files
- Comments generated: 3
- Review effort level: Balanced
| assert!( | ||
| fitted.capacity() < fitted_before, | ||
| "shrink_to_fit must release excess capacity on the Windows allocator" | ||
| ); |
| ## Deferred, and why | ||
|
|
||
| Two things were deliberately left out of the reshape rather than forgotten: |
| //! This is the part of the crate its name refers to. A queue shape owns a | ||
| //! [`Doorbell`] and keeps it in agreement with its own emptiness; a client that | ||
| //! wants to park on the queue *and* on an I/O completion *and* on a shutdown | ||
| //! event in one wait borrows the handle and hands it to | ||
| //! `WaitForMultipleObjects` alongside the others. | ||
| //! | ||
| //! # Manual-reset, and level-triggered | ||
| //! | ||
| //! The event is manual-reset, so it means "there is something to take" rather | ||
| //! than "something arrived". That is the difference between a state and an | ||
| //! edge, and only the state composes: `WaitForMultipleObjects` may report any | ||
| //! one of several signalled handles, so a waiter routinely learns about one | ||
| //! ready source while ignoring another. An auto-reset event consumed by that | ||
| //! wait would lose the second source's only edge. A level survives being | ||
| //! ignored, and will still be there on the next pass. |
…nt and width Adds three duplicated implementations of `reserving_mpsc`'s claim protocol -- 32/32 and 16/48 over `AtomicU64`, 64/64 over `AtomicU128` -- and measures them through `probe-queue-contention` in both regimes, so `D-37`'s shipping decision has a number it currently lacks. Built as a duplicated path in this crate rather than in `windows-waitable-queues`: the measurement adds no third-party dependency to a publishable crate and cannot disturb the branch being peeled off PR #56. `CW-1.6` owns merge-or-delete. Re-apportioning the bits is free (16/48 tracks 32/32 within noise in both regimes) and moves the SH-14.1 recurrence from 2^32 to 2^48. Widening the word costs 2-3x isolated, growing with contention, but only 5-12% drained -- and the drained figure understates it, because a slower producer earns fewer refusals and refusal retries are inside the timed region. The control caught a defect in the first run: the duplicates had not padded `head` and the claim word onto separate cache lines, reporting 3.7x against the shipping shape on a different scaling curve entirely. Completed item: CW-1.1: Add portable-atomic and record whether AtomicU128 is lock-free on this target Completed item: CW-1.2: Implement the three claim-word layouts as self-contained u64-item queues Completed item: CW-1.3: Wire the three layouts into probe-queue-contention Completed item: CW-1.4: Run the probe and capture the report Completed item: CW-1.5: Record the measurement in DESIGN-NOTES.md Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… splits The reservation half holds 2^32 where hundreds would do, and trading it away is what buys position bits: 2^12 reservations leaves over a year before recurrence, 2^8 leaves twenty years, against today's 37 seconds. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ed options Records the engineer's direction that options are right if quality is maintained and ramifications are available, and binds both halves: each layout states its own reservation ceiling, capacity ceiling, and time-to-recurrence where it is named; quality is per-layout; the default is not weakened. Adds CW-2.4 for the documentation and CW-2.5 for the SH-14.1 disclosure sweep, since every existing statement of that hazard is scoped to a 32-bit position. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ation, not the layouts The item read as though it might decide which layouts to offer. It does not: multiple layouts ship as options per M2. CW-1.6 is only about the private copy of the protocol in this crate, which M2 makes obsolete because the shipping type can then be instantiated at any layout. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…easurement falsified D-36 deferred fixing SH-14.1 because the fix was believed to be the D-35 claim-protocol replacement, gated on an open question. Re-apportionment is a second fix neither D-36 nor D-37 considered, measured free by CW-1.4, moving the recurrence from ~37 seconds to ~20 years. M2 was also parked on not disturbing a branch under review; that branch has no PR open and the u64 layouts need no new dependency. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ied apart Decoupling the reservation ceiling is numerically invisible at 32/32: MAX_RESERVED is 2^32-1 against a 2^31 capacity ceiling, so the cap can never bind and no test can reach it. It becomes observable only once a layout narrows the reservation half, so landing them separately would have committed a branch nothing could exercise. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Behaviour-preserving groundwork for CW-2.1's configurable apportionment: the position, head, and per-slot sequence become 64-bit, so a position wider than 32 bits can be represented at all. The split stays 32/32, so nothing observable changes and all 310 tests pass unmodified in substance. The wrapping arithmetic is centralised in advance() and distance(). A position is now carried in a u64 but is only POSITION_BITS wide, so it wraps where the packing says rather than where the type does -- and an omitted mask is not a compile error, it is a position that escapes its half of the claim word. The Drop loop was the sharpest case: unmasked it would have walked past the wrap and never reached its terminating position. The packing tests were rewritten against POSITION_MASK and MAX_RESERVED rather than u32's extremes, which would have silently stopped testing the edges once the apportionment changed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…electable Offers `Balanced` (32/32, the default), `Enduring` (16/48), and `Perpetual` (8/56) as compile-time choices, and decouples the reservation ceiling from the capacity so that an asymmetric division is usable at all. The claim word packs a reservation count and a position into one `u64` because one compare-and-swap must update both. The 32/32 split was not forced by the platform: it followed from requiring the count's half to hold the entire capacity, since every slot could be reserved at once. Capping outstanding reservations at the layout's own ceiling instead leaves the capacity bounded only by the ring, which is what lets the position take 48 or 56 bits. That is the breaking part, and it is a changed promise rather than a broken one: `reserve` now returns `None` once `L::MAX_RESERVED` reservations are outstanding, however empty the queue is. Under the default layout the ceiling is 2^32 against a 2^31 capacity, so it can never bind and nothing observable changes -- which is also why this could not be verified without the layouts, and why the two checklist items were merged during execution. What it buys, measured by probe-queue-contention: a deeper position costs nothing outside noise, because all three issue the same `lock cmpxchg` on the same `u64` and differ only in shift and mask constants. The recurrence behind SH-14.1 moves from 2^32 pushes to 2^48 or 2^56 -- roughly 37 seconds, 28 days, and 20 years at the crate's disclosed sustained rate. `ClaimLayout::VALID` is forced by `build` rather than left to be evaluated, because an associated constant in a generic context is only checked where it is used. Verified by sabotage: a layout with 31 position bits fails the build naming the assertion, and a first attempt that appeared to pass turned out not to have applied the edit at all. Completed item: CW-2.1: Introduce the layout as a compile-time parameter, widen the position to 64 bits, and decouple the reservation ceiling from the capacity Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e as a dependency question core::arch::x86_64::cmpxchg16b is stable on the pinned toolchain, so a 64/64 layout needs no third-party crate and D-7's and D-37's dependency cost does not apply. The real costs are hand-written unsafe, x86-64 only, and a target-feature decision -- against a 2-3x measured cost on the claim and Perpetual already reaching ~20 years on a plain AtomicU64. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… layout The claim word's integer type becomes part of the layout, so `reserving_mpsc` can pack into a `u128` where a caller asks for it. `Wide` divides that word 64 / 64: the position needs 2^64 pushes to recur, which is a guarantee rather than an argument about deployment lifetimes. **Off by default, and that is the point.** Rust's standard library has no 128-bit atomic, so this is the only thing in the crate that costs a third-party dependency. Without the feature the dependency tree is `windows-sys` alone and every layout uses `AtomicU64`; with it, `portable-atomic` appears. Verified with `cargo tree` in both configurations rather than assumed from the manifest. `default-features = false` on the dependency is load-bearing: with its defaults, `portable-atomic` silently substitutes a global lock where the native instruction is unavailable, which would put a mutex in the claim path while still compiling. The word type is abstracted behind `ClaimWord` rather than by widening everything to 128 bits, so a `u64` layout still issues `u64` instructions exactly as before -- the arithmetic is monomorphised to each width. Positions stay in a `u64` throughout, since no layout gives them more than 64 bits, and the reservation ceiling stays capped at `u32::MAX` because the count is reported to callers as a `u32`. Two const assertions moved with it: the position must be narrower than its own word rather than narrower than 64, and it may not exceed the `u64` it is carried in. 314 tests pass by default, 319 with all features, including five that exercise `Wide` through the whole protocol rather than only its constants -- the packing at a maximal position, where a carry into the count would show, and delivery of both a push and a reservation. Completed item: CW-2.3: Decide whether a 128-bit claim word ships at all -- decided yes, behind an opt-in feature, so callers who do not want the dependency do not carry it Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…W-1.6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…perseding D-36 D-36 decided 0.1.0 would ship SH-14.1 disclosed rather than fixed, because the only known fix was the D-35 claim-protocol replacement, gated on an open question -- so disclosing beat delaying. That premise is false. Re-apportioning the claim word is a second fix neither D-36 nor D-37 considered, and it costs nothing measurable, so the hazard is now a number the caller sets rather than one the crate imposes. Recorded as D-41, which supersedes D-36. D-37 is partly superseded too: the wide word ships as a layout behind the `dwcas` feature rather than as a separate `reserving_mpsc_wide` shape gated by target, though its reasoning about `portable-atomic`'s default features still stands. Swept `2^32` / `32-bit position` / `sound below the wrap` across src/, tests and *.md for the crate: 21 sites, 9 rewritten, 12 already correctly scoped or out of scope. The out-of-scope ones are `slotwise_mpsc`'s note about `usize` on 32-bit targets, which is a different subject, and the table rows and prose that now say `Balanced` explicitly. `permit_mpsc`'s rationale needed care rather than a scoping edit: a deeper position moves the recurrence out of reach without removing the decision/operation separation that causes it, which is why that shape remains interesting. The three new examples are doctests, so a renamed layout breaks the build rather than leaving the documentation teaching a name that no longer exists -- doctests go from 9 to 12. Two defects the CI rustdoc flags caught, both invisible to `cargo test`: a public doc linking to the private `build`, and a link to `Wide` that resolves only when `dwcas` is on. Docs now build clean under all three feature configurations. Completed item: CW-2.4: Document the layouts as a choice, in the crate documentation and the README, with the rollover table Completed item: CW-2.5: Reopen D-36 with the measurement in hand, then sweep every statement of the hazard Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… duplicate `windows-waitable-queues` takes the claim layout as a parameter now, so the probe instantiates the real type at each of `Balanced`, `Enduring`, `Perpetual` and `Wide` rather than carrying its own copy of the protocol. The duplicate existed so the layouts could be compared before the shipping crate had them; that reason is spent, and 501 lines go with it. The direct `portable-atomic` dependency goes too. The probe now reaches a 128-bit word through the queue crate's `dwcas` feature, so there is one place that decides how that atomic is configured instead of two that could disagree. The two timing functions are generic over the layout where their `permit_mpsc` neighbour is deliberately duplicated, and the difference is worth stating: that twin compares two *different types*, which a generic could only unify behind a trait, putting an indirection inside the timed region. These are the *same type* at different layout parameters, so this monomorphises to what a hand-written copy would produce. **Re-measuring on the real type corrected the result.** The duplicate reported the 128-bit exchange at 2.37x and 2.99x the default at sixteen and thirty-two producers; the shipping type reports 3.83x and 3.99x. The stand-in was understating the cost of the very layout it was built to evaluate, by the widest margin exactly where the decision is most sensitive. The apportionment finding survived unchanged -- both `u64` re-apportionments still track the default within noise, so `Perpetual`'s twenty years of headroom really is free. The 1.26x offset the duplicate carried is gone: running the same configuration twice through the shipping type agrees within noise, 50.3 ns against 52.1 ns at thirty-two producers, because both rows are now the same code. Completed item: CW-1.6: Delete the duplicated implementation in claim_layout.rs, keeping only what CW-2.3 leaves no other way to measure -- which, since Wide ships, is nothing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
All items complete, so the named-feature checklist moves to COMPLETED-CHECKLIST.md and its PLANS.md row moves to COMPLETED-PLANS.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…EADMEs The queues README gained a Cargo features section -- dwcas was user-facing and undocumented, and it is the only thing in the crate that costs a third-party dependency, so a reader deciding whether to enable it needs the trade stated. The status summary now says the claim word's apportionment is a caller's choice rather than describing reserving_mpsc as though it had one fixed layout. The probes README's 'what is measured' section gained the queue-contention entry, including that the probe instantiates the shipping type at each layout rather than a stand-in, and why. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
The placement adapter currently hides retained topology grouping and efficiency-class disagreements, producing potentially inaccurate classifications.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 84/231 changed files
- Comments generated: 3
- Review effort level: Balanced
| let core_id = core | ||
| .label_from(Source::RelationshipWalk) | ||
| .unwrap_or(index as u32); | ||
| for id in core.processors.iter() { | ||
| core_of.insert(id, core_id); |
| let DomainKind::Core { | ||
| efficiency_class, .. | ||
| } = core.kind | ||
| else { | ||
| continue; | ||
| }; | ||
| for id in core.processors.iter() { | ||
| class_of.insert(id, efficiency_class); | ||
| } |
| [dependencies.portable-atomic] | ||
| version = "1.15.0" | ||
| default-features = false |
Adds two Windows-only crates, replaces
windows-topology-sys' model, and puts the release machinery in place to ship them.windows-waitable-queueswindows-topology-syswindows-placement-probepublish = false-- shipped as a prebuilt binarytopology-plannerwindows-waitable-queuesBounded producer/consumer queues whose readiness is a waitable Windows
HANDLE.That is the crate's reason to exist.
crossbeam-channelblocks on its own internal primitive and exposes noHANDLE, and itsSelectaccepts only channel operations, so a thread that must wake on "a message arrived or my I/O completed or shutdown was signalled" cannot express that wait. A queue whose readiness is aHANDLEcomposes withWaitForMultipleObjects,MsgWaitForMultipleObjects, thread-pool waits, and alertable waits.Every item is behind
cfg(windows); the crate builds to an empty shell elsewhere.Three shapes, none canonical -- there is no type named
Queue, so a caller names the shape it wants:spsc!Cloneproducer)slotwise_mpscCloneproducer)reserving_mpscreserving_mpscadds reservations:reserve()claims capacity up front and returns aReservationredeemed bysendor released on drop. A granted reservation is always redeemable, so a caller can establish it has room before doing work whose result must not be dropped. The queue stays connected while a reservation is outstanding.The doorbell is a lazily created manual-reset event per queue. A consumer that only polls never allocates a kernel object; one that waits calls
doorbell()for a borrowedHANDLE(ordoorbell_owned()for anOwnedHandle).arm()clears the doorbell and then asks whether an item is takeable, so a push landing in that window cannot have its signal erased.windows-topology-sysThe published 0.1.0 modelled a ladder of levels with optional rungs. This replaces it with a model of observed connectivity, which is the shape of question consumers actually ask.
Topologyis nowMachineMemoryTopology-- the old name claimed more than the type delivers.ObservationandSourcerecord which observer saw a relation, so CPU Sets and the relationship walk sit side by side instead of merging into one lossy answer.Observed<T>replacesOption<T>where absence was ambiguous.Known/Absent/NotObservedare three distinct facts, carried by serde as three distinct encodings (number, explicitnull, omission).Granularityorders what a set of processors shares by observed set inclusion, withminimal_sharedas the meet and the whole machine as the top. "How close are these two processors" is therefore answered from observed membership across every domain kind at any depth, rather than from a firmware-asserted cachelevel.proximityreads an inclusion-ordered partitioning rather than re-implementing the partitioning rule.distancesandDistancesare deleted. The crate does not go below the Win32 topology APIs, so a fact Win32 does not report is not a fact the crate has. The field was never populated.Domain::idis removed in favour of per-observation labels: one id cannot name a domain two observers labelled differently.Coherenceon the topology. Windows offers no transactional way to read the relationship walk and the CPU-set enumeration together, so a processor hot-added between the calls appears in one and not the other; a second pass settles that, and what survives the bound is reported as a genuine disagreement rather than hidden. Grouping disagreements are deliberately not retried -- they are persistent in the field, so re-reading cannot settle them.windows-placement-probeA measurement tool that reports where the OS actually places threads and memory, released as a prebuilt binary rather than a crate. Date-based version (
2026.902.0) because its output is dated evidence, not an API contract.It places every online processor, refuses to invent NUMA membership, reports a cache relationship the topology never established as unknown rather than as same-or-cross, and publishes its record backup by rename so a failed write cannot leave a truncated file under a complete record's name.
topology-plannerPlanning and design documents for a future planner consuming an abstracted topology description. No code ships: the topology crate is justified as the refined view of what the platform publishes, with an adapter absorbing the planner's needs.
Breaking changes for consumers
windows-topology-sys--Topologyrenamed;Domain::id,distancesandDistancesremoved;Option<T>becomesObserved<T>onMemory::memory_bytes;usable()removed (it consulted a CPU-set byte this build never populates and returned false for every processor, and it encoded policy in a facts crate).windows-file-watcher-- reopen-by-id removed.windows-waitable-queues-- first release, so nothing to migrate.Versions this merge proposes
windows-topology-syswindows-waitable-queueswindows-file-watcherwindows-ioring-syswindows-thread-ambient-syswindows-file-watcher-example-test-harnessThe other six released crates get no release: they carry only
test:,docs:andrefactor:commits since their tags.windows-ioring-sysis pinned to 0.2.1 because no public item in it changed on this branch -- release-please attributes commits by the paths they touch, and two topology-scoped breaking commits reached it through an example and one doc-comment heading. A 0.3.0 would send consumers looking for a migration that does not exist.tools/check-commit-scope.ps1is wired into the pre-commit gate and flags this class of cross-crate attribution.Known limitations, disclosed deliberately
reserving_mpsccan lose an item after 2^32 pushes, on every target -- not only 32-bit ones. Its claim position is a 32-bit half of a packed word by construction. A producer that checks for room, is descheduled, and resumes after a complete wrap can write into a slot whose emptiness was decided a generation earlier, silently overwriting an item the consumer had not taken. Measured exposure: 37 seconds to ~4 minutes of sustained pushing. The crate docs lead with this, andslotwise_mpscdoes not have the hazard (64-bit positions on every target).permit_mpscis experimental and exempt from the crate's semver promise. Behind the non-defaultexperimental-permit-claimfeature; it exists to be measured against the shipping claim protocol, and will be merged intoreserving_mpscor deleted.SYSTEM_CPU_SET_INFORMATION::AllFlagsreads constant zero on this build even afterSetProcessDefaultCpuSetssucceeds, so the bit meanings are neither confirmed nor falsifiable here.Fullreturn is out of scope for the queue crate's first release.PushErroris#[non_exhaustive], so adding to it later is not breaking.Dependencies
No third-party dependency is added and no existing external dependency changes version. The only
Cargo.lockadditions are the two new workspace-local crates.What a consumer pulls in on a default build:
wtf-stringwindows-waitable-queueswindows-sys->windows-linkwindows-topology-syswindows-sys->windows-linkwindows-file-watcherlog,windows-sys->windows-linkBoth new publishable crates cost exactly one external dependency, and it is the one every other crate here already uses.
serde(topology),serde/serde_json(file watcher) andwindows-core(wtf-string) are optional and off by default. Across the whole workspace the external set is 14 crates, with no duplicate versions.cargo publish --dry-runsucceeds forwindows-waitable-queues,windows-topology-sys,windows-file-watcherandwindows-ioring-sys. Nothing pinswindows-topology-sysorwindows-waitable-queuesto a version, so the bumps above cannot break resolution inside the workspace.CI and release automation
numa-spikesjob running the standalone NUMA spikes through the scratch-crate procedure their README documents, so that instruction cannot rot silently. Observational, but it fails when a spike fails to build or run.release-placement-probe.ymlbuilds, verifies and attacheswindows-placement-probebinaries for x86_64 and ARM64, checking that artifacts carry a build-identity stamp and that unofficial builds cannot be released by accident. Released binaries carry a GitHub artifact attestation -- a signed statement binding the exact bytes to this repository, workflow and commit, verifiable withgh attestation verify. They are not Authenticode-signed, and the "official" stamp in--versionis a self-reported build-identity marker, not a signature:build.rsreads it from environment variables, so it catches an accidental local build rather than a forgery.