Skip to content

Add waitable queues and the placement probe, and reshape the topology model - #56

Open
MikeGrier wants to merge 358 commits into
mainfrom
mikegrier/deferred-namespace-ops
Open

Add waitable queues and the placement probe, and reshape the topology model#56
MikeGrier wants to merge 358 commits into
mainfrom
mikegrier/deferred-namespace-ops

Conversation

@MikeGrier

@MikeGrier MikeGrier commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Adds two Windows-only crates, replaces windows-topology-sys' model, and puts the release machinery in place to ship them.

Crate State after merge
windows-waitable-queues new, publishable -- first release
windows-topology-sys reshaped -- new model replaces 0.1.0's
windows-placement-probe new, publish = false -- shipped as a prebuilt binary
topology-planner new -- planning documents only, no code

windows-waitable-queues

Bounded producer/consumer queues whose readiness is a waitable Windows HANDLE.

That is the crate's reason to exist. crossbeam-channel blocks on its own internal primitive and exposes no HANDLE, and its Select accepts only channel operations, so a thread that must wake on "a message arrived or my I/O completed or shutdown was signalled" cannot express that wait. A queue whose readiness is a HANDLE composes with WaitForMultipleObjects, MsgWaitForMultipleObjects, thread-pool waits, and alertable waits.

Every item is behind cfg(windows); the crate builds to an empty shell elsewhere.

Three shapes, none canonical -- there is no type named Queue, so a caller names the shape it wants:

Module Producers Storage Full behaviour
spsc one (!Clone producer) bounded ring refuses
slotwise_mpsc many (Clone producer) bounded array, per-slot sequence refuses
reserving_mpsc many bounded array, packed claim word refuses, honours reservations

reserving_mpsc adds reservations: reserve() claims capacity up front and returns a Reservation redeemed by send or released on drop. A granted reservation is always redeemable, so a caller can establish it has room before doing work whose result must not be dropped. The queue stays connected while a reservation is outstanding.

The doorbell is a lazily created manual-reset event per queue. A consumer that only polls never allocates a kernel object; one that waits calls doorbell() for a borrowed HANDLE (or doorbell_owned() for an OwnedHandle). arm() clears the doorbell and then asks whether an item is takeable, so a push landing in that window cannot have its signal erased.

windows-topology-sys

The published 0.1.0 modelled a ladder of levels with optional rungs. This replaces it with a model of observed connectivity, which is the shape of question consumers actually ask.

  • Topology is now MachineMemoryTopology -- the old name claimed more than the type delivers.
  • Relations are a set with per-relation provenance, not reduced on insert. Observation and Source record which observer saw a relation, so CPU Sets and the relationship walk sit side by side instead of merging into one lossy answer.
  • Observed<T> replaces Option<T> where absence was ambiguous. Known / Absent / NotObserved are three distinct facts, carried by serde as three distinct encodings (number, explicit null, omission).
  • Granularity orders what a set of processors shares by observed set inclusion, with minimal_shared as the meet and the whole machine as the top. "How close are these two processors" is therefore answered from observed membership across every domain kind at any depth, rather than from a firmware-asserted cache level.
  • proximity reads an inclusion-ordered partitioning rather than re-implementing the partitioning rule.
  • distances and Distances are deleted. The crate does not go below the Win32 topology APIs, so a fact Win32 does not report is not a fact the crate has. The field was never populated.
  • Domain::id is removed in favour of per-observation labels: one id cannot name a domain two observers labelled differently.
  • discover retries until the two sources agree, up to three passes, and states the outcome as a new public Coherence on the topology. Windows offers no transactional way to read the relationship walk and the CPU-set enumeration together, so a processor hot-added between the calls appears in one and not the other; a second pass settles that, and what survives the bound is reported as a genuine disagreement rather than hidden. Grouping disagreements are deliberately not retried -- they are persistent in the field, so re-reading cannot settle them.

windows-placement-probe

A measurement tool that reports where the OS actually places threads and memory, released as a prebuilt binary rather than a crate. Date-based version (2026.902.0) because its output is dated evidence, not an API contract.

It places every online processor, refuses to invent NUMA membership, reports a cache relationship the topology never established as unknown rather than as same-or-cross, and publishes its record backup by rename so a failed write cannot leave a truncated file under a complete record's name.

topology-planner

Planning and design documents for a future planner consuming an abstracted topology description. No code ships: the topology crate is justified as the refined view of what the platform publishes, with an adapter absorbing the planner's needs.


Breaking changes for consumers

  • windows-topology-sys -- Topology renamed; Domain::id, distances and Distances removed; Option<T> becomes Observed<T> on Memory::memory_bytes; usable() removed (it consulted a CPU-set byte this build never populates and returned false for every processor, and it encoded policy in a facts crate).
  • windows-file-watcher -- reopen-by-id removed.
  • windows-waitable-queues -- first release, so nothing to migrate.

Versions this merge proposes

Crate From To
windows-topology-sys 0.1.0 0.2.0
windows-waitable-queues 0.0.1 0.1.0
windows-file-watcher 0.1.3 0.2.0
windows-ioring-sys 0.2.0 0.2.1 (pinned)
windows-thread-ambient-sys 0.2.0 0.2.1
windows-file-watcher-example-test-harness 0.1.2 0.1.3

The other six released crates get no release: they carry only test:, docs: and refactor: commits since their tags.

windows-ioring-sys is pinned to 0.2.1 because no public item in it changed on this branch -- release-please attributes commits by the paths they touch, and two topology-scoped breaking commits reached it through an example and one doc-comment heading. A 0.3.0 would send consumers looking for a migration that does not exist. tools/check-commit-scope.ps1 is wired into the pre-commit gate and flags this class of cross-crate attribution.


Known limitations, disclosed deliberately

  • reserving_mpsc can lose an item after 2^32 pushes, on every target -- not only 32-bit ones. Its claim position is a 32-bit half of a packed word by construction. A producer that checks for room, is descheduled, and resumes after a complete wrap can write into a slot whose emptiness was decided a generation earlier, silently overwriting an item the consumer had not taken. Measured exposure: 37 seconds to ~4 minutes of sustained pushing. The crate docs lead with this, and slotwise_mpsc does not have the hazard (64-bit positions on every target).
  • permit_mpsc is experimental and exempt from the crate's semver promise. Behind the non-default experimental-permit-claim feature; it exists to be measured against the shipping claim protocol, and will be merged into reserving_mpsc or deleted.
  • CPU-set flag bit positions are unverified. SYSTEM_CPU_SET_INFORMATION::AllFlags reads constant zero on this build even after SetProcessDefaultCpuSets succeeds, so the bit meanings are neither confirmed nor falsifiable here.
  • Producer-side backpressure beyond a Full return is out of scope for the queue crate's first release. PushError is #[non_exhaustive], so adding to it later is not breaking.

Dependencies

No third-party dependency is added and no existing external dependency changes version. The only Cargo.lock additions are the two new workspace-local crates.

What a consumer pulls in on a default build:

Crate External tree
wtf-string nothing
windows-waitable-queues windows-sys -> windows-link
windows-topology-sys windows-sys -> windows-link
windows-file-watcher log, windows-sys -> windows-link

Both new publishable crates cost exactly one external dependency, and it is the one every other crate here already uses. serde (topology), serde/serde_json (file watcher) and windows-core (wtf-string) are optional and off by default. Across the whole workspace the external set is 14 crates, with no duplicate versions.

cargo publish --dry-run succeeds for windows-waitable-queues, windows-topology-sys, windows-file-watcher and windows-ioring-sys. Nothing pins windows-topology-sys or windows-waitable-queues to a version, so the bumps above cannot break resolution inside the workspace.

CI and release automation

  • Probe jobs measuring topology, doorbell cost, and request cost.
  • A numa-spikes job running the standalone NUMA spikes through the scratch-crate procedure their README documents, so that instruction cannot rot silently. Observational, but it fails when a spike fails to build or run.
  • release-placement-probe.yml builds, verifies and attaches windows-placement-probe binaries for x86_64 and ARM64, checking that artifacts carry a build-identity stamp and that unofficial builds cannot be released by accident. Released binaries carry a GitHub artifact attestation -- a signed statement binding the exact bytes to this repository, workflow and commit, verifiable with gh attestation verify. They are not Authenticode-signed, and the "official" stamp in --version is a self-reported build-identity marker, not a signature: build.rs reads it from environment variables, so it catches an accidental local build rather than a forgery.
  • A CI check asserts every release-managed crate actually has a publish trigger.

Copilot AI lite review requested due to automatic review settings August 31, 2026 21:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Confirmed correctness issues in new code paths (notably recv_timeout potentially waiting forever and test hook state leaking on panic) should be fixed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This pull request expands the workspace’s Windows observability and measurement tooling (new probes + CI jobs), introduces new crates (windows-waitable-queues, windows-placement-probe), and hardens topology trust semantics by adding an explicit provenance marker to windows-topology-sys.

Changes:

  • Add windows-waitable-queues (bounded, waitable HANDLE-backed queues) and wire it into workspace + release tracking.
  • Add windows-placement-probe and expand windows-platform-probes (new probe binaries + tests), then run probes/spikes in CI for ongoing fleet measurement.
  • Add Provenance to windows-topology-sys so serialized / hand-built topologies cannot silently pass as “measured”.
File summaries
File Description
tools/run-numa-spikes.ps1 Builds/runs standalone NUMA “spike” instruments in a scratch crate and captures transcripts.
.github/workflows/ci.yml Runs additional probes (topology, doorbell cost, request cost) and adds a “numa-spikes” observational job with artifact upload.
release-please-config.json Adds windows-waitable-queues to release-please configuration.
.release-please-manifest.json Adds crates/windows-waitable-queues version for release tracking.
Cargo.toml Adds windows-placement-probe and windows-waitable-queues to workspace members.
Cargo.lock Records new workspace crates and their dependency edges.
.gitignore Ignores placement-probe-v*.json backup records to avoid accidental commits.
PLANS.md Adds planning rows / trackers for the new crate and related checklists.
crates/windows-waitable-queues/PLANS.md Adds per-component plans pointer back to the root checklist.
crates/windows-waitable-queues/Cargo.toml Declares the new publishable windows-waitable-queues crate and its windows-sys feature set.
crates/windows-waitable-queues/src/race_hooks.rs Adds deterministic race-window hooks for tests to drive correctness-critical interleavings.
crates/windows-waitable-queues/src/options.rs Adds an Options builder for construction-time queue switches (disposal policy, high-water tracking).
crates/windows-waitable-queues/src/metrics.rs Adds queue metrics (refusals + optional high-water).
crates/windows-waitable-queues/src/metrics/tests.rs Tests metric arithmetic and concurrency behavior in isolation.
crates/windows-waitable-queues/src/disposal.rs Adds teardown policy plumbing for handing undrained items to a caller-provided sink.
crates/windows-waitable-queues/src/disposal/tests.rs Tests disposal/teardown behavior including panicking sinks.
crates/windows-waitable-queues/src/capacity.rs Centralizes bounded-capacity validation shared across queue shapes.
crates/windows-waitable-queues/src/blocking.rs Adds shared blocking recv / recv_timeout loop that parks on a Win32 event handle.
crates/windows-topology-sys/src/topology.rs Adds Topology::provenance and stamps Measured only in discover(), with serde downgrade-on-load.
crates/windows-topology-sys/src/topology/tests.rs Updates/extends tests to pin provenance downgrade semantics and safe defaults.
crates/windows-topology-sys/src/provenance.rs Introduces Provenance type plus serde downgrade helper.
crates/windows-topology-sys/src/provenance/tests.rs Tests provenance ordering, defaults, downgrade rules, and rendering.
crates/windows-topology-sys/src/lib.rs Exposes Provenance from the crate.
crates/windows-topology-sys/DESIGN-NOTES.md Documents the provenance decision (D-12) and its rationale/semantics.
crates/windows-ioring-sys/design-sessions/spikes/README.md Documents why certain spikes are checked in “ready but unrun” and how to interpret/run them.
crates/windows-ioring-sys/CHECKLIST.md Adds/updates queued repairs from the NUMA-sharding measurement (M20).
crates/windows-platform-probes/Cargo.toml Adds new probe binaries and depends on shipping crates for measurement fidelity.
crates/windows-platform-probes/src/lib.rs Exposes new probe modules.
crates/windows-platform-probes/src/tests.rs Adds topology probe consistency tests against real host counters.
crates/windows-platform-probes/src/request_cost.rs Adds request construction timing probe logic.
crates/windows-platform-probes/src/bin/topology.rs New probe-topology binary emitting both human output and a single JSON line for mining.
crates/windows-platform-probes/src/bin/doorbell_cost.rs New probe-doorbell-cost binary measuring doorbell relative costs.
crates/windows-platform-probes/src/bin/request_cost.rs New probe-request-cost binary reporting request construction costs and ratios.
crates/windows-platform-probes/src/bin/queue_contention.rs New probe-queue-contention binary exploring queue tail-claim contention scaling.
crates/windows-placement-probe/Cargo.toml Introduces windows-placement-probe crate and build identity stamping (publish=false for now).
crates/windows-placement-probe/README.md Documents purpose, privacy posture, usage, and provenance expectations for runners.
crates/windows-placement-probe/DESIGN-NOTES.md Records publication/provenance constraints and why crates.io must not become the primary path yet.
crates/windows-placement-probe/build.rs Stamps commit/dirty/source into the binary (CI vs local vs unknown).
crates/windows-placement-probe/src/lib.rs Defines the crate’s module surface and documentation for the measurement tool.
crates/windows-placement-probe/src/build_identity.rs Implements build identity model + trust ordering and “official build” predicate.
crates/windows-placement-probe/src/build_identity/tests.rs Tests build identity trust semantics and build-script stamping shape.
crates/windows-placement-probe/src/submission.rs Renders the “paste-able” submission payload and includes a truncation-detecting checksum.
crates/windows-placement-probe/src/peer_index_cache/tests.rs Tests correctness/honesty of memory placement reporting (esp. on single-node hosts).
crates/windows-placement-probe/src/machine/tests.rs Tests registry-based machine description collection policy and safety invariants.
crates/windows-placement-probe/schema/v1.txt Schema golden for v1 submission records.
crates/windows-placement-probe/schema/v2.txt Schema golden for v2 submission records (adds memory_node).
CHECKLIST.md Records completed tooling item(s) related to sabotage harness promotion.
COMPLETED-PLANS.md Archives the completed topology provenance checklist entry.
COMPLETED-CHECKLIST.md Archives the completed topology provenance checklist in detail.
Review details
  • Files reviewed: 81/95 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread crates/windows-waitable-queues/src/blocking.rs Outdated
Comment thread crates/windows-waitable-queues/src/race_hooks.rs
Copilot AI review requested due to automatic review settings August 31, 2026 22:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

There are a few correctness/documentation issues in the new queue crate (broken rustdoc links and a test hook that can leak state on panic) that should be fixed before merging.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

crates/windows-waitable-queues/src/race_hooks.rs:78

  • Hook::with does not clear/restore the thread-local hook if body panics (e.g., a failing assertion inside the test closure). That can leak the hook into later tests on the same thread and cause cascading/irreproducible failures. Use a drop guard to restore the previous hook even during unwinding.
  • Files reviewed: 81/95 changed files
  • Comments generated: 2
  • Review effort level: Lite

Comment thread crates/windows-waitable-queues/src/blocking.rs
Comment thread crates/windows-waitable-queues/src/metrics.rs Outdated
Copilot AI review requested due to automatic review settings August 31, 2026 22:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

There are a few correctness/behavior issues in the new queue code/docs (panic cleanup in race_hooks::Hook::with, potential tight-loop at sub-millisecond timeouts in recv_timeout, and Metrics rustdoc inconsistencies) that should be addressed before approval.

Review details

Suppressed comments (4)

Previously missed (2) — in code that hasn't changed since the last review.

crates/windows-waitable-queues/src/metrics.rs:13

  • The module docs say Metrics holds three counters including "Doorbell rings", but this type currently only stores refused and high_water (doorbell rings are tracked in doorbell::Doorbell). This makes the rustdoc misleading for readers looking for where to get the ring count.

This issue also appears on line 20 of the same file.
crates/windows-waitable-queues/src/blocking.rs:140

  • When remaining is non-zero but < 1ms, remaining.as_millis() truncates to 0, so this computes millis = 0 and can spin in a tight loop until the deadline. Clamping to at least 1ms avoids the busy-wait at the end of the timeout budget.

crates/windows-waitable-queues/src/metrics.rs:20

  • The docs below still describe a 3-counter design (including a "Rings" counter) and explain why "two of the three are free". Since ring counting lives in doorbell::Doorbell rather than Metrics, this section should be updated to describe only the counters actually accumulated here.
    crates/windows-waitable-queues/src/race_hooks.rs:77
  • Hook::with claims to install a hook only for the duration of body, but if body panics (or unwinds via ? through a panic), the hook is never cleared because the cleanup happens after body() returns. This can leak the hook into subsequent operations on the same thread and make later tests flaky or misleading.
  • Files reviewed: 81/95 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

MikeGrier pushed a commit that referenced this pull request Aug 31, 2026
CI's rustdoc job has failed on all three runs of pull request #56. `main` is
green, so the branch broke it -- most likely the `mpsc` -> `slotwise_mpsc`
rename, which moved every item these links named.

Twelve sites across three crates. The job denies `broken_intra_doc_links`,
`private_intra_doc_links` and `invalid_rust_codeblocks`, so eleven of these were
errors rather than warnings.

`MIN_CAPACITY` did not exist anywhere in the workspace: the public docs for
`slotwise_mpsc::bounded` referred readers to a constant that was never written.
The rule it was supposed to explain lives in the private `BOUNDS`, so the
sentence now states the rule itself -- a capacity of one cannot distinguish
"published" from "free" under the sequence protocol.

The links into private items are delinked rather than repointed, because a
public page cannot link to a page rustdoc does not generate. Where the private
name is still useful to a maintainer it stays as inline code.

Also corrects SH-2.4, which described these as pre-existing warnings to clear
before publication. They were neither pre-existing nor warnings, and nothing was
waiting on the release -- the release was waiting on them.

Completed item: SH-2.4: Clear the eight rustdoc warnings in windows-waitable-queues before it is published

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 31, 2026 22:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

There are correctness issues in new test infrastructure and CI tooling (panic-unsafe hook scoping and missing runtime failure handling in the NUMA spike runner) that should be fixed before merging.

Review details

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

crates/windows-waitable-queues/src/options.rs:49

  • The rustdoc example comment says “The peak, not the depth right now.”, but the next line asserts rx.len() == 1 (current depth). This reads as a contradiction for users learning the API. Adjust the comment so it matches what’s being asserted.

crates/windows-waitable-queues/src/race_hooks.rs:78

  • Hook::with does not clear the thread-local hook if body panics, which can leak the hook into later code on the same thread (e.g., if a test uses catch_unwind / asserts a panic) and make failures non-deterministic. Make the hook clearing panic-safe via an RAII guard that resets the slot in Drop.
  • Files reviewed: 82/96 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

MikeGrier pushed a commit that referenced this pull request Aug 31, 2026
Three findings from the Copilot review on pull request #56.

**`recv_timeout` could become unbounded.** The blocking loop clamped an
oversized remaining duration to `u32::MAX`, which is the same value as
`INFINITE` -- imported three lines above the clamp in the same file. A timeout
longer than about 49.7 days therefore waited forever instead of timing out, and
the loop that was supposed to re-check the deadline never regained control to do
it. The existing comment reasoned carefully about clamping versus truncating and
was right about everything except the one value it chose.

The reviewer's suggested fix, clamping to `u32::MAX - 1`, is incomplete, and a
boundary test written for it is what showed that: a duration of exactly
`u32::MAX` milliseconds *converts* successfully, so the fallback never fires and
`INFINITE` is returned by the conversion rather than by the clamp. Both guards
are needed, and they cover different inputs. The clamp is now derived from
`INFINITE` rather than written as a number, and the arithmetic is extracted so
it can be tested -- the failing case takes 49 days to observe through the public
API and so could never be a test of the loop.

**A hook survived a panic.** `Hook::with` removed the installed hook with a
statement after the body, which an unwind skips. A test that installs a hook and
then fails an assertion -- the ordinary way for a test to fail -- left it
installed to fire inside whatever ran next on that thread, in the facility this
crate's central correctness argument rests on. Now removed by a guard, using
`try_with` so teardown cannot replace an unwind already in progress.

**Eighteen broken documentation links, from a report of two.** `../../` from
`src/*.rs` resolves to `crates/`, which holds no `DESIGN-NOTES.md`; the crate's
own notes are one level up. Sweeping the workspace found sixteen more of the
same, in `spsc`, `traits`, `reserving_mpsc` and `slotwise_mpsc`.

The sweep also caught its own repair damaging a correct link:
`windows-file-watcher/src/contract.rs` used `../../../DESIGN-NOTES.md` to reach
the *workspace* notes deliberately, and a blanket prefix replacement shortened
it. Reverted, and every relative markdown link in every Rust source is now
checked to resolve -- 36 of them, none broken.

The review's fourth point, that `race_hooks` puts a thread-local lookup on
production hot paths, does not hold: `mod race_hooks` and all four call sites
are already `#[cfg(test)]`.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 31, 2026 23:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

There are a few correctness/docs issues (notably build-stamp invalidation and misleading docs) that should be fixed before merging.

Review details

Suppressed comments (3)

Previously missed (3) — in code that hasn't changed since the last review.

crates/windows-waitable-queues/src/options.rs:49

  • The doc example comment says rx.len() is "The peak", but len() reports the current depth (peak is high_water()). This is misleading for API consumers reading the docs.
    crates/windows-placement-probe/build.rs:33
  • Watching only .git/HEAD is usually insufficient to keep the stamped commit fresh: on most repos HEAD contains a stable ref: refs/heads/<branch> line and does not change when new commits are made. This can leave PLACEMENT_PROBE_COMMIT_OUT stale after committing without editing files.
    crates/windows-platform-probes/src/request_cost.rs:157
  • These Win32 parameters are currently magic numbers, which makes it hard to tell what kind of open is being timed (and easy to accidentally change semantics later). Prefer the named windows-sys constants so the request is self-documenting.
  • Files reviewed: 84/98 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings August 31, 2026 23:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Release tracking for windows-waitable-queues is added, but the publish automation/tag triggers are not updated to actually publish it, and there are also concrete doc/identity-stamping issues identified in the new code.

Review details

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

crates/windows-waitable-queues/src/metrics.rs:14

  • The module docs say this type tracks three counters (refusals, doorbell rings, peak depth), but Metrics only stores refused and high_water. Either the docs are stale or a counter is missing; as written it misleads readers about where doorbell ring counts live.

This issue also appears on line 20 of the same file.
crates/windows-placement-probe/build.rs:33

  • build.rs prints rerun-if-* directives, which disables Cargo's default "rerun the build script when any package file changes" behavior. As a result, local edits to src/ (which make the tree dirty) will not cause the stamp to be recomputed, so dirty/commit can go stale across rebuilds.

crates/windows-waitable-queues/src/metrics.rs:24

  • Follow-on to the docs above: the rest of this section describes rings as a counter maintained here ("two of the three are free..."). With doorbell ring counts not stored in Metrics, this section should be updated so it only explains the two counters Metrics actually owns.
  • Files reviewed: 84/98 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

MikeGrier pushed a commit that referenced this pull request Aug 31, 2026
Five findings from the Copilot reviews on pull request #56. These arrived only
in review *bodies* rather than as inline threads, so the thread query used last
time could not see them.

**The build stamp did not update on commit.** `build.rs` watched
`../../.git/HEAD`, and on a branch that file holds `ref: refs/heads/<branch>` --
a line that does not change when you commit. Since a `rerun-if-changed`
directive replaces cargo's default of watching the whole package, the script
never re-ran and the binary kept whatever commit it was first built with.

Measured rather than reasoned about: `.git/HEAD` had not been touched in 21
hours while fourteen commits landed, and a freshly rebuilt binary reported
`b6f23ec9f3bc` against a `HEAD` of `0b96b14746bb` -- six behind. CI hides this
completely, because there the commit arrives through `PLACEMENT_PROBE_COMMIT`,
so the stamp was wrong only on local builds, which are exactly the ones whose
commit is their only traceability. The whole submission record rests on this
field.

Now resolves what `HEAD` points at and watches that too, handling a detached
`HEAD` (which changes on its own), a packed ref (where `packed-refs` changes
instead), and a `.git` file redirecting to a worktree. Paths are emitted only
when they exist, since a missing path makes cargo re-run the script on every
build.

**`recv_timeout` busy-waited below a millisecond.** A remainder under 1 ms
truncates to zero, and a zero wait returns at once, so the loop re-armed and
re-waited without sleeping. Arming clears the doorbell, which is a `ResetEvent`
syscall, so this was a syscall storm rather than merely a hot loop. Clamped to a
millisecond: overshooting a blocking deadline by less than a timer tick is the
right trade, and sub-millisecond precision is not available from a blocking wait
at any price.

Also: the `Options` doctest labelled `rx.len()` as "the peak" when it is the
current depth; the `metrics` module described a ring counter that lives on
`Doorbell` without saying so; and `request_cost` passed `0x8000_0000`, `1` and
`3` to an open request instead of `GENERIC_READ`, `FILE_SHARE_READ` and
`OPEN_EXISTING`, which this repository's conventions forbid. The three constants
were checked against `windows-sys` rather than assumed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 31, 2026 23:40
@MikeGrier

Copy link
Copy Markdown
Owner Author

Working through the five findings raised in the last four review bodies. These arrived as suppressed comments in the review summaries rather than as inline threads, so they were invisible to a reviewThreads query -- worth noting for anyone else automating this.

All five were checked against the code before acting, and all five were real. Fixed in dd2e9c5 and 81cff89.

1. Build stamp invalidation (build.rs:33) -- the most serious of these

Correct, and worse in practice than the report suggests. Since a rerun-if-changed directive replaces cargo's default of watching the whole package, watching only .git/HEAD meant the script re-ran essentially never.

Measured rather than argued:

.git/HEAD mtime:   2026-08-30 22:12:39   <- 21 hours and ~14 commits ago
ref file mtime:    2026-08-31 19:24:02   <- matches the last commit exactly

and the consequence, on a freshly rebuilt binary:

actual HEAD:  0b96b14746bb
binary says:  !!UNOFFICIAL!! v0.1.0 b6f23ec9f3bc DIRTY [LOCAL]   <- six commits behind

CI hides this completely, because there the commit arrives through PLACEMENT_PROBE_COMMIT rather than from git. So the stamp was wrong only on local builds -- which are precisely the builds marked [LOCAL], whose commit is the only traceability they have, and the whole submission record rests on that field.

Now resolves what HEAD points at and watches that too, covering a detached HEAD (changes on its own), a packed ref (packed-refs changes instead), and a .git file redirecting to a worktree. Paths are emitted only when they exist, because a missing path makes cargo re-run the script on every build.

Verified end to end: committed, rebuilt without touching any source file, and the stamp followed 0b96b14 -> dd2e9c5.

2. Sub-millisecond tight loop (blocking.rs:140)

Correct, and the cost is higher than a spin. The loop re-arms each turn, and arming clears the doorbell -- so a zero-millisecond wait made it a ResetEvent syscall storm for the last fraction of the timeout budget, not just a hot loop.

Clamped to one millisecond. Overshooting a blocking deadline by less than a timer tick is the right trade: the granularity is coarser than a millisecond anyway, so sub-millisecond precision is not purchasable from a blocking wait at any price -- what a caller would get instead is a burning core. Two tests cover it, and both fail if the lower bound is removed.

3. Metrics rustdoc describes a counter it does not hold (metrics.rs:13, :20)

Correct. Metrics holds refused and high_water only; rings are counted on Doorbell::rings. The design of placing each counter where its cost is already paid was described, but never actually said where rings ended up, so a reader looking for the ring count on Metrics was left without a pointer. Both places now name Doorbell and link to it.

4. Options doctest contradicts itself (options.rs:49)

Correct, and it is a rendered doctest, so it was teaching the wrong thing on the API page. The comment "The peak, not the depth right now." sat directly above assert_eq!(rx.len(), 1), which is the depth right now. Reworded to describe both lines: len is the current depth, high_water is the peak.

5. Magic Win32 parameters (request_cost.rs:157)

Correct, and this repository's conventions forbid bare numeric literals of this kind outright. Now GENERIC_READ, FILE_SHARE_READ and OPEN_EXISTING. The three values were checked against windows-sys rather than assumed: 2147483648, 1, 3 -- matching the literals they replaced.


Unrelated, but found by CI on the same push and worth flagging here: slotwise_mpsc computed its high-water depth after the release store that publishes a slot, so the consumer could drain past that position and the subtraction wrapped. The peak was recorded through fetch_max, so one race poisoned the metric permanently. Fixed in 0b96b14 by moving the read above the publication, which makes the subtraction unable to go negative rather than clamping it after the fact.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

tools/run-numa-spikes.ps1 currently builds scratch spike crates without a lockfile and does not surface non-zero cargo run exits, which can cause silent probe failures and CI flakiness over time.

Review details
  • Files reviewed: 83/98 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Processor-group handling, provenance stamping, probe reporting, and release plumbing contain correctness and reliability gaps.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 83/98 changed files
  • Comments generated: 18
  • Review effort level: Balanced

Comment thread crates/windows-placement-probe/src/core_affinity.rs Outdated
Comment thread crates/windows-placement-probe/build.rs
Comment thread .github/workflows/release-placement-probe.yml Outdated
Comment thread crates/windows-waitable-queues/src/lib.rs Outdated
Comment thread crates/windows-waitable-queues/README.md Outdated
Comment thread crates/windows-platform-probes/src/bin/topology.rs Outdated
Comment thread crates/windows-platform-probes/src/bin/core_affinity.rs Outdated
Comment thread crates/windows-platform-probes/src/bin/peer_index_cache.rs Outdated
Comment thread crates/windows-platform-probes/src/bin/request_cost.rs Outdated
Comment thread crates/windows-platform-probes/src/bin/core_affinity.rs Outdated
Copilot AI review requested due to automatic review settings September 1, 2026 00:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The long-path probe leaks process state and temporary trees, while deserialization preserves topology anomalies contrary to the restored-data contract.

Review details
  • Files reviewed: 91/247 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

… from a document

Two findings from Copilot review 5118406119, a headline-only round. Both real.

The long-path probe leaked process state and a temporary tree. `measure` called
SetCurrentDirectoryW and never moved back, and built a 40-deep tree under %TEMP%
that nothing removed. Neither is excused by the probe binaries exiting straight
afterwards: this is a library function, so a test or any other caller keeps
running in a process whose current directory now points into a temp directory.
Seven stale trees were sitting on this machine, which is how the leak was
confirmed rather than argued.

The tree is the sharper half. It is deliberately longer than MAX_PATH, which is
exactly what stops Explorer and `del` from removing it -- litter from a probe
about long paths is litter that is awkward to clear by hand.

Both are now released by an `Apparatus` guard, so the early returns on every
apparatus failure clean up too. Restoring the directory before removing the tree
is load-bearing rather than tidy: a process's current directory holds a handle on
it, so removal while parked inside silently fails. Sabotage confirmed exactly
that -- deleting only the restore also broke the tree test.

Writing the tests found a third thing: `measure` cannot be called concurrently.
It borrows the current directory, which is one per process, so a unique root per
call would not fix it. The three tests collided until they took a lock, and
`measure` now documents the constraint instead of leaving the next caller to
discover it.

Deserialization asserted enumeration anomalies into a restored topology.
`enumeration_anomalies` was `serde(default)` while its own first paragraph has
always said the list is "empty for a hand-built or deserialized topology, which
asked nothing" -- a contract contradicted by its own attribute. An anomaly is a
fact about an enumeration and a deserialized topology performed none, which is
the same reasoning that already made `coherence` skip_deserializing. It now does
too, and is still written out, so a dump still carries the diagnosis.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The 248-file concurrency-heavy change has unresolved long-path probe correctness issues and requires final human review.

Review details
  • Files reviewed: 91/248 changed files
  • Comments generated: 2
  • Review effort level: Balanced

observation.apparatus_error = Some(error);
return observation;
}
let current_dir_len = root.as_os_str().len();

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 231750b. You were right, and the workspace already had the answer: wtf-string exists for exactly this, so resolved_len is now counted with Wtf16String::from_os_str(...).len() -- the string held in the encoding Windows uses, so its len is the number under test rather than a conversion of one.

The relative part is built from ASCII constants and so cannot differ today; it is measured the same way anyway, because a unit that is only correct while the input happens to be ASCII is one waiting to be wrong. The field now documents the unit, and a test pins that the two counts agree for ASCII and diverge for U+00E9 / U+4E2D -- the direction that matters, since bytes over-count and would report a refusal as expected when it was not.

Comment on lines +54 to +58
let all = [
Provenance::Synthetic,
Provenance::Restored,
Provenance::Measured,
];

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 231750b. The comment promised something the code could not keep -- an array is not exhaustive over an enum -- so the list now comes from a match, and the compiler refuses to build until a new variant is added to it.

Verified by adding a fourth Provenance variant: the build fails at this test's own line with non-exhaustive patterns: Provenance::Inferred not covered, which is the guarantee the comment claimed.

Mike Grier and others added 2 commits September 4, 2026 20:10
…enance sweep exhaustive

Three findings from the unresolved PR #56 review threads.

The long-path probe measured against MAX_PATH in the wrong unit. `OsStr::len`
counts Rust's platform encoding, which is WTF-8; MAX_PATH counts UTF-16 code
units. The two agree for ASCII and diverge for anything else, so a %TEMP% with a
non-ASCII character made `resolved_len` too large and could report an attempt on
the wrong side of the ceiling -- in a probe whose entire output is which side of
that ceiling a path landed on.

Now counted with `wtf_string::Wtf16String`, which is this workspace's own answer
to exactly this question: it holds the string in the encoding Windows uses, so
its `len` is the number under test rather than a conversion of one. The relative
part is built from ASCII constants and so cannot currently differ, and is
measured the same way regardless -- a unit that is only correct while the input
happens to be ASCII is one waiting to be wrong.

The Provenance pairwise test claimed to be exhaustive and was not. Its comment
promised that "a variant added later cannot quietly acquire an upgrade path"
while the list was a hand-written array that a new variant would leave untouched.
It now comes from a `match`, so the compiler refuses to build until the variant
is added -- verified by adding one and confirming the test's own line fails to
compile.

The topology crate's PLANS.md still advertised M6 as in progress with every M6
item checked, directing a reader at finished work. M6 is archived to
COMPLETED-CHECKLIST.md and recorded in COMPLETED-PLANS.md. The `Deferred, and
why` section stays in CHECKLIST.md: it is live context about two things left out
on purpose, not completed work, and it rode along in the first move.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
SH-4.13 groups three unresolved threads with one root: ProcessorSet cannot
represent every u8 processor id while Processor::id is public and constructible,
so a deserialized or hand-built topology panics in machine_processors() and in
granularity's insert. Per-site validation would close the reported path and leave
the next open, so the decision is about ProcessorSet's representable range.

SH-4.14 records that probe-long-path-aware builds without its manifest on
windows-gnu, making the aware/unaware pair measure one configuration while
claiming two. Gated on CI building a GNU target, since a fix would otherwise ship
unexercised.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The topology checklist retains prohibited non-action history, and the new architecture-dependent representability API lacks direct boundary coverage.

Review details
  • Files reviewed: 91/248 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Release planning contradicts the permanent non-publication decision, and an active checklist contains non-actionable historical rationale.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

crates/windows-placement-probe/Cargo.toml:28

  • The permanent publish = false decision has not been propagated to the active workspace plan. PLANS.md:25 still calls this a publishable tool and says M5+ will publish it to crates.io, while CHECKLIST-placement-tool.md:460-499 records the reversal. Update the plan row so future work does not act on the superseded distribution contract.
  • Files reviewed: 91/248 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

…mespace-ops

# Conflicts:
#	crates/windows-file-watcher/COMPLETED-CHECKLIST.md
#	crates/windows-file-watcher/DESIGN-NOTES.md
#	crates/windows-file-watcher/src/directory/tests.rs
#	crates/windows-file-watcher/src/queue.rs
#	crates/windows-file-watcher/src/queue/tests.rs
#	crates/windows-file-watcher/tests/reopen_by_id_cannot_be_watched.rs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Sensitive partial probe records remain trackable, allocator-dependent tests assert behavior Rust does not guarantee, and documentation still contradicts implemented contracts.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 84/228 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment on lines +310 to +313
assert!(
fitted.capacity() < fitted_before,
"shrink_to_fit must release excess capacity on the Windows allocator"
);
Comment on lines +10 to +12
## Deferred, and why

Two things were deliberately left out of the reshape rather than forgotten:
Comment on lines +5 to +19
//! This is the part of the crate its name refers to. A queue shape owns a
//! [`Doorbell`] and keeps it in agreement with its own emptiness; a client that
//! wants to park on the queue *and* on an I/O completion *and* on a shutdown
//! event in one wait borrows the handle and hands it to
//! `WaitForMultipleObjects` alongside the others.
//!
//! # Manual-reset, and level-triggered
//!
//! The event is manual-reset, so it means "there is something to take" rather
//! than "something arrived". That is the difference between a state and an
//! edge, and only the state composes: `WaitForMultipleObjects` may report any
//! one of several signalled handles, so a waiter routinely learns about one
//! ready source while ignoring another. An auto-reset event consumed by that
//! wait would lose the second source's only edge. A level survives being
//! ignored, and will still be there on the next pass.
Mike Grier and others added 16 commits September 4, 2026 22:52
…nt and width

Adds three duplicated implementations of `reserving_mpsc`'s claim protocol --
32/32 and 16/48 over `AtomicU64`, 64/64 over `AtomicU128` -- and measures them
through `probe-queue-contention` in both regimes, so `D-37`'s shipping decision
has a number it currently lacks.

Built as a duplicated path in this crate rather than in `windows-waitable-queues`:
the measurement adds no third-party dependency to a publishable crate and cannot
disturb the branch being peeled off PR #56. `CW-1.6` owns merge-or-delete.

Re-apportioning the bits is free (16/48 tracks 32/32 within noise in both
regimes) and moves the SH-14.1 recurrence from 2^32 to 2^48. Widening the word
costs 2-3x isolated, growing with contention, but only 5-12% drained -- and the
drained figure understates it, because a slower producer earns fewer refusals
and refusal retries are inside the timed region.

The control caught a defect in the first run: the duplicates had not padded
`head` and the claim word onto separate cache lines, reporting 3.7x against the
shipping shape on a different scaling curve entirely.

Completed item: CW-1.1: Add portable-atomic and record whether AtomicU128 is
lock-free on this target
Completed item: CW-1.2: Implement the three claim-word layouts as self-contained
u64-item queues
Completed item: CW-1.3: Wire the three layouts into probe-queue-contention
Completed item: CW-1.4: Run the probe and capture the report
Completed item: CW-1.5: Record the measurement in DESIGN-NOTES.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… splits

The reservation half holds 2^32 where hundreds would do, and trading it away is what buys position bits: 2^12 reservations leaves over a year before recurrence, 2^8 leaves twenty years, against today's 37 seconds.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ed options

Records the engineer's direction that options are right if quality is maintained and ramifications are available, and binds both halves: each layout states its own reservation ceiling, capacity ceiling, and time-to-recurrence where it is named; quality is per-layout; the default is not weakened. Adds CW-2.4 for the documentation and CW-2.5 for the SH-14.1 disclosure sweep, since every existing statement of that hazard is scoped to a 32-bit position.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ation, not the layouts

The item read as though it might decide which layouts to offer. It does not: multiple layouts ship as options per M2. CW-1.6 is only about the private copy of the protocol in this crate, which M2 makes obsolete because the shipping type can then be instantiated at any layout.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…easurement falsified

D-36 deferred fixing SH-14.1 because the fix was believed to be the D-35 claim-protocol replacement, gated on an open question. Re-apportionment is a second fix neither D-36 nor D-37 considered, measured free by CW-1.4, moving the recurrence from ~37 seconds to ~20 years. M2 was also parked on not disturbing a branch under review; that branch has no PR open and the u64 layouts need no new dependency.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ied apart

Decoupling the reservation ceiling is numerically invisible at 32/32: MAX_RESERVED is 2^32-1 against a 2^31 capacity ceiling, so the cap can never bind and no test can reach it. It becomes observable only once a layout narrows the reservation half, so landing them separately would have committed a branch nothing could exercise.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Behaviour-preserving groundwork for CW-2.1's configurable apportionment: the position, head, and per-slot sequence become 64-bit, so a position wider than 32 bits can be represented at all. The split stays 32/32, so nothing observable changes and all 310 tests pass unmodified in substance.

The wrapping arithmetic is centralised in advance() and distance(). A position is now carried in a u64 but is only POSITION_BITS wide, so it wraps where the packing says rather than where the type does -- and an omitted mask is not a compile error, it is a position that escapes its half of the claim word. The Drop loop was the sharpest case: unmasked it would have walked past the wrap and never reached its terminating position.

The packing tests were rewritten against POSITION_MASK and MAX_RESERVED rather than u32's extremes, which would have silently stopped testing the edges once the apportionment changed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…electable

Offers `Balanced` (32/32, the default), `Enduring` (16/48), and `Perpetual`
(8/56) as compile-time choices, and decouples the reservation ceiling from the
capacity so that an asymmetric division is usable at all.

The claim word packs a reservation count and a position into one `u64` because
one compare-and-swap must update both. The 32/32 split was not forced by the
platform: it followed from requiring the count's half to hold the entire
capacity, since every slot could be reserved at once. Capping outstanding
reservations at the layout's own ceiling instead leaves the capacity bounded
only by the ring, which is what lets the position take 48 or 56 bits.

That is the breaking part, and it is a changed promise rather than a broken one:
`reserve` now returns `None` once `L::MAX_RESERVED` reservations are
outstanding, however empty the queue is. Under the default layout the ceiling is
2^32 against a 2^31 capacity, so it can never bind and nothing observable
changes -- which is also why this could not be verified without the layouts, and
why the two checklist items were merged during execution.

What it buys, measured by probe-queue-contention: a deeper position costs
nothing outside noise, because all three issue the same `lock cmpxchg` on the
same `u64` and differ only in shift and mask constants. The recurrence behind
SH-14.1 moves from 2^32 pushes to 2^48 or 2^56 -- roughly 37 seconds, 28 days,
and 20 years at the crate's disclosed sustained rate.

`ClaimLayout::VALID` is forced by `build` rather than left to be evaluated,
because an associated constant in a generic context is only checked where it is
used. Verified by sabotage: a layout with 31 position bits fails the build
naming the assertion, and a first attempt that appeared to pass turned out not
to have applied the edit at all.

Completed item: CW-2.1: Introduce the layout as a compile-time parameter, widen
the position to 64 bits, and decouple the reservation ceiling from the capacity

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e as a dependency question

core::arch::x86_64::cmpxchg16b is stable on the pinned toolchain, so a 64/64 layout needs no third-party crate and D-7's and D-37's dependency cost does not apply. The real costs are hand-written unsafe, x86-64 only, and a target-feature decision -- against a 2-3x measured cost on the claim and Perpetual already reaching ~20 years on a plain AtomicU64.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… layout

The claim word's integer type becomes part of the layout, so `reserving_mpsc`
can pack into a `u128` where a caller asks for it. `Wide` divides that word 64 /
64: the position needs 2^64 pushes to recur, which is a guarantee rather than an
argument about deployment lifetimes.

**Off by default, and that is the point.** Rust's standard library has no
128-bit atomic, so this is the only thing in the crate that costs a third-party
dependency. Without the feature the dependency tree is `windows-sys` alone and
every layout uses `AtomicU64`; with it, `portable-atomic` appears. Verified with
`cargo tree` in both configurations rather than assumed from the manifest.

`default-features = false` on the dependency is load-bearing: with its defaults,
`portable-atomic` silently substitutes a global lock where the native
instruction is unavailable, which would put a mutex in the claim path while
still compiling.

The word type is abstracted behind `ClaimWord` rather than by widening
everything to 128 bits, so a `u64` layout still issues `u64` instructions
exactly as before -- the arithmetic is monomorphised to each width. Positions
stay in a `u64` throughout, since no layout gives them more than 64 bits, and
the reservation ceiling stays capped at `u32::MAX` because the count is reported
to callers as a `u32`.

Two const assertions moved with it: the position must be narrower than its own
word rather than narrower than 64, and it may not exceed the `u64` it is carried
in.

314 tests pass by default, 319 with all features, including five that exercise
`Wide` through the whole protocol rather than only its constants -- the packing
at a maximal position, where a carry into the count would show, and delivery of
both a push and a reservation.

Completed item: CW-2.3: Decide whether a 128-bit claim word ships at all --
decided yes, behind an opt-in feature, so callers who do not want the dependency
do not carry it

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…W-1.6

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…perseding D-36

D-36 decided 0.1.0 would ship SH-14.1 disclosed rather than fixed, because the
only known fix was the D-35 claim-protocol replacement, gated on an open
question -- so disclosing beat delaying. That premise is false. Re-apportioning
the claim word is a second fix neither D-36 nor D-37 considered, and it costs
nothing measurable, so the hazard is now a number the caller sets rather than
one the crate imposes.

Recorded as D-41, which supersedes D-36. D-37 is partly superseded too: the wide
word ships as a layout behind the `dwcas` feature rather than as a separate
`reserving_mpsc_wide` shape gated by target, though its reasoning about
`portable-atomic`'s default features still stands.

Swept `2^32` / `32-bit position` / `sound below the wrap` across src/, tests and
*.md for the crate: 21 sites, 9 rewritten, 12 already correctly scoped or out of
scope. The out-of-scope ones are `slotwise_mpsc`'s note about `usize` on 32-bit
targets, which is a different subject, and the table rows and prose that now say
`Balanced` explicitly. `permit_mpsc`'s rationale needed care rather than a
scoping edit: a deeper position moves the recurrence out of reach without
removing the decision/operation separation that causes it, which is why that
shape remains interesting.

The three new examples are doctests, so a renamed layout breaks the build rather
than leaving the documentation teaching a name that no longer exists -- doctests
go from 9 to 12.

Two defects the CI rustdoc flags caught, both invisible to `cargo test`: a
public doc linking to the private `build`, and a link to `Wide` that resolves
only when `dwcas` is on. Docs now build clean under all three feature
configurations.

Completed item: CW-2.4: Document the layouts as a choice, in the crate
documentation and the README, with the rollover table
Completed item: CW-2.5: Reopen D-36 with the measurement in hand, then sweep
every statement of the hazard

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… duplicate

`windows-waitable-queues` takes the claim layout as a parameter now, so the
probe instantiates the real type at each of `Balanced`, `Enduring`, `Perpetual`
and `Wide` rather than carrying its own copy of the protocol. The duplicate
existed so the layouts could be compared before the shipping crate had them;
that reason is spent, and 501 lines go with it.

The direct `portable-atomic` dependency goes too. The probe now reaches a
128-bit word through the queue crate's `dwcas` feature, so there is one place
that decides how that atomic is configured instead of two that could disagree.

The two timing functions are generic over the layout where their `permit_mpsc`
neighbour is deliberately duplicated, and the difference is worth stating: that
twin compares two *different types*, which a generic could only unify behind a
trait, putting an indirection inside the timed region. These are the *same type*
at different layout parameters, so this monomorphises to what a hand-written
copy would produce.

**Re-measuring on the real type corrected the result.** The duplicate reported
the 128-bit exchange at 2.37x and 2.99x the default at sixteen and thirty-two
producers; the shipping type reports 3.83x and 3.99x. The stand-in was
understating the cost of the very layout it was built to evaluate, by the widest
margin exactly where the decision is most sensitive. The apportionment finding
survived unchanged -- both `u64` re-apportionments still track the default
within noise, so `Perpetual`'s twenty years of headroom really is free.

The 1.26x offset the duplicate carried is gone: running the same configuration
twice through the shipping type agrees within noise, 50.3 ns against 52.1 ns at
thirty-two producers, because both rows are now the same code.

Completed item: CW-1.6: Delete the duplicated implementation in
claim_layout.rs, keeping only what CW-2.3 leaves no other way to measure --
which, since Wide ships, is nothing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
All items complete, so the named-feature checklist moves to COMPLETED-CHECKLIST.md and its PLANS.md row moves to COMPLETED-PLANS.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…EADMEs

The queues README gained a Cargo features section -- dwcas was user-facing and undocumented, and it is the only thing in the crate that costs a third-party dependency, so a reader deciding whether to enable it needs the trade stated. The status summary now says the claim word's apportionment is a caller's choice rather than describing reserving_mpsc as though it had one fixed layout.

The probes README's 'what is measured' section gained the queue-contention entry, including that the probe instantiates the shipping type at each layout rather than a stand-in, and why.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The placement adapter currently hides retained topology grouping and efficiency-class disagreements, producing potentially inaccurate classifications.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 84/231 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment on lines +707 to +711
let core_id = core
.label_from(Source::RelationshipWalk)
.unwrap_or(index as u32);
for id in core.processors.iter() {
core_of.insert(id, core_id);
Comment on lines +713 to +721
let DomainKind::Core {
efficiency_class, ..
} = core.kind
else {
continue;
};
for id in core.processors.iter() {
class_of.insert(id, efficiency_class);
}
Comment on lines +69 to +71
[dependencies.portable-atomic]
version = "1.15.0"
default-features = false
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants