Repository navigation
Conversation
There was a problem hiding this comment.
Verdict: MUST_FIX: on macOS the host populate never reaches its read-a-byte-a-page fallback, because Unsupported is turned into a plain io::Error before os::populate_span sees it.
Must fix
1. host_lock::populate_span loses OsError::Unsupported, so a macOS load fails
host_lock.rs:889 converts os::advise_populate's Unsupported with .map_err(io::Error::from). host_lock.rs:909 then wraps it as OsError::Io. os/src/lib.rs:36 treats Io as "no advice" only when raw_os_error() == Some(EINVAL). The converted error has no errno, so the fallback is skipped.
Path: Mac, HostSet::populate (host_lock.rs:317) → populate_span → Err("madvise(MADV_POPULATE_READ): unsupported on this platform: …"). The load stops. The macOS NO_POPULATE_READ text (host_lock.rs:883) is never printed, and Populated::Touched is unreachable there. The PR body says the macOS body "reads the pages in instead". The unit test (host_lock.rs:1209) injects only EINVAL and ENOMEM, so no test turns red. Engram's populate (engram/src/lib.rs:695) passes OsError straight through and is fine.
Fix: keep the typed error through the seam.
fn advise_populate(span: &[u8]) -> Result<(), os::OsError> {
// SAFETY: `span` is a live slice (the caller's contract above), so its
// pages are inside one live mapping.
unsafe { os::advise_populate(span.as_ptr().cast_mut().cast(), span.len()) }
}fn populate_span(
span: &[u8],
page: u64,
advise: impl Fn(&[u8]) -> Result<(), os::OsError>,
) -> Result<Populated, String> {
// SAFETY: `span` is a live slice (the caller's contract above), so its
// pages are inside one live mapping.
unsafe {
os::populate_span(span.as_ptr().cast_mut().cast(), span.len(), page, || {
advise(span)
})
}
.map_err(|e| format!("madvise(MADV_POPULATE_READ): {e}"))
}In the test at host_lock.rs:1212 and :1218, wrap the errors in os::OsError::Io(..). Add one case that runs on every platform: populate_span(&span, page, |_: &[u8]| Err(os::OsError::Unsupported { call: "madvise(MADV_POPULATE_READ)" })) must be Ok(Populated::Touched). This case fails on the current code.
Non-blocking
⚠ Engram populate changed on Linux: engram/src/lib.rs:688. Before, a refused MADV_POPULATE_READ (EINVAL, kernel older than 5.14) was an EngramError. Now it reads a byte a page and returns Ok. The PR says the Linux bodies moved "unchanged". Say in the PR body that this is intended, or keep the error. The Populated value is dropped here, so the caller cannot tell the two paths apart.
⚠ Three new tests only assert is_ok() or > 0: a_files_bytes_are_written_out (os/src/lib.rs:819), the_page_cache_drops_a_files_clean_pages (:793), the_free_bytes_of_a_directory_are_counted (:784). A sync_range or fadvise_dontneed that returns Ok(()) without the syscall passes. So does a free_bytes that drops * f_frsize. Assert free_bytes against a libc::statvfs product computed in the test.
⚠ The PR says no other crate names a Linux-only libc item, but engram::faults_of keeps its own RUSAGE_THREAD = 1 (engram/src/lib.rs:923) beside os::thread_faults. The shapes differ, so this is a question.
Checked
- Duplicates: 31 new items checked. The dryer pair (os/src/lib.rs:366 and :384) is read: the two bodies are cfg alternatives, and a shared helper would add elements, so it is not raised. The other cfg-paired functions share a shape on purpose.
- Complexity: crapper flagged nothing.
- Silent failure: 9 sites followed.
faults_ofnow asserts thegetrusageresult (good).thread_faultsasserts on Linux and returnsUnsupportedon macOS.pin_toandfloat_everywheremap errors toboolas before. The only lost error is item 1. - Tests: mutation check on the populate test. Planting "
Unsupportedis not no-advice" turns nothing red (item 1). Mask, pin and rename tests turn red on their plausible mutants. - Gates: no gate value moved. The
PIN(2026-10-21)lines forbloomery-os, the ratchet and the lists carry a reason. The unsafe counts for engram, gguf, model and threads match the moved blocks. - Hot path: none (load time and tools), so no performance term and no headroom items.
- Escape catalog: none matched (no shell change).
Notes
- The PR body has no
nvidia-smiline. The change touches no kernel, andcanon-checkis the proof asked for a move.
🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)
Native compile census on aarch64-apple-darwin, the Metal backend design aimed at the M5 Ultra, the CPU path cut to what M0 and M2 need, gates on a Mac, and the phased plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… macOS (issue #593) bloomery-os holds every Linux-only libc call the host chain made (affinity, per-thread rusage, O_DIRECT, posix_fadvise, sync_file_range, MADV_POPULATE_READ, mincore, statvfs, renameat2), each Linux body moved unchanged beside its macOS body or a named OsError::Unsupported. threads, gguf, engram and model call it. Tests that read Linux-only state are ignored on macOS by name. engram's fault counter now refuses by name when getrusage does instead of reading zeros. tools/recipes.py reads the owner's cfg fence, so gguf and bloomery-os join the Mac's pure crates; its self-test holds that a Linux-only item outside the fence is still rejected. Native cargo check of the eight host crates on aarch64-apple-darwin, and the same as cross checks for x86_64- and aarch64-unknown-linux-gnu: rc 0. mac-check test: 16 crates, 1577 passed, 0 failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hout a local path Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…m threads, gguf and model Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es test checks blocks times size The populate seam turned os::advise_populate's error into an io::Error before os::populate_span read it, so OsError::Unsupported lost its variant and a macOS load stopped instead of reading a byte a page. The seam now passes OsError through; the test adds the Unsupported case (red on the old seam). free_bytes' test now fails a count of blocks without their size. The hip-native lab's lock gains bloomery-os (contrib-check t0, --locked). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n with the Linux case orphan-tests requires a gate-* recipe to run every test. gate-os runs the crate's lib tests (class row common/host, owner os_calls). A test fenced to macOS runs on no gate, so each named refusal now sits in the same fn as its call's Linux case, one cfg block each, as floating_widens_the_mask already did. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Since the last review:
- Fixed: the populate seam preserves
OsError::Unsupportedat host_lock.rs:889. - Answered: the PR body explicitly identifies engram's older-Linux fallback as intentional.
- Still open: the weak-test advisory is partly addressed at os/src/lib.rs:789; the remaining gaps are below.
- Answered: the PR body explains that engram retains separate major/minor fault counts and defers moving that pair.
Verdict: CLEAN: the prior blocker is fixed, and the two added commits introduce no must-fix defect.
Non-blocking
⚠ The byte-count assertion still accepts blocks: os/src/lib.rs:789. Dropping * f_frsize still passes whenever f_bavail is a positive multiple of 512. Compare against the statvfs product to prevent a unit regression; divisibility alone does not establish bytes.
⚠ The syscall tests still accept no-ops: os/src/lib.rs:807 and os/src/lib.rs:831. Replacing Linux fadvise_dontneed or sync_range with Ok(()) passes. Add invalid-descriptor refusal assertions inside these tests so deleting the syscall turns them red.
Checked
- Duplicates: 0 new Rust items; the new recipe and owner row have no existing equivalent. The dryer pair at os:366/384 contains platform alternatives; no new duplicate.
- Complexity: crapper flagged no function.
- Silent failure: 2 populate seams followed. Unsupported and EINVAL reach page touching; ENOMEM reaches the caller as a named error. Engram preserves the typed error too.
- Tests, static mutation check: restoring the lossy conversion at host_lock:910 fails the new assertion at :1225. Returning success from macOS refusal bodies at os:198/218/268/420 fails the four merged tests. Linux constant-zero faults at os:258 still pass the monotonic check; the no-op and unit mutants above also survive.
- Gates: no threshold moved. The merged tests retain both platforms' cases; gate-os uses the bounded runner and preserves its exit status.
- Adversarial checks: Unsupported, ENOMEM and an empty span follow the intended branches; an empty fallback span performs no read.
- Hot path: none in the added commits; populate is load-time work.
- Escape catalog: C1–C4 do not match the added code.
Notes
Validation here was static; no build or gate was executed. The supplied evidence lacks a complete NVIDIA gate-run block.
🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)
What and why
First step of #593 (bloomery on Apple Silicon). The host chain now builds natively on aarch64-apple-darwin. A new leaf crate,
bloomery-os, owns every Linux-onlylibccall the host crates made. Each Linux body moved over unchanged, and each has a macOS body beside it or returns a namedOsError::Unsupported. One Linux behaviour changes on purpose: engram's populate on a kernel withoutMADV_POPULATE_READ(EINVAL, before Linux 5.14) now reads one byte a page, as the host-lock populate already did. It used to return an error. The pages end up resident either way. The PR also addsdocs/contrib/mac.md, the port map and Metal design aimed at the M5 Ultra.Release notes: Under the hood — the host crates build on macOS (Apple Silicon); the Metal backend is not in yet.
Code shape
crates/osis the one owner. threads, gguf, engram and model call it. One Linux-only item stays outside it:engram::faults_ofkeeps its ownRUSAGE_THREADbecause it needs major and minor faults apart, andos::thread_faultsreturns their sum. Moving that pair intobloomery-osis a follow-up.Unsupported { call }.engram::faults_ofused to ignore agetrusagerefusal and read zeros; it now panics by name.tools/check-comments.shrc 0.bloomery-oscarryPIN(2026-10-21):.cargo fmtis clean.cargo clippyon the touched crates for x86_64-unknown-linux-gnu: 0 warnings. On aarch64-apple-darwin, one unused-import warning incrates/qdot/src/lib.rs:71comes from main (f956b65ff), not from this change.Duplication found or removed: none.
What I ran
just canon-check 3b1236e80→canon-check: 3 bins, 990 entries: 990 identical, 0 reordered-only, 0 differs, 0 only-on-one-side. No kernel moved.tools/gate-batch.sh --list … --tier fixture --ledgerwith gate-threads, gate-engram, gate-ops, gate-1-1, gate-r8, gate-model-hf and gate-qdot →DONE total=7 red=0.gate-osadded (the crate's lib tests;orphan-testsrequires a recipe). The 7 gates above plus gate-os →DONE total=8 red=0. The populate seam keepsOsErrortyped; its newUnsupportedcase is red on the old seam and green now. Thefree_bytestest is red on a mutant that dropsf_frsize.cargo checkof os, threads, gguf, qdot, engram, engram-lab, model and model-hf with--all-targets: rc 0 native, rc 0 for x86_64-unknown-linux-gnu, rc 0 for aarch64-unknown-linux-gnu.tools/mac-check.sh test: 17 crates, 1594 passed, 0 failed.python3 tools/recipes.py --self-test: ok (208 gate recipes).tools/check-unsafe.sh: ok.bloomery-osis seeded at 24 blocks and 5 fns with aPIN(2026-10-21):line, and the pins of threads, gguf and model come down by the blocks that moved.cargo test -p bloomery-model --lib, 413 passed and 10 failed. The causes are next in M0b: the 16 KiB page refused byhost_available_read(6),/procreads (2), and a refusedposix_fadvisethat the r8 and host-lock callers treat as fatal (2).Hardware (if the change is for a machine)
Apple M1 Pro, macOS 27, for the native build and tests. The gates ran on the maintainers' workstation.
🤖 Generated with Claude Code