Repository navigation
SEP 46: calibrating the native backend to its machine - #582
Merged
Merged
Conversation
…nto feature/sep-0046-machine-calibration
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
jtravs
added
performance
Documentation build overview
60 files changed ·
|
The operation thresholds, medium and compact widths, compact bound and stable-team rule leave core.hpp. Each workspace now copies a CpuExecution record from its problem and never reads them from anywhere global, so two live workspaces may differ. The values come from solprop_native's built-in table (_execution.py), keyed by operating system, with ADR-073's division on Apple silicon's spinning top tier folded in; the __APPLE__ switches and the divisor leave the native source. A state reads back the record it applied (State.cpu_execution). Without a profile every workspace gets exactly the former constants, frozen per platform in test_execution.py. One edge moves: a user FFT start equal to the default is no longer divided on a spinning Mac. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
- _store.py: the per-user store in the platform's cache directory (SOLPROP_CACHE_DIR), generations published whole and named by an atomic swap of `current`, exclusive-create claims per allocation class with stale-claim removal, inconclusive attempts, and read-only SOLPROP_CALIBRATION_PATH directories. - _topology.py and _allocation.py: cores and their kinds, shared L2, L3, NUMA and socket groups and SMT from sysfs, sysctl or the Windows API; allocation classes, matched exactly or by the nearest smaller class of the same span structure (workspace settings only). - _fingerprint.py and _build.json: the machine, the system, the build's configuration (written beside the extension) and a digest of the chosen extension, read without loading it. - _profile.py: the row a process takes. At native import an exact row sets the pool width and the wait unless the user did; each workspace takes the row's workspace settings only where they were fitted under the process settings in force. A run's metadata records the profile and the settings applied, read back from the state, beside the table's. - solprop.calibration and `solprop calibrate --show/--export/--import/ --clear`, which load those modules by path and never start the native runtime. The test suites run with SOLPROP_CALIBRATION=off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
The solprop static job checks without solprop-native installed, so the calibration package's type-only native imports carry the registry's reportMissingImports ignore. The native gate's strict pyright refuses redefined constants, an unknown argument and loosely typed JSON. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
A calibration's children, and a scan's workers in phase 5, ask the native import for a wait. A user's wait variables still win; the request replaces a calibration's and the default rule's choice. (The previous commit's import of requested_wait needs this one.) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
- calibration/selection.py: section 6.5's rule, pure: interleaved pairs, the median ratio as score, A/A noise at least 2 %, admission only when a candidate wins every round by more than the noise, at most four candidates, combined validation by window, and the drop order. - calibration/child.py: one candidate's process settings (width and a requested wait), pinned before the runtime starts, timing the trial steps of the calibration problems over JSON lines. - calibration/problems.py: five synthetic problems that solprop ships (a Raman GNLSE, a four-mode helium capillary with plasma, an averaged hydrogen capillary, a radial argon filament, a free-space line). - calibration/controller.py: one monotonic deadline per level, the built-in child timed twice a round for the A/A noise, host load and steal, validation of a winner on fresh rounds, publication that keeps other classes' and higher levels' rows, nested allocations (one socket, NUMA node, L3 group) at standard and thorough, and every child reaped. `solprop calibrate [--quick|--thorough] [--budget S]` and solprop.calibration.calibrate(). On the Xeon (one socket, 18 cores) a standard calibration takes 23 s and keeps the built-in team: passive waiting is 1.26-1.32 times slower and half the socket level. Across both sockets the A/A noise is 10-23 %, so that allocation is reported inconclusive and only its sockets get rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
`--baseline-env` and `--candidate-env NAME=VALUE` set an arm's own variables over the threading preset, so that SOLPROP_CALIBRATION=off against =use compares the built-in table with a machine calibration in one interpreter. The plan records them, and the A/A note holds only when they agree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
After the process settings are chosen, the grain is fitted in that process's child. Its workspaces' execution_sizing gives each timed operation class's work sizes; a class's candidates are the one-step moves of its threshold (and, at thorough, its full size) past the nearest of them, at most four, with a medium width of 2 or 8 and a compact team of 4 or 16 beside them. Each is timed only on the problems whose widths it changes, interleaved with the child's own record, and admitted by section 6.5's rule; the best of each setting are validated together on fresh rounds of every window, and dropped together if that fails. The row's workspace settings are the admitted changes, fitted under the chosen process. Quick tries the medium width and compact team only. The Adams trial's passes and a coupled flow run in no trial-step window, so their thresholds keep the table's. On one socket of the Xeon (18 cores) a standard calibration takes 36 s and admits a medium width of 8 and a compact team of 4: the windows' geometric mean 0.885 (line 0.71, radial 0.81); every stage, FFT and response threshold move lost by 2-126 %. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
…test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
Native: a workspace plans its host transforms under an FftPlanning, set while it is built (FftPlanningScope). The policy (legacy) is unchanged. A profile forgets FFTW's wisdom before planning and after, imports each plan's own wisdom by its key (kind, sizes, distances, placement, alignment, threads) and seeds it with FFTW_WISDOM_ONLY at its group's effort (serial rows, threaded rows, transverse planes); a plan that no longer builds is recorded and planned at estimate, and a shape it does not hold is seeded at its group's effort. A capture seeds each transform alone, from no wisdom, within a time limit, and keeps its wisdom. State.fft_planning reads back the groups, the captures and the misses. Calibration: after the grain, the chosen child captures every transform at measure (and patient at thorough), within 1 s a plan at standard and 20 s at thorough. The policy's efforts with that wisdom are the base; each group's other effort (serial rows at estimate or patient, threaded rows at measure, planes at estimate) is timed in the step on the problems holding it, admitted by section 6.5's rule and validated together. The row keeps the efforts and the generation the wisdom; a run applies them where the row's workspace settings hold, and never beside the user's SOLPROP_FFTW_PLAN or SOLPROP_FFTW_WISDOM. Quick keeps the policy. The busy check also reads the allocation's SMT siblings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
Simulation.calibrate(budget) and `solprop calibrate --recipe FILE` build fresh native workspaces of the simulation's translated problem in the calling process, under the process's record and under each grain candidate of the simulation's own work sizes, and time trial steps of its own operators at its start and a third and two thirds along it (a low-storage march times its own step). Speed alone never admits a candidate: every candidate's trial steps must end finite and within ten times the run's relative tolerance of the base's. The winners are validated together and stored under the simulation's execution descriptor: its shape, grid, integrator, every operation it runs with its work, the process settings and the fingerprint. A later backend of an equal descriptor (a rerun, a scan point) rebuilds its workspace with the refined record, while the machine record it refined still holds, and records it as its workspace source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
- The first native workspace of a process (registry.select) in an allocation with no row runs the quick calibration after a CalibrationNotice (a warning, and a log record), under SOLPROP_CALIBRATION=auto; once per process, never in a scan's worker or a device build, not again within a day of an inconclusive attempt, and not while another process on this host holds the claim, for which it waits. A process that has not loaded native yet then takes the whole profile; one that has refreshes its workspace settings and FFT planning under its process settings (solprop_native.refresh_profile), and the backend reads the profile's state from the package at each use. - Pool calibrates its workers' allocation at the standard level before the probe worker, in children pinned like one worker, unless a standard or thorough row matches. Its workers are marked (SOLPROP_CALIBRATION_ROLE=worker) and ask for passive waiting through SOLPROP_NATIVE_WAIT_REQUEST instead of setting OMP_WAIT_POLICY, so that a user's own wait, then the workers' row, decide; their SOLPROP_THREADS is the partition within which the row may narrow the pool. The Slurm script asks the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
`track` and `ab` calibrate each build before the series, once per interpreter and with its arm's variables (`--calibrate`, thorough by default; none, or an arm under SOLPROP_CALIBRATION=off, calibrates nothing). Their workers run SOLPROP_CALIBRATION=use, or off, and never measure inside a timed window. Each row records the profile its run applied (`calibration`: built-in, or its level and generation, with +refinement), a new history column. The docs build and the galleries run with the calibration off, so that their committed output shows no first-use notice and does not depend on the machine. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
…mes) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
- docs/guide/performance/calibration.md, first in the performance section: what is measured at each level, where profiles live, the first-use notice and SOLPROP_CALIBRATION, scans and clusters, reading --show, refining one simulation, and the overrides. - Installation, the README and the source-build guide end with `solprop calibrate`; the tuning guide opens with it and keeps the environment variables as overrides; the backend record's `calibration` entry is explained. - cpu-performance.md (the table's new home), benchmarking.md (series run calibrated; per-arm environments), ADR-074 (store, first use, precedence), and the release notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
…es; a CI test on one CPU Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
…of an A/B (#361) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
A thorough calibration on 18 cores of an Intel Xeon Gold 6240 spent its whole budget in the grain stage, leaving the FFT stage none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
…ren take it Calibration children now start with solprop._process.child_environment, so NumPy's and SciPy's BLAS threads go idle at once rather than spin on the CPUs being timed. The ledger keeps both branches' new sections. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
jtravs
marked this pull request as ready for review
November 8, 2026 22:50
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bf7f26179f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| fingerprint=plan.digest, | ||
| allocation=allocation.record(), | ||
| ) | ||
| claim = plan.store.claim(plan.digest, allocation.key(), plan.deadline.epoch()) |
There was a problem hiding this comment.
With the default SOLPROP_CALIBRATION=auto, a first native run fails if the cache directory is read-only or cannot be created. Store.claim() creates directories and opens a claim file, but this call is outside the try block, and neither first_use.ensure() nor backend selection catches its OSError. Consequently, an optional calibration prevents an otherwise valid simulation from starting, contrary to SEP 46 §9's stated built-in-table fallback. Handle store write failures on the automatic path, including publication failures, by warning and continuing without calibration.
Useful? React with 👍 / 👎.
Comment on lines
+1514
to
+1517
| builtin = _Setup(record=record, planning=None) | ||
| if timed: | ||
| plan.say(f"timing {len(timed)} FFT planning candidates") | ||
| noise, verdicts = _session_of(plan, child, builtin, timed, until, 4000) |
There was a problem hiding this comment.
At the standard and thorough levels, this baseline does not use the default planning policy described above it: Candidate.environment() forces SOLPROP_FFTW_PLAN=estimate, and planning=None reaches fftw_seed_effort(), which therefore selects estimate for every transform. Normal unprofiled runs instead measure serial rows and transverse planes. Both candidate admission and combined validation consequently compare against the wrong baseline; for example, the planes estimate candidate is compared against an already-estimated plane rather than the measured plane it would replace. Construct a baseline that explicitly exercises the real default policy before accepting and publishing planning changes.
Useful? React with 👍 / 👎.
| smt = len(held) > len(cores) | ||
| return AllocationClass( | ||
| cores=len(cores), | ||
| kinds=tuple(len(kinds[kind]) for kind in sorted(kinds)), |
There was a problem hiding this comment.
On a hybrid CPU, enumerating only the kinds present in the allocation loses their identities: a performance-only allocation and an efficiency-only allocation both produce kinds=(N,). When their other span counts agree (for example, single-core masks), match() treats them as exact matches and applies the same process width, waiting policy and workspace settings despite their different core kinds; publishing one also replaces the other's row. Preserve counts for every machine-wide kind, including zero counts, so these allocations have distinct keys.
Useful? React with 👍 / 👎.
Comment on lines
+385
to
+390
| for directory in directories: | ||
| for path in (directory / "current", *directory.glob("*.json")): | ||
| with contextlib.suppress(FileNotFoundError): | ||
| path.unlink() | ||
| removed += 1 | ||
| return removed |
There was a problem hiding this comment.
solprop calibrate --clear only removes files under machines/, leaving stage-2 profiles under problems/<fingerprint>/ intact. Those refinements can still apply after clearing: _refined_record() requires a fingerprint and a matching base execution record, not an existing machine generation. Thus a refinement fitted over the built-in record remains active under SOLPROP_CALIBRATION=use even after the user explicitly forgets this machine's calibration. Clear the corresponding refinement directory as well, while preserving live claims.
Useful? React with 👍 / 👎.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
…ing refinements - First use: a calibration store that cannot be written (read-only, missing, full) leaves the run on the built-in table with one warning, instead of failing it. - The FFT stage's base is the plan policy's efforts planned in place with no stored wisdom; the children plan at estimate otherwise, so "no planning" was an all-estimate base, not the policy a run uses. - Allocation classes count every kind of core the machine has, held or not, and the span says which are held: two fast cores and two slow ones are different classes. - `solprop calibrate --clear` also forgets the machine's stage-2 refinements, which would otherwise still apply over the table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
This was referenced Nov 8, 2026
…rated An RDW tuning scan (four-mode helium capillary over pressure and energy) on a whole node, at 128 x 1, 4 x 4 and 8 x 16 workers by threads, against the table and spinning workers. The workers' rows kept the table, and spinning or passive workers ran the same within the node's spread; a slower calibrated arm at 8 x 16 did not repeat in nine further scans. A one-core allocation's standard calibration ran out of its 90 s before three rounds in series and published nothing, to try again a day later, though one CPU has no team, waiting or grain to choose. A one-CPU allocation is now never measured, implicitly or by `solprop calibrate`, so a single-threaded scan neither waits nor warns. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
…ws first_use The controller's tests classify eight and four CPUs as different allocations, which a four-CPU runner does not: they now run on a sixteen-CPU topology of their own. The layers figure gains `calibration/first_use`, the exception test roots `CalibrationNotice` at `SolpropWarning`, and the front end's tests turn calibration off as solprop's own do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
This was referenced Nov 9, 2026
Every phase is implemented and reported, and the verification, scan-worker and phase 6 reports cover section 7. The SEP moves to the archive with its index row and toctree entry; the roadmap drops it; links follow it. The open questions go to #591-#595, beside #229, #314, #361, #584 and #587. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
jtravs
changed the title
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
- The scripted controller fixture's fake topology leaked into `_profile`, which binds `read_topology` at import: when a controller test first imported it, later first-use tests classified a sixteen-CPU machine and missed their store entries. Patch both names. - The short-budget test relied on setup outlasting 1 ms; with a fake topology it no longer did. A reserve of the whole budget now leaves no round, however fast. - The pinning test skipped on macOS and Windows, which check_skips refuses. Windows now checks its process affinity mask, macOS that pinning leaves it alone; the Windows pin declares its pointer-wide argument types on a private WinDLL. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
…udget Coverage requires every line: three branches of the calibration had no test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR implements SEP 46 and sets it Final (tracking issue #579). All phases are on this branch, as SEP 0 asks, and each phase's report is in the SEP's Part II.
What it does
solprop calibratefits the native backend's CPU execution to the machine it runs on. Children pinned to the job's CPUs measure:A setting replaces the built-in value only if:
The profile is kept per user, keyed by machine, build and allocation. Every run applies it and records it in the run's metadata. A first run calibrates quickly by itself, after a notice. A scan calibrates its workers' allocation first, except single-threaded workers, which have nothing to choose. Without a profile, every run uses exactly the former settings, now a built-in table.
Status
CpuExecutionreplaces thecore.hppconstants.Simulation.calibrate,--recipe).docs/guide/performance/calibration.md, ADR-074 and the release notes.ctc0.87). It chose the Axion's grain.fft_size="auto"'s cost model is one fit from Intel and Apple CPUs #229, One-thread-worker scans have no measurement on native #314, The node chunk is sized at translation, whatever the device, and each chunk is a kernel launch #361, Native CPU: a stage's forward transforms as one batched region (#560 item 4) #584 and Native CPU: transforms' width by rows and row length, not element count #587.Decisions (maintainer)
Closes #579
Closes #328
Closes #332
Related, staying open: #81, #329, #303, #314, #333, #136, #361, #366, #229, #583, #584, #587, #591, #592, #593, #594, #595.
🤖 Generated with Claude Code
https://claude.ai/code/session_01XGENPvJBFJdUL6p1LDpss5