gnlse_raman_shock scales poorly with threads: slower at 16 than at 1 on an EPYC 9B45, and varies between processes #81

What happens

gnlse_raman_shock (one mode, 16384 samples, 8469 accepted steps, Runge-Kutta) on the native backend, in the 0.1.0 record (benchmarks/history/):

On the M5 Max at 18 threads, two collections on the same build gave 31.31 s (31.20 to 31.38) and 20.96 s (20.95 to 20.96). Each mode holds for all three repeats and for its process's warm-up (0.287 s against 0.187 s), so it is fixed per process. Only the 20.96 s row is in the CSV.

It is not only the M5 Max. A sweep on an idle 2x Xeon Gold 6240, on the released wheel, unpinned, one run per process, shows no gain beyond the spread past four threads, and a spread between processes as large as what scaling gains. From 8 threads up every column runs the same team of 8, so those columns differ only by process:

Why it matters

The scalar GNLSE with Raman is a common run. At the default width it is, at best, modestly faster than one thread, on one platform slower, and its time varies between processes.

Context

• cpu_compact_threads: Native performance push #59's compact team caps a one-column problem at 8 threads. These numbers are after that change. benchmarks.md, "Compact CPU teams" holds the earlier sweep.
• Roadmap, "The thread budget on chips with efficiency cores": the macOS budget counts every core, which may bear on the two modes there.
• docs/dev/benchmarking.md: how the record is taken. To repeat a width: python -m benchmarks track --case gnlse_raman_shock --threads 8 --production 16.

To do

• Repeat the sweep on an idle EPYC 9B45 (Google Cloud c4d-highcpu-32, SMT off) and M5 Max, or say what you ran on so that a maintainer can repeat the rest.
• Find where the threads go: FFT, stage passes or thread wake-ups.
• Find what is fixed per process: repeat one width many times with SOLPROP_FFTW_PLAN=measure and with pinned threads. Thread placement is the first suspect: threaded rows are at estimate by default, so they have no measured plan to vary, and the one-thread column is the stable one.
• Needs a decision: propose here whether the cap of 8 should be lower or per machine, and whether the macOS budget (ADR-056) should count only the fastest cores. A maintainer decides.

Done when

The cause of the poor scaling and of the variation between processes is recorded here with measurements. Then either a 16-thread run is no slower than a one-thread run on the EPYC 9B45, or the reason it cannot be is stated and the default width changed to suit.

Checked against commit 9d5d92f; the code links are pinned to it.

Also

Two more threading questions from the native Adams work, both on mode-averaged cases. This issue so far covers gnlse_raman_shock under Runge-Kutta.

The same pattern on mode-averaged Adams cases. SEP 39's phase 0 saw he_rdw_averaged and h2_raman_averaged slower at sixteen threads than at one on a shared 16-core domain of an AMD EPYC 7763, and said to re-measure before calling it a defect. The 0.1.0 record answers part of that. Native, default stepper (Multistep, exponential), median of three, one thread / all threads, read from benchmarks/history/:

So the EPYC 7763 is faster at full width in this record. The 2x Xeon Gold 6240 is slower on both, outside the spread of the three repeats. Windows' h2_raman_averaged is level within that spread, as is its argon_blueshift_averaged (2.62 / 2.67 s). These are compact problems, so the cap of 8 applies by reading.

An unexplained slowdown on a host march. SEP 39's phase 3 and benchmarks.md record that on an Apple M5 Max at sixteen threads (October 2026) h2_raman_averaged took 2.24 s on the Python march with the staged trial, in all three rounds, against 1.74 s with host packing and 1.48 s natively. The SEP suspects the OpenMP workers' idle policy meeting the shorter host gaps between native calls, and it was not investigated. The native march, the default, does not show it, but the configurations that step solprop's march with a HostMarchWarning may. Whether it shares a cause with the two M5 Max modes above is unknown; it read alike in all three rounds. Read, not re-timed.

• Add he_rdw_averaged and h2_raman_averaged to the thread-scaling runs, on a 2x Xeon Gold 6240 above all.
• Re-time the host march on an idle M5 Max with OMP_WAIT_POLICY varied, then explain it or close it here as not reproduced. If the idle policy is the cause, fix it on the host march or record the cost beside HostMarchWarning.

Added on 2026-11-01, checked against commit 8a9f764.