Skip to content

cpu: report FP results as the MIPS FPU does - #406

Merged
jamesbraid merged 3 commits into
techomancer:mainfrom
jamesbraid:ide-fpu-fix
Nov 8, 2026
Merged

jamesbraid merged 3 commits into
techomancer:mainfrom
jamesbraid:ide-fpu-fix

Conversation

@jamesbraid

@jamesbraid jamesbraid commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

The FPU handlers returned host results where the R4400, R5000 and R10000 differ, and compiled code and the interpreter disagreed in places. This makes both follow the hardware, and removes a slow path in compiled FP code.

Fixes

  • NaN encoding: the original MIPS encoding (fraction MSB set means signalling), and the default NaN 0x7FBFFFFF, in the interpreter and in compiled code (NEC VR10000 manual, table 2-2).
  • Invalid and Overflow are reported as the hardware does: inf-inf, 0*inf, 0/0 and inf/inf deliver the default NaN with Invalid, and overflow sets Inexact. The Octane PROM's ide FPU test needs this.
  • Denormal results are flushed to a signed zero when FCSR.FS is set. Compiled code used to trap with Unimplemented Operation for every denormal result.

Testing

  • cargo test --features jitv2 --lib passes on every commit (release) and on the tip in release and debug.
  • New tests compare compiled code against the interpreter for infinite operands, denormal results with FS set and clear, NaN operands and compare predicates.

Performance

Real clock, guest MIPS (higher is faster), median of 10 interleaved runs each on one pinned CCD. A positive change means this branch is faster:

kernel main here change
fpu/whetstone 418.3 432.8 +3.5%
fpu/transcend 313.2 383.3 +22.4%
fpu/linpack 596.7 641.7 +7.5%
fpu/matmul 577.3 631.2 +9.3%
int/alu (control) 450.1 451.4 +0.3%
mem/copy (control) 545.6 546.7 +0.2%

The FP kernels are significant (p < 0.01) and the controls do not move. Main's whetstone and transcend also fall into a slow mode in some runs; this branch does not. Host-instruction counts in deterministic mode fall far more (whetstone -45%), but that counts JIT compile work that normally runs on other threads, so wall time is the better measure.

The R4400, R5000 and R10000 use the original MIPS NaN encoding: a NaN
with the fraction MSB set is signalling, one with it clear is quiet. The
default NaN is 0x7FBFFFFF for that reason. iris used the IEEE 754-2008
convention, the reverse, in the interpreter and in compiled code.

The evidence for the original encoding is the NEC VR10000 instruction
manual, U12754E, first edition, August 2000, printed page 226, table
2-2, and the results of iris's own R4400 in cpu-tests: fpu/qnan_operand
feeds 0x7FC00000 to ADD and gets Invalid and the default NaN, not
Unimplemented, and the signalling-predicate compares agree. Upstream
issue 361 asked about it.

Swap the sense of the fraction-MSB test in is_snan and is_qnan and in
their compiled twins, including the compare path, and make the default
NaN 0x7FBFFFFF and 0x7FF7FFFF_FFFFFFFF. With the bit set the fraction
is nonzero by construction, so the interpreter's signalling test
shrinks to one mask and compare, and an ordinary operand skips the
subnormal and quiet-NaN checks after one exponent test. Compiled code
swaps a comparison and costs the same.

The tests that fix a NaN's bit pattern move with this commit. The new
ones cover the default NaN as an operand (Unimplemented, no Invalid), a
signalling NaN operand (Invalid, trapped and untrapped), and a
compiled-versus-interpreter check over ADD, MUL, DIV, SQRT and the
quiet and signalling compare predicates. Without this commit the rest
of the series still builds and passes its tests; the Octane PROM's
ide test then rejects the default NaN's bit pattern.
@jamesbraid
jamesbraid force-pushed the ide-fpu-fix branch 2 times, most recently from ad267e6 to c968e9c Compare November 2, 2026 15:47
@jamesbraid
jamesbraid marked this pull request as draft November 2, 2026 16:29
@jamesbraid
jamesbraid force-pushed the ide-fpu-fix branch 2 times, most recently from 4474a7f to 4bce574 Compare November 2, 2026 18:54
The Octane PROM's ide FPU test checks results the host FPU gets wrong
for MIPS: infinity minus infinity, zero times infinity, 0/0 and
infinity/infinity are Invalid Operations that deliver the default NaN
(0x7FBFFFFF), 0/0 is not a divide by zero, and MAX+MAX is Overflow with
Inexact. iris returned the host's NaN with no flags. 12 of the 14
subtests passed.

Classify those results on every CPU model, in both engines, at no cost
to an ordinary result.

ADD, SUB, MUL and DIV test the result's magnitude once, in the same
branch as a check that neither operand has a zero exponent. That test
replaces the old operand and denormal-result tests, and the top of the
range, which is where Invalid and Overflow live, goes to a cold
classifier shared by both engines. The operand flags those ops used to compute up front are
gone: an ordinary result proves there were none, and the classifier
finds a divide by zero itself.

Compiled code reaches the classifier through one call per instruction.
The same call takes a denormal, infinite or NaN operand and a denormal
result, where the interpreter's operand checks and denormal rule run, so
the instruction carries one call and one trap check, not a copy of those
rules at each of two exits. Zero, as an operand or a result, is common
in real code and is written without the call. Generated code for the page holding
Whetstone and its transcendental kernel comes to 245 bytes per guest
instruction, against 400 before this series.

SQRT, RECIP, RSQRT, the multiply-add family and CVT.S.D swap the host's
NaN for the default NaN and add V. They do it the same way round: one compare of the result's
magnitude, and only an infinity or a NaN goes on to classify the
operands. In compiled RECIP, RSQRT and the multiply-add family the
compare shares the branch these ops already took on a stale Cause, so the
ordinary path gains no branch. A finite result cannot come from a NaN operand, a zero
divisor or a negative square root, so the signalling-NaN tests on up to
three operands no longer run for ordinary arithmetic. CVT.S.D checks its
source for a signalling NaN only after the conversion has already left
the exact path, since a NaN is never equal to itself.

Host instructions per FP op for this and the next commit together,
perf stat instructions:u, ordinary operands, ship profile, against
the commit before this series (slope
over a 120-op region, three alternating rounds, counts do not move
between rounds). R10000 numbers; R5000 and R4400 differ by a constant:
  compiled add.d, mul.d, div.d                    36 -> 32
  compiled sqrt.s, cvt.s.d                        no change
  interpreter add.d, mul.d, div.d                 197-200 -> 171
  interpreter madd/msub/nmadd/nmsub               168-181 -> 153-159
  interpreter recip.s, rsqrt.s                    145 -> 136, 153 -> 137
  interpreter sqrt.s, cvt.s.d                     no change
Every measured op, in both engines, on the R4400, R5000 and R10000, is
within 0.1 instruction of the old count or cheaper. The compiled
madd/msub/nmadd/nmsub, recip and rsqrt were recounted from the generated
code of a 120-op run of each (R10000, host instructions on the ordinary
path, before -> after): madd.d 52 -> 24, recip.s 39 -> 24, rsqrt.s
48 -> 27, with the same three conditional branches per op as before.

The tests that check results run on the ordinary passthrough models in
both engines, and compiled code is checked against the interpreter for
each rule. This commit delivers the IEEE 754-2008 default NaN, 0x7FC00000. The
Octane's R10000 delivers 0x7FBFFFFF; the next commit moves the NaN
encoding, and with it the ide test's NaN checks pass.
With FS set and neither Underflow nor Inexact enabled, an R10000
replaces a denormal ADD/SUB/MUL/DIV/SQRT result with a signed zero and
sets Cause and Flag U|I. The interpreter does that. Compiled code
trapped with Unimplemented Operation instead, for every denormal
result, whatever FS said.

The trap decision negated the "FS is set" compare with bnot. A
Cranelift compare yields 0 or 1, and bnot(1) is 0xFE, which still
reads as true, so "FS clear or U|I enabled" was always true. The same
mistake in the round-to-integer path was fixed earlier. This one uses
a compare against zero too.

The new equivalence test halves the smallest normal number in both
formats with FS set, FS clear and FS set with Underflow enabled, and
checks the compiled result against the interpreter and the manual.

The trap and flush blocks are marked cold too. They hang off the cold
denormal block but were not cold themselves, so Cranelift laid them out
among the hot instructions of every FP op. On the benchmark suite that
cost fpu/linpack its fast mode in more runs (median 565 against 619
MIPS on main) and left fpu/transcend about 1% behind main; with the
blocks cold linpack matches main and transcend is 8% ahead.
@jamesbraid
jamesbraid force-pushed the ide-fpu-fix branch 4 times, most recently from 29c9f8e to 18b942b Compare November 3, 2026 03:32
@jamesbraid
jamesbraid force-pushed the ide-fpu-fix branch 5 times, most recently from 0927da0 to 11a418d Compare November 7, 2026 03:34
@jamesbraid jamesbraid changed the title cpu: raise the FP exceptions IRIX's diagnostics check for cpu: report FP results as the MIPS FPU does Nov 7, 2026
@jamesbraid
jamesbraid marked this pull request as ready for review November 7, 2026 20:14
@jamesbraid
jamesbraid merged commit 068d0e0 into techomancer:main Nov 8, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant