cpu: fix R10000 TLB widths and cache diagnostics, add the R12000 and R14000 by jamesbraid · Pull Request #403 · techomancer/iris

jamesbraid · 2026-11-02T11:08:48Z

iris treated the R10000 like an R4400 in several places. That broke 64-bit IRIX address spaces and the Octane PROM's own diagnostics. This fixes those, adds the R12000 and R14000, and replaces the per-machine CPU constants with one table of the parts SGI actually shipped. Replaces #404 and #405.

Fixes

TLB widths: 44-bit virtual and 40-bit physical addresses (FrameMask, full EntryLo PFN). 64-bit processes above 1 TiB no longer take Address Errors.
TLB refill vector is chosen by the address space that missed, not the current mode (R10000 UM pp. 333-334). The PROM's utlb test needs this.
Overlapping TLB writes invalidate the old entry and set Status.TS. The 2-CPU Tezro needs this.
Uncached stores nobody answers complete instead of bus-erroring. The Octane's ide hits this.
Cache diagnostics: primary caches keep Index Store Data, secondary tags are masked to the bits the part holds, and Config reports the real secondary size. The Fuel's R14000 PROM fails without the tag mask.
Partial LWR keeps the upper word in 64-bit mode. The manuals leave this open; it matches what IRIX's CPU test expects and the sgi-origami QEMU model.

CPU models

src/cpu/parts.rs holds one const per processor SGI shipped: family, MHz, revision and secondary cache. The PRId is computed from family and revision, Count is half the clock, and the r10000_195 name is derived. _EARLY marks earlier silicon of the same family and clock. Each machine lists the parts it shipped with, default first, as its own CPUS; no machine maps itself to a PRId, clock or revision. [machine] cpu stays the string you wrote and is resolved once the machine is final: none is the machine's first part, a family name (cpu = "r14000") or a part name picks the machine's match, and anything else is refused by validate() with the accepted names. with_cpu! picks the cache and executor type once at machine build, so the hot path has no per-model branch.

machine	parts (default first)
Indy, Indigo2	`r4400_150`, `r5000_180` (rev 1.0)
Indigo2 IMPACT (IP28)	`r10000_195` (rev 2.5)
O2	`r5000_180` (rev 2.1), `r10000_250`, `r12000_400`
Octane	`r10000_195` (rev 2.7), `r12000_360`, `r14000_600`

Guest-visible changes: the Indy and Indigo2 R4400 now report rev 5.0 at 150 MHz (was 4.0 at 66), the R5000 rev 1.0 at 180 MHz, and the O2 R10000 rev 3.4 at 250 MHz. An Indy no longer accepts an R10000; the R10000 Indigo2 is the IP28, which now defaults to it. The bare-metal bench rig (test device on) still accepts any part.

Testing

Indy, Indigo2, IP28, O2 (R5000, R10000, R12000) and Octane (R10000, R12000, R14000) all boot IRIX to login, hinv shows the expected chip, and halt -p powers off.
Octane ide: icache2 and utlb now pass; the rest fail exactly as on main.
cargo test --features jitv2 --lib, release and debug, passes.

Performance

It got faster. In deterministic mode (same guest instructions and checksums in all 46 kernels), the whole suite takes 1.27% fewer host instructions on the R4400 and 1.73% fewer on the R10000. sys/tlb_miss takes 16% and 20% fewer, and runs about 20% faster in real time. The one kernel that got slower is mem/unaligned on the R10000, 1.0% more host instructions, because the compiled partial LWR checks 64-bit mode at run time. Every other kernel is within noise.

The R10000 family has 44-bit virtual and 40-bit physical addresses, a FrameMask register and a wider EntryLo; iris kept the R4400's 40 and 36. The R4400 and R5000 are unchanged throughout. Virtual width. EntryHi, the TLB match and the xuseg bounds check all stopped at VA bit 39, but VPN2 is 43:13 and the user segment is 16 TiB (R10000 UM pp. 219, 288). A 64-bit kernel can put an n64 user stack above 40 bits (0x0fff_ffff_aee4, for one), and EntryHi dropped VA[43:40] so such an address took an Address Error. The width now comes from the part's VA_BITS in each place: the EntryHi write mask, the TLB's 64-bit compare (Tlb::set_va_bits, installed by the executor beside set_cache_attrs), the xuseg bounds check, the micro-TLB's cacheable segment test and the EntryHi a TLB exception loads. The mask was spelled three times and is now one const fn, mips_tlb::vpn2_mask. All of it is a const of the CPU model or set once at construction; the TLB lookup is unchanged. This reverses a comment in the executor (b7d2a18) that said widening the EntryHi mask breaks the IP28 and Octane kernels. FrameMask (CP0 register 21) is new with the R10000. Bits 15:0 clear EntryLo bits 33:18 as a TLB write passes them on, and the rest of the register reads as zero (R10000 UM v2.0 p. 260). iris had no register 21, so a kernel that sets it to keep software flags out of the physical address got no effect. The core holds the register; the executor sets which bits are writable, all 16 on the R10000 and none on the R4400 and R5000. TLBWI and TLBWR pass the masked EntryLo values and TLBR returns the masked entry. The register is saved with the rest of CP0 and loads as zero from an older save. CpuModel::R10K_FAMILY says a part is in the R10000 family. Nothing changes while FrameMask is zero, its reset value. EntryLo holds a 28-bit PFN in bits 33:6, a 40-bit physical address, and the two-bit uncached attribute in bits 63:62; 61:34 are reserved (R10000 UM v2.0 pp. 239-240). iris kept 30 bits, so the attribute and PFN bits 33:30 read back as zero and never reached the TLB. The core has an EntryLo mask the executor sets per part: 0x3FFF_FFFF on the R4400 and R5000, 0xC000_0003_FFFF_FFFF on the R10000. The attribute is stored and read back only. Bits 33:32 are physical 39:38, where the Octane's guest keeps software flags; FrameMask clears them on a TLB write, so the page lands at its real address. A TLB write also clears the PFN bits below the page size, so TLBR does not return what was written to them (R10000 UM v2.0 pp. 299-300). The manual documents this for the R10000 only; the later parts follow it as a model choice. Translation already ignored those bits, so only TLBR changes. The Octane PROM's tlb diagnostic writes 0xa5a5a5a4 and 0x5a5a5a58 with 16 KB pages and expects 0xa5a5a524 and 0x5a5a5a18 back; it reported 320 compare errors. On the Octane, the IP28 and an O2 set to the R10000, EntryHi keeps 44 bits and mfc0 of EntryLo returns bits 63:62 and PFN bits 33:30 if the guest wrote them. Tests: a page at 0x123_4000_0000 is kept by the R10000 and masked, with an Address Error, by the R4400; and a TLB write on each, checking FrameMask, the wider EntryLo and the PFN bits below a 16 KB page through TLBR.

A PROM's cache walk reads and writes the caches with Index Load Data, Index Store Data and Index Store Tag. The R10000 models dropped or kept the wrong things. Primary-cache data. Only the secondary cache had a data array, so an Index Load Data on a primary cache always returned zero and an Index Store Data was dropped. The primary caches now have data arrays, and each store is masked to its diagnostic field (R10000 UM pp. 262, 271-272): 36 data bits and 1 parity bit for the instruction cache, 32 and 4 for the data cache, 64 and 10 for the secondary. An Index Store Data to the secondary cache writes a quadword, so the other doubleword becomes zero and takes the same check bits (p. 91). A mapped primary Index Load Data or Index Store Data still takes TLB Refill and Invalid, so CACHE now translates the operand for those two; the other index operations still do not. The secondary data slot is now the address >> 3 where it was >> 4, so the walk gets one slot per doubleword instead of two doublewords sharing one. Secondary tags. Index Store Tag stored all 64 bits of a secondary-cache tag, so a walking-1s test read back bits the R10000 defines as reserved, which the manual says return zero (R10000 UM p. 270, figure 11-31). MRU, the physical tag, state, virtual index and ECC are kept, which assembled is 0x8000_000f_ffff_cdff, and the rest dropped. The mask is CpuModel::SECONDARY_TAG_MASK, all ones except on the R10000 family. MRU stays because the PROM writes it as TagHi bit 31 and reads it back. The later parts share the tag layout (the R12000 manual agrees; the R14000 is taken to match), and a PROM's tag test on the R14000 expects the reserved bits to read as zero. Index Store Data is a diagnostic walk's operation, so it runs out of line in a cold function. Which encodings translate, store a tag, load a tag or load ECC is decided by CacheOp::decode and tested as a bit of a mask computed once per model (CpuModel::CACHE_OPS), not by a chain of compares: written inline the extra case grew cache_op and exec_cache in every CPU model and moved the interpreter's hot loops, which cost sys/llsc about 2% on a ship build. This affects the Octane, the IP28 and an O2 with an R10000. Tests: a primary round trip and the quadword store; an all-ones tag store on the R10000 reads back the mask with an R4400 control that keeps TagLo bits 12 and 13, and the walking-1s round trip expects the reserved bits as zero.

Refill vector. The R10000 picks the TLB refill vector from the address space the missing address lies in and that space's Status bit: user addresses follow UX, xsseg SX, and kseg3, sseg/ksseg and xkseg KX. The current mode only decides which space an address is in (R10000 UM v2.0 pp. 333-334, 342). The manual contrasts this with the R4400, which picks by the current mode's bit; iris did the R4400 thing on every model. So a 64-bit kernel (KX=1, UX=0) missing on a kuseg address went to the XTLB vector at 0x080 where the part uses 0x000. The Octane PROM's utlb diagnostic loads VA 0 in exactly that state and waits at the 32-bit vector; it got "Read TLB miss exception" from the XTLB vector and failed. On the R10000 family a refill miss now reads the region of the address, sign-extended from 32 bits in 32-bit mode, and tests UX, SX or KX. Only the miss path changes; the TLB compare width still follows the mode. Overlapping writes. The R10000 and R12000 do not shut down on a multiple match. A TLB write that would let one lookup match two entries first invalidates every entry it overlaps, the wired ones included, then writes the new entry and sets Status.TS (NEC U10278EJ4V0UM, Status register TS bit p.223 and "Avoiding TLB Conflict" p.300). iris stored the entry and left the duplicates matchable, so which one answered depended on how the lookup map happened to be rebuilt. On the two-processor Tezro the firmware flushes the TLB with 64 invalid entries on the kernel's page number, and the kernel then writes the valid global 16 MiB entry over them; a later write elsewhere left an invalid duplicate answering, and the fetch took TLB Invalid. The R14000 gets the same rule as a model choice; no manual for it was read. The R4400 and R5000 keep their behaviour. An overlap is two entries a lookup could match together: same region and page number under the wider of the two page masks, and equal ASIDs or either entry global. Invalid entries count. The removed entry reads back as an unwritten slot except for its PageMask: the kernel TLBRs its wired sentinel entries and reuses PageMask for the next write, and a zeroed mask made it write a per-CPU mapping as 4 KiB. The cost is confined to TLB writes on these models. An entry that overlaps the new one covers a lookup-map slot the new one covers, so a write whose slots are all empty or its own skips the scan. A PageMask that is not a run of ones, which software can write, defeats the map's ranges and turns the shortcut off. A refill that replaces a one-page entry nothing else covers skips the walk over the other entries. TLBWI and TLBWR are never compiled by jitv2, so both reach the same write and set TS in the interpreter. Tests: a 64-bit kernel missing on kuseg and xsseg on the R10000 and on the R4400, which chooses by mode; the Tezro sequence with the removed entries' PageMask kept; and TS from TLBWI and TLBWR. They fail with the rule off.

Uncached stores nothing answers. A store that nothing answers raised a synchronous Data Bus Error with a precise PC on every CPU. The R10000 cannot do that. It sends an uncached store as a write request, and a write request gets no response; Bus Error comes only from an error response to a read or upgrade request, or from uncorrectable read data (R10000 UM pp. 104, 316). The processor never learns that a store failed. The R12000 manual documents the same no-response write path, and the later parts are taken to match, so the test is the R10000 family. CpuModel:: POSTS_UNCACHED_STORES, true for the family's caches, makes a refused store log and complete in write_data_impl and in the masked-store path (SWL, SWR, SDL, SDR), which the interpreter and the compiled code both go through. An uncached SC or SCD that is refused completes and succeeds (rt = 1); that is a model choice, since the manual says nothing about uncached store-conditional. Loads, instruction fetches, cached accesses and the R4400 and R5000 keep the exception, and device return values are untouched because BUS_ERR also tells non-CPU callers that a request was declined. The error goes no further: nothing in iris raises an interrupt or sets an error register for a store the Octane's bus refuses. The log line prints the first four refused stores and then each power of two, with the running count, so a probe loop cannot flood stderr. The Octane's ide diagnostics store 0x591204 and then 0x4 to physical 0x1e130400, where nothing answers; before, the first store took a Data Bus Error, now both print one line and the run carries on. O2 memory alias. A mapped page whose physical address has a bit set above bit 31 could reach the PCI window instead of RAM on the O2. The O2 aliases RAM at 0x8000_0000, and fold_phys tested that window on the full translated address before cutting it to the 32 bits the board decodes. IRIX keeps software bits in its page-table entries just above the PFN, so such an address missed the window, then lost the high bit in the mask and landed on the PCI range. Cut to the decoded bits first, then apply the alias, then cut its target. The extra AND runs only on an alias hit. Only the O2 has a physical alias. Partial LWR. In 64-bit mode a partial LWR (offset 0 to 2) sign-extended the merged word on every CPU. The manual calls the upper word of a partial LWR implementation specific (NEC U12754E pp. 125-126), and the reference results observed for the R10000 keep the old bits 63:32 at all three partial offsets; with sign extension a test that compares those results miscompares. The interpreter and the JIT now merge the old upper word on the R10000; a full word (offset 3) still sign-extends. The upper word is kept only when 64-bit operations are enabled for the current mode, matching QEMU; otherwise it sign-extends. The later parts are not covered, since nothing shows what they do. The choice is a JitConsts field set from CpuModel::PARTIAL_LWR_KEEPS_UPPER, so compile workers see it. Whether 64-bit operations are on is a byte in the core that the executor sets wherever it re-derives the privilege state, so a compiled LWR spends a byte load and a compare on it rather than decoding Status. The code-cache fingerprint gains lwr_upper=1 only when it is set, so the R4400 and R5000 machines keep their warm caches. The uncached-store and LWR changes affect the Octane, IP28 and an O2 with the R10000; the alias change affects the O2. The R4400 and R5000 are unchanged. Tests: SW, SWL and SC on the R10000 against a bus nothing answers, with the R5000 and a load still faulting; a page at 0x1_8000_0000 with the O2's mask and window expects RAM at 0; and LWR compiled and interpreted, at a partial and a full offset, with the mode's 64-bit bit set and clear, on the R10000 and the R5000, plus the byte the compiled LWR reads across Status writes and ERET.

Add the R12000 and R14000, and replace the per-machine CPU constants with one table of the parts SGI shipped. The Octane shipped with an R12000 or R14000 and the O2 with an R12000, but `cpu` took only r4400, r5000 and r10000. An Octane hinv could not show the later parts, and CP0 Config reported 1 MB of secondary cache for every R10000-family part, so a 2 MB model printed "1 Mbyte". The new parts share the R10000's programming model (44-bit virtual, 40-bit physical, the R10000 cache operations) through model::r10k_family, and Config SS is now log2(secondary size / 512 KB). On the R12000 and later, Index Writeback Invalidate (D) also clears the line's primary data tag, as Index Load Tag then shows; the R10000 keeps it. They keep the upper word on a partial LWR as the R10000 does; for the R12000 and later that is a model choice, as is the R14000's TLB conflict rule. Each machine kept its own facts about the CPU: the Octane had three Count constants and a PRId, the O2 two Count rates, the IP28 another, and machine.rs, ip30_machine.rs and ip32_machine.rs each matched on the CPU type to pick a cache model. Now src/cpu/parts.rs holds one const per part: family, clock, revision and secondary cache size, as IRIX hinv reports them. The PRId is computed from the family and revision, Count is half the clock, and the name (r10000_195) is derived. Where two shipped parts share a family and clock, the earlier silicon takes an _EARLY suffix: the Indy and Indigo2 R5000 is rev 1.0 where the O2's is 2.1, and the IP28's R10000 is rev 2.5 where the Octane's is 2.7. Each machine lists the parts it shipped with in its own CPUS, first is the default: Indy, indigo2_ip22 r4400_150, r5000_180 (rev 1.0) indigo2_ip28 r10000_195 (rev 2.5) O2 r5000_180, r10000_250, r12000_400 Octane r10000_195, r12000_360, r14000_600 `[machine] cpu` stays the string the user wrote. It is resolved once, by MachineConfig::cpu_part, after the profile is final: no cpu is the machine's first part, a family name (`r14000`) or a part name (`r14000_600`) is the machine's part that matches, and anything else is refused by validate() with a message naming the parts it takes. The `[clock] fixed_mhz` override applies in ClockConfig::count_hz, and with_cpu! is the one place a part becomes a cache and executor type. The O2's R10000 and R12000 have 64-byte secondary lines (with 128 its PROM panics before its banner); the Octane R12000's 128 is a model choice. The R10000 Indigo2 is the IP28 board: Linux builds the IP22 sources for the R4X00 and R5000 and the IP28 sources only for the R10000, so an Indy or indigo2_ip22 no longer takes an R10000 and indigo2_ip28 takes nothing else. The GUI's Processor pickers read the same lists. The bare-metal benchmark suite runs on an Indy and measures every part there; it does not run on the IP28's memory map (RAM starts at 0x20000000, out of reach of the KSEG0 image). A config with the test device on therefore takes any part, so iris-bench keeps its r10000, r12000 and r14000 cells and a product config keeps the rule. Guest-visible changes: the Indy and Indigo2 R4400 is rev 5.0 at 150 MHz (Count 75 MHz; was rev 4.0 and 33) and their R5000 rev 1.0 at 180 MHz (was rev 2.1); the O2 R10000 is rev 3.4 at 250 MHz (was rev 2.5 at 180). The Octane and IP28 report what they did. validate() also refuses an O2 R4400, which only the constructor did. Tests: the PRId encoding, a family name picking a machine's first part of it, Config reporting each secondary cache, and Index Writeback Invalidate (D) on the R12000 with the R10000 as the control.

jamesbraid · 2026-11-08T00:57:14Z

think this is good to go now

jamesbraid added 4 commits October 30, 2026 05:16

jamesbraid mentioned this pull request Nov 2, 2026

cpu: keep R10000 cache index data and mask secondary-cache tags #404

Closed

jamesbraid force-pushed the cpu-r10000-fixes branch from 3409bcb to 77dab04 Compare November 2, 2026 15:48

jamesbraid marked this pull request as draft November 2, 2026 16:29

jamesbraid force-pushed the cpu-r10000-fixes branch 4 times, most recently from 0760792 to fe0f658 Compare November 3, 2026 09:59

jamesbraid mentioned this pull request Nov 3, 2026

tezro: add the SGI Tezro, with the V12 and up to four processors #409

Draft

jamesbraid force-pushed the cpu-r10000-fixes branch 2 times, most recently from 6612684 to a977dae Compare November 6, 2026 02:24

jamesbraid mentioned this pull request Nov 6, 2026

cpu: add the R12000 and R14000 as selectable models #405

Closed

jamesbraid force-pushed the cpu-r10000-fixes branch 3 times, most recently from ca0c6b2 to f8383df Compare November 7, 2026 01:45

jamesbraid changed the title ~~cpu: fix R10000 TLB widths, refused stores, the O2 alias and partial LWR~~ cpu: fix R10000 TLB widths, refused stores, the O2 alias, partial LWR and cache diagnostics Nov 7, 2026

jamesbraid marked this pull request as ready for review November 7, 2026 03:43

jamesbraid changed the title ~~cpu: fix R10000 TLB widths, refused stores, the O2 alias, partial LWR and cache diagnostics~~ cpu: fix R10000 TLB widths and cache diagnostics, add the R12000 and R14000 Nov 7, 2026

jamesbraid marked this pull request as draft November 7, 2026 05:18

jamesbraid force-pushed the cpu-r10000-fixes branch from f1ad395 to c207d29 Compare November 7, 2026 05:34

jamesbraid force-pushed the cpu-r10000-fixes branch 5 times, most recently from f1720a5 to 7d02f2e Compare November 7, 2026 18:13

jamesbraid mentioned this pull request Nov 7, 2026

fuel: add the SGI Fuel (IP34 board, IP35 platform) #368

Open

jamesbraid force-pushed the cpu-r10000-fixes branch from 7d02f2e to 081cabd Compare November 7, 2026 19:14

jamesbraid marked this pull request as ready for review November 7, 2026 19:22

jamesbraid mentioned this pull request Nov 7, 2026

cpu: serve loads, stores and fetches from lines the guest stages with CACHE index operations #448

Draft

jamesbraid force-pushed the cpu-r10000-fixes branch 2 times, most recently from cc0d446 to ef7e59c Compare November 7, 2026 23:52

jamesbraid merged commit b7ff730 into techomancer:main Nov 8, 2026
14 checks passed

Navigation Menu

Repository navigation

cpu: fix R10000 TLB widths and cache diagnostics, add the R12000 and R14000 - #403

cpu: fix R10000 TLB widths and cache diagnostics, add the R12000 and R14000#403
jamesbraid merged 5 commits into
techomancer:mainfrom
jamesbraid:cpu-r10000-fixes

jamesbraid commented Nov 2, 2026 •

edited

Loading

Uh oh!

jamesbraid commented Nov 8, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jamesbraid commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fixes

CPU models

Testing

Performance

Uh oh!

jamesbraid commented Nov 8, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jamesbraid commented Nov 2, 2026 •

edited

Loading