Skip to content

mips32: Reject MIPS64-only encodings instead of asserting - #91

Open
zardus wants to merge 1 commit into
masterfrom
feature/mips64-nodecode
Open

zardus wants to merge 1 commit into
masterfrom
feature/mips64-nodecode

Conversation

@zardus

@zardus zardus commented Aug 16, 2026 •

Copy link
Copy Markdown
Member

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

A MIPS64-only encoding is a Reserved Instruction on a 32-bit MIPS, but priv/guest_mips_toIR.c decodes a large group of them without consulting mode64 first. The block reported in pyvex issue 76 is six ordinary MIPS32 instructions followed by daddu $a0, $v0, $zero (0x0040202d) at 0x3c5f0:

$ pyvex.lift(8fc2002c ... 8f998324, 0x3c5d8, ARCH_MIPS32_BE, opt_level=0)
  size=0  instructions=0  jumpkind=Ijk_NoDecode  next=0x0003c5d8

IRSB {
   NEXT: PUT(pc) = 0x0003c5d8; Ijk_NoDecode
}

The four loads, the store and the addiu ahead of the daddu are lost with it, and next points at the start of the block rather than at the encoding that failed, so the caller is handed no bytes at all and no address to resume from.

Root cause

These cases build 64-bit IR and hand it to a 32-bit guest register. libVEX then longjmps out of the whole translation, from putIReg's vassert(typeOfIRExpr(irsb->tyenv, e) == ty) for ld, from the IR sanity check (Iex.Binop: arg tys don't match op tys) for dsll32, and from vassert(mode64) for dmtc1. The unwind happens after disInstr_MIPS_WRK has already emitted IR for the preceding instructions, and past the code that would have kept it.

Fix

is_MIPS64_only_insn recognises the encodings that fail this way, and disInstr_MIPS_WRK consults it before the per-opcode dispatch:

   if (!mode64 && is_MIPS64_only_insn(cins))
      goto decode_failure;

decode_failure is the path every other unsupported encoding already takes, so the block ends at the offending address with everything before it intact:

  size=24  instructions=7  jumpkind=Ijk_NoDecode
  instruction_addresses=(0x3c5d8, 0x3c5dc, 0x3c5e0, 0x3c5e4, 0x3c5e8, 0x3c5ec, 0x3c5f0)
   39 | ------ IMark(0x3c5f0, 0, 0) ------
   40 | PUT(pc) = 0x0003c5f0
   NEXT: PUT(pc) = t21; Ijk_NoDecode

Only encodings that fail today are listed, so the predicate cannot take away a translation that currently succeeds. In three places it is narrower than the mnemonic, because this file shares those opcode slots with instructions a 32-bit guest does decode: SPECIAL function 0x1C–0x1F also hold the MIPSR6 DMUL/DMUH/DDIV/DMOD group, selected by sa, so only sa == 0 is rejected; COP1 rs 1 and 5 are DMFC1 and DMTC1 only when the low 11 bits are clear; and SPECIAL3 DBSHFL selects DSBH and DSHD through sa. It also takes hwcaps, because three whole opcodes change meaning on a MIPSR6 guest rather than merely sharing a field, and there the same bytes are a 32-bit instruction the decoder handles: DADDI is BOVC/BEQC/BEQZALC, and BBIT032 and BBIT132 are JIC/BEQZC and JIALC/BNEZC. Those three are rejected only when the guest is not MIPSR6. The three narrowed cases above need no such test, because at the field values they cover this file builds 64-bit IR whatever hwcaps says. LWU, SD, BBIT0 and BBIT1 decode in 32-bit mode and are left alone, and LLD and SCD test mode64 themselves. BBIT032 and BBIT132, the Cavium OCTEON "plus 32" branches, test a bit above 31 and abort on a non-MIPSR6 32-bit guest, so there they are covered.

Testing

tests/test_mips32_reserved.py, which lands with the consumer, names one encoding per affected group and asserts the two-instruction prefix survives each; on the merge base it fails at its first entry with assert 0 == 8, the block being empty where the prefix should be. Its companion tests pin that MIPS64 still decodes the same encodings, and that seven encodings a coarser guard would have swallowed still keep their prefix on MIPS32. Four of them decode: SD, LWU, BBIT0 and BBIT1, whose opcodes the predicate never covers. Three sit inside a covered slot at a field value it excludes — the MIPSR6 DMUL form, a COP1 encoding with the low bits set, and a DBSHFL that is neither DSBH nor DSHD. The whole PyVEX suite is 93 passed at this head.

A two-build A/B over 114,719 encodings per guest, each lifted behind two decodable instructions, moves 16,786 results on each 32-bit guest, every one of them an abort becoming a clean failure that keeps the prefix, with no encoding losing a translation and no result changing on either 64-bit guest. That covers the guest pyvex builds, which pins Cavium with MIPS32R1 and R2; the MIPSR6 case is read from the decoder rather than lifted, because these bindings cannot construct a MIPSR6 guest. Validation: #91 (comment)

Consumed by angr/pyvex#567, which carries the regression tests and the submodule bump. Merge order: this lands first, then angr/pyvex#567, whose vex submodule is pinned to this head. Merging the pyvex half on its own leaves pyvex master's submodule pointing at a commit that is on no vex branch. Nothing in this repository reads a sync: line — .github/workflows/build.yml is the only workflow and it is three build jobs — so the line below is a note to the reader.

sync: angr/pyvex#567

session: sharpen

@zardus

zardus commented Aug 27, 2026 •

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head bc077e2083ba0a20efb8f4d277ffa99361828b2f against baseline e3062871112afded52ae155757113a5873e9819c, exercised through angr/pyvex#567, which pins exactly this head and carries the tests; this repository has no test suite of its own.

Re-ported 2026-09-16, and the figures below replace every earlier one on this pull request. The previous head 561795c1 was written against libVEX before the Valgrind 3.27.1 import (1ecc894) and does not apply to priv/guest_mips_toIR.c as it now stands: that import rewrote the file by +22,145/-14,229 and split the single opcode switch into disInstr_MIPS_WRK_00/_10/_20/_30 and _Special/_Special2/_Special3. The patch was written again against the new file, so this is a new candidate rather than a rebase, and the corpus figures that were here before were measured on the old libVEX and are withdrawn rather than carried forward.

  • Native syntax check: gcc -std=gnu99 -DAVX_512 -Ipub -Ipriv -Wall -Wextra -Werror=implicit-function-declaration -fsyntax-only priv/guest_mips_toIR.c — exit 0, and the diagnostic set is identical to the baseline's (8 -Wsign-compare, 10 -Wunused-parameter, all pre-existing). Run against the baseline file too, so the instrument is known to speak; a positive control with an undeclared call exits 1.
  • Downstream regression: pytest --import-mode=append -q tests/test_mips32_reserved.py in the coordinated PyVEX checkout — 2 failed, 4 passed on the baseline, 6 passed here. The two failures are the tests asserting the prefix survives, one stopping at AssertionError: ld $a0, -0x3098($v1) assert 0 == 8 and the other failing the same way on bbit032; the other four pass on both sides, which is what makes them a guard against a predicate written too coarsely. The whole PyVEX suite is 93 passed here.

Encoding-space A/B

Two builds of the same PyVEX worktree differing only in priv/guest_mips_toIR.c, each rooted in its own Nix store path, so neither can be the other. 114,719 encodings per guest: every (opcode, function) and every (opcode, rs) slot with three register and six immediate patterns each, plus a seeded sample of 100,000 drawn from the whole 32-bit space. Each is lifted behind two decodable instructions and two trailing nops, so a lost prefix is visible, and the result is compared on size, jumpkind, next and instruction count.

guest encodings changed abort becomes a clean failure keeping the prefix translations lost
MIPS32 BE 114,719 16,786 16,786 0
MIPS32 LE 114,719 16,786 16,786 0
MIPS64 BE 114,719 0 0 0
MIPS64 LE 114,719 0 0 0

Every change is in one direction: a lift that returned an empty IRSB pointing at the start of the block now ends at the offending instruction with the prefix intact. No encoding moves any other way, and no encoding this guest decodes today stops decoding -- the guest being the one pyvex builds, Cavium with MIPS32R1 and R2. On both 64-bit guests not one of the 114,719 results differs, which is the check that the guard is inert when mode64 is set.

The predicate is narrower than the mnemonics, and that is load-bearing

An earlier draft of this re-port tested only the opcode and function, as the pre-import patch did, and the sweep above caught it taking away 111 translations on MIPS32 BE that the baseline decodes, plus 36 encodings whose clean ILLEGAL_INSTRUCTON it turned into a decode failure. The import had given those slots sub-switches:

slot also holds narrowed to
SPECIAL function 0x1C–0x1F the MIPSR6 DMUL/DMUH/DDIV/DMOD group, selected by sa sa == 0
COP1 rs 1 and 5 other COP1 formats that use the low 11 bits (cins & 0x7FF) == 0
SPECIAL3 function 0x24 (DBSHFL) the rest of the byte-shuffle group, selected by sa sa 2 or 5

tests/test_mips32_reserved.py::test_the_guard_is_narrower_than_the_mnemonic pins each of those, using an encoding that differs from a rejected one only in the deciding field. The 111 sit in the first two rows only -- 104 in SPECIAL and 7 in COP1; the DBSHFL narrowing contributed none of them, because the encodings it would have swallowed already raise ILLEGAL_INSTRUCTON rather than decoding, and it is kept because the slot is shared, not because the sweep caught it.

Reading the file turned up a second class the sweep could not see. Three whole opcodes change meaning on a MIPSR6 guest, where the same bytes are a 32-bit instruction this file decodes: DADDI (0x18) is BOVC/BEQC/BEQZALC, and BBIT032 (0x36) and BBIT132 (0x3E) are JIC/BEQZC and JIALC/BNEZC. So is_MIPS64_only_insn takes hwcaps and rejects those three only when VEX_MIPS_CPU_HAS_MIPSR6 is clear.

The test is deliberately confined to those three. An earlier draft stood down for every slot holding a MIPSR6 alternative anywhere in it, which is the wrong criterion: in the SPECIAL doubleword multiply group, in COP1 and in SPECIAL3 DBSHFL the MIPSR6 encodings sit at other values of the field this predicate already narrows on, and at the values it does cover the file builds 64-bit IR whatever hwcaps says -- DMFC1 and DMTC1 have no hwcaps test at all. That draft gave up 9 encodings in this sample for nothing, and DSBH/DSHD besides.

The MIPSR6 arm is unmeasured either way: pyvex_c/pyvex.c pins VexArchMIPS32 to Cavium with MIPS32R1 and R2 and overrides the caller's VexArchInfo, so a MIPSR6 MIPS32 guest cannot be lifted through the bindings the sweep uses. It is read from the decoder, not run.

One group added since the previous head

BBIT032 (opcode 0x36) and BBIT132 (opcode 0x3E), the Cavium OCTEON "branch on bit plus 32" instructions, test a bit above 31 and are MIPS64-only. Every one of the 1,778 and 1,763 sampled encodings in those two slots aborts on a 32-bit guest, with the delay slot supplied; their 32-bit siblings BBIT0 (0x32) and BBIT1 (0x3A) decode normally and are left alone. The previous head's description said the OCTEON encodings were decoded in 32-bit mode and left alone, which is true of BBIT0 and BBIT1 and was wrong about these two.

LLD (0x34) and SCD (0x3C) were on the previous head's list and are not on this one: the import gave both their own mode64 test, so on a 32-bit guest they already raise ILLEGAL_INSTRUCTON and keep the prefix.

Caveats: this repository has no test suite, so the executable evidence is the consumer's. The sweep covers the encoding space, not real binaries; the corpus and CFG measurements from the pre-import head are withdrawn and have not been retaken. 455 of the 114,719 sampled encodings still abort on a 32-bit guest in the SPECIAL2, SPECIAL3 and COP1 slots; those are mixed slots rather than a clean MIPS64-only group and this change does not address them.

@zardus

zardus commented Aug 28, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

The block reported in pyvex issue 76 -- six MIPS32 instructions then daddu $a0, $v0, $zero (0x0040202d) at 0x3c5f0 -- lifted as MIPS32, and one encoding from each affected group behind a two-instruction prefix at 0x80010938, before and after this change.

Before -- the whole superblock is discarded and next points at its own start address, so nothing decoded ahead of the reserved encoding survives and there is no address to resume from:

vex master (875f7c9), via pyvex bdd5441
pyvex: .../wt/mips-base/pyvex/__init__.py

$ pyvex.lift(8fc2002cac4000008fc200148fc3000c8fc6002427c800100040202d0060282d8fc700288f998324, 0x3c5d8, ARCH_MIPS32_BE, opt_level=0)
  size=0  instructions=0  jumpkind=Ijk_NoDecode  next=0x0003c5d8
  instruction_addresses=()

IRSB {
   

   NEXT: PUT(pc) = 0x0003c5d8; Ijk_NoDecode
}

$ for each group, pyvex.lift(3c038004 2462cf68 <encoding>, 0x80010938, ARCH_MIPS32_BE)
  encoding                 word       size insns next
  ld    $a0, -0x3098($v1)  dc64cf68      0     0 0x80010938
  daddiu $v0, $v1, 0x7d    6462007d      0     0 0x80010938
  daddu $v0, $v1, $a0      0064102d      0     0 0x80010938
  dsll  $v0, $v0, 0x17     000215f8      0     0 0x80010938
  dsll32 $v1, $v1, 0       0003183c      0     0 0x80010938
  dsrl32 $v0, $v0, 0x14    0002153e      0     0 0x80010938
  dmult $v0, $v1           0043001c      0     0 0x80010938
  ldl   $t0, 0($v0)        68480000      0     0 0x80010938
  ldr   $t0, 7($v0)        6c480007      0     0 0x80010938
  sdl   $t0, 0($a2)        b0c80000      0     0 0x80010938
  sdr   $t0, 7($a2)        b4c80007      0     0 0x80010938
  dmtc1 $zero, $f0         44a00000      0     0 0x80010938
  dext  $a1, $v1, 0, 1     7c650003      0     0 0x80010938
  dsbh  $a1, $a0           7c0428a4      0     0 0x80010938
  dclz  $a1, $v1           70642824      0     0 0x80010938

After -- the reserved encoding takes the ordinary decode_failure path, so every instruction ahead of it is kept and the block ends at the offending address:

with this change (vex 561795c, via pyvex dca2871)
pyvex: .../wt/mips-567/pyvex/__init__.py

$ pyvex.lift(8fc2002cac4000008fc200148fc3000c8fc6002427c800100040202d0060282d8fc700288f998324, 0x3c5d8, ARCH_MIPS32_BE, opt_level=0)
  size=24  instructions=7  jumpkind=Ijk_NoDecode  next=t21
  instruction_addresses=('0x3c5d8', '0x3c5dc', '0x3c5e0', '0x3c5e4', '0x3c5e8', '0x3c5ec', '0x3c5f0')

IRSB {
   t0:Ity_I32 t1:Ity_I32 t2:Ity_I32 t3:Ity_I32 t4:Ity_I32 t5:Ity_I32 t6:Ity_I32 t7:Ity_I32 t8:Ity_I32 t9:Ity_I32 t10:Ity_I32 t11:Ity_I32 t12:Ity_I32 t13:Ity_I32 t14:Ity_I32 t15:Ity_I32 t16:Ity_I32 t17:Ity_I32 t18:Ity_I32 t19:Ity_I32 t20:Ity_I32 t21:Ity_I32

   00 | ------ IMark(0x3c5d8, 4, 0) ------
   01 | t6 = GET:I32(r30)
   02 | t5 = Add32(t6,0x0000002c)
   03 | t0 = t5
   04 | t7 = LDbe:I32(t0)
   05 | PUT(r2) = t7
   06 | PUT(pc) = 0x0003c5dc
   07 | ------ IMark(0x3c5dc, 4, 0) ------
   08 | t9 = GET:I32(r2)
   09 | t8 = Add32(t9,0x00000000)
   10 | t1 = t8
   11 | STbe(t1) = 0x00000000
   12 | PUT(pc) = 0x0003c5e0
   13 | ------ IMark(0x3c5e0, 4, 0) ------
   14 | t11 = GET:I32(r30)
   15 | t10 = Add32(t11,0x00000014)
   16 | t2 = t10
   17 | t12 = LDbe:I32(t2)
   18 | PUT(r2) = t12
   19 | PUT(pc) = 0x0003c5e4
   20 | ------ IMark(0x3c5e4, 4, 0) ------
   21 | t14 = GET:I32(r30)
   22 | t13 = Add32(t14,0x0000000c)
   23 | t3 = t13
   24 | t15 = LDbe:I32(t3)
   25 | PUT(r3) = t15
   26 | PUT(pc) = 0x0003c5e8
   27 | ------ IMark(0x3c5e8, 4, 0) ------
   28 | t17 = GET:I32(r30)
   29 | t16 = Add32(t17,0x00000024)
   30 | t4 = t16
   31 | t18 = LDbe:I32(t4)
   32 | PUT(r6) = t18
   33 | PUT(pc) = 0x0003c5ec
   34 | ------ IMark(0x3c5ec, 4, 0) ------
   35 | t20 = GET:I32(r30)
   36 | t19 = Add32(t20,0x00000010)
   37 | PUT(r8) = t19
   38 | PUT(pc) = 0x0003c5f0
   39 | ------ IMark(0x3c5f0, 0, 0) ------
   40 | PUT(pc) = 0x0003c5f0
   41 | PUT(pc) = 0x0003c5f0
   42 | t21 = GET:I32(pc)
   NEXT: PUT(pc) = t21; Ijk_NoDecode
}

$ for each group, pyvex.lift(3c038004 2462cf68 <encoding>, 0x80010938, ARCH_MIPS32_BE)
  encoding                 word       size insns next
  ld    $a0, -0x3098($v1)  dc64cf68      8     3 0x80010940
  daddiu $v0, $v1, 0x7d    6462007d      8     3 0x80010940
  daddu $v0, $v1, $a0      0064102d      8     3 0x80010940
  dsll  $v0, $v0, 0x17     000215f8      8     3 0x80010940
  dsll32 $v1, $v1, 0       0003183c      8     3 0x80010940
  dsrl32 $v0, $v0, 0x14    0002153e      8     3 0x80010940
  dmult $v0, $v1           0043001c      8     3 0x80010940
  ldl   $t0, 0($v0)        68480000      8     3 0x80010940
  ldr   $t0, 7($v0)        6c480007      8     3 0x80010940
  sdl   $t0, 0($a2)        b0c80000      8     3 0x80010940
  sdr   $t0, 7($a2)        b4c80007      8     3 0x80010940
  dmtc1 $zero, $f0         44a00000      8     3 0x80010940
  dext  $a1, $v1, 0, 1     7c650003      8     3 0x80010940
  dsbh  $a1, $a0           7c0428a4      8     3 0x80010940
  dclz  $a1, $v1           70642824      8     3 0x80010940

A MIPS64-only instruction is a Reserved Instruction on a 32-bit MIPS, so
the decoder has to report a decode failure for it. This file instead
decodes ld, daddiu, the doubleword shifts and multiplies, ldl/ldr,
sdl/sdr, dmfc1/dmtc1, dext/dins, dsbh/dshd, dclz/dclo, the OCTEON
bbit032/bbit132 branches and their siblings unconditionally, assigning an
I64 to an I32 guest register. That trips either the vassert in putIReg or
the IR sanity check at the end of the superblock, and both unwind out of
the whole translation: the caller gets an empty IRSB whose Ijk_NoDecode
points at the start of the block instead of at the instruction that
failed, so the instructions decoded ahead of it are thrown away too.

Reject those encodings before the per-opcode dispatch, which reaches the
same decode_failure every other unsupported encoding uses and ends the
block cleanly at the offending instruction.

Three of the tests are narrower than the mnemonic, because the slot is
shared with an encoding a 32-bit guest does decode and the narrowing is
on the field that selects between them: the SPECIAL doubleword multiply
and divide group is taken at sa == 0, where the MIPSR6 DMUL/DMUH/DDIV/
DMOD group sits at sa 2 and 3; COP1 rs 1 and 5 are DMFC1 and DMTC1 only
with the low 11 bits clear; and SPECIAL3 DBSHFL selects DSBH and DSHD
through sa 2 and 5, leaving DBITSWAP and DALIGN alone.

Three whole opcodes change meaning on a MIPSR6 guest rather than merely
sharing a field, so the predicate takes hwcaps and rejects them only when
the guest is not MIPSR6: DADDI is BOVC/BEQC/BEQZALC there, and BBIT032
and BBIT132 are JIC/BEQZC and JIALC/BNEZC. The narrowed cases need no
such test, because at the field values covered here the file builds
64-bit IR whatever hwcaps says.

LWU, SD, BBIT0 and BBIT1 decode in 32-bit mode and are left alone, and
LLD and SCD test mode64 themselves.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@zardus
zardus force-pushed the feature/mips64-nodecode branch from 561795c to bc077e2 Compare September 16, 2026 02:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant