Skip to content

PE: Resolve the machine type of a ReadyToRun image built for another system - #757

Open
zardus wants to merge 1 commit into
masterfrom
feature/fix-cle-readytorun-machine
Open

zardus wants to merge 1 commit into
masterfrom
feature/fix-cle-readytorun-machine

Conversation

@zardus

@zardus zardus commented Aug 17, 2026 •

Copy link
Copy Markdown
Member

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

A .NET ReadyToRun assembly published for a target other than Windows cannot be
loaded at all. On binaries/tests/x86_64/readytorun_linux_x64.dll:

EXCEPTION: archinfo.arch.ArchNotFound: Can't find architecture info for architecture 0xfd1d with '' bits and unsure endness

The failure is at architecture selection, so nothing about the image is
recoverable: no sections, no symbols, no entry point.

Root cause

The backend takes the COFF machine type at face value:

machine_type = self._pe.FILE_HEADER.Machine
self.set_arch(archinfo.arch_from_id(pefile.MACHINE_TYPE.get(machine_type, hex(machine_type))))

The ReadyToRun compiler exclusive-ors that field with a constant naming the
target operating system, so that the Windows loader refuses a file that is not
for it. 0xfd1d is IMAGE_FILE_MACHINE_AMD64 ^ 0x7b79, and 0x7b79 is the
value the .NET runtime gives IMAGE_FILE_MACHINE_NATIVE_OS_OVERRIDE for Linux in
src/coreclr/inc/pedecoder.h. MACHINE_TYPE.get misses, hex(0xfd1d) is not an
architecture name, and arch_from_id raises.

Fix

machine_type_name undoes the override before naming the architecture. It tries
the six documented constants only when the raw machine type is unknown and the
image really is ReadyToRun -- a managed assembly whose CLR header points at a
native header beginning with RTR\0 -- so a corrupt Machine field elsewhere
cannot be rescued by an exclusive-or that happens to land on a valid value. os
stays "windows", because the file is still a PE.

loaded readytorun_linux_x64.dll: arch=AMD64 os=windows entry=0x180000000

Testing

tests/test_pe.py::TestPEBackend::test_readytorun_machine_os_override reads the
Machine field out of the file, asserts it is 0x8664 ^ 0x7b79, then loads the
fixture and asserts arch.name == "AMD64" and is_dotnet; it raises
ArchNotFound on the merge base.

#805 changes how a machine-type name becomes an archinfo.Arch, at the same two
call sites this branch rewrites, so the two conflict in cle/backends/pe/pe.py
and whichever merges second needs a rebase. The resolution is
arch_from_machine_type(machine_type_name(...)) at both sites, with both helper
functions kept. #805 can land first: it asserts the mapping rather than loading
a PE, so it needs no fixture, while this branch needs the one named in the
sync: line below.

Fixes #751. Validation: #757 (comment)

sync: angr/binaries#183

session: sharpen

@zardus

zardus commented Aug 17, 2026 •

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 304d3e5462a8e86d517121d4ed5a5b5dd0c1a733 against baseline 0e77ade3c39a3cee05f65051e57955675e1ac21b.

  • Regression: pytest tests/test_pe.py::TestPEBackend::test_readytorun_machine_os_override — fails against baseline cle with archinfo.arch.ArchNotFound: Can't find architecture info for architecture 0xfd1d, passes on head
  • Focused: pytest tests/ in cle — 226 passed, 9 skipped on head; 225 passed, 9 skipped on baseline
  • Lint/type: ruff check, ruff format --check, and run-ci-diff-checks.py --repository cle (the merge-base pylint and pyright comparison the hosted jobs apply) — all changed files improve or hold
  • Workspace gate: run-all-tests.sh --jobs 2 in an ANGR_FEATURE shell with cle, binaries and angr adopted — all selected suites passed: workspace checks, the test-inputs fixture check, every configured pre-commit hook, the per-feature instancing suite, cle 235 passed 9 skipped, angr 2,471 passed 46 skipped 2 xfailed 260 subtests, angr Rust 35 passed. archinfo, pypcode, pyvex, claripy and angr-management are skipped as unadopted and untouched. A first run at the default four workers lost tests/procedures/libc/test_strtol.py to SIGKILL; memory.events recorded exactly one new oom_kill, and the test passes alone in 140s, so the run was repeated at two workers

Corpus A/B, 1,557 PE objects, one process per object:

Baseline Head
objects with machine 0xfd1d 37 37
of those, loading 0 37
ArchNotFound over the whole population 47 10
objects whose loaded memory, relocations, symbols, entry point or hints changed — 37
  • Every difference: the 37 objects that previously raised ArchNotFound now load as AMD64. Nothing else in the population changes; the 1,520 objects whose machine type pefile resolves take the unchanged branch and are byte-identical.
  • All 37 carry a 72-byte IMAGE_COR20_HEADER whose ManagedNativeHeader points at RTR\0, and all 37 use the Linux override 0x7b79. No other object in the population reaches the new code.
  • The 10 remaining ArchNotFound are LOONGARCH64 (0x6264), which pefile resolves and archinfo has no architecture for. Unchanged, and not something this change can address.
  • Once loaded, those 37 images contribute 138,411 base relocations, which PE: Build base relocations on every architecture the backend resolves #755 builds and applies; this change only resolves the architecture.
  • What this does not buy: CFGFast still recovers nothing from these images with its defaults, because a ReadyToRun assembly has no entry point, no symbols and no exception directory to seed from, and a forced complete scan mostly decodes the ReadyToRun metadata rather than the native code. Indexing the native methods needs the RuntimeFunctions section of the ReadyToRun header, which nothing here reads. The claim is that the file opens and resolves AMD64, not that analysis of it works.

Reproduction without the corpus:

$ dotnet new classlib -o readytorun --framework net9.0 && cd readytorun
$ dotnet publish -c Release -r linux-x64 --self-contained false -p:PublishReadyToRun=true
$ python -c 'import cle; cle.Loader("bin/Release/net9.0/linux-x64/publish/readytorun.dll")'

Caveats: the corpus is a private dataset, so objects are named by machine type and count rather than by path; the fixture in angr/binaries#183 is built by exactly the recipe above with .NET SDK 9.0.316. Only the Linux override is exercised by real images here; the other five constants come from the .NET runtime's pedecoder.h. PE.__init__ still sets os = "windows" for these images.

CI on this PR: all 20 checks are green at head 1b8dcae9990f0cc8d57166560447965747e0655b,
read 2026-09-03, with the fixture pull request angr/binaries#183 still open.
Test macos-15 and Test windows-2022 are among them, green in run
33270221948.

Correction, 2026-09-03. An earlier version of this comment said that
Test macos-15 fails and Test windows-2022 is cancelled with it because
cle's own matrix job checks out angr/binaries with no ref: and so gets
master. That described run
32006703061 of 2026-08-17 and no
longer holds: cle master gained the missing resolution in 5125f1ba ("Check
out a referenced angr/binaries pull request on macOS and Windows", 2026-08-18).

Rebased 2026-08-27. This record was measured at 5eacf2dfc9415f2ef1d4d660e5d884f05e87c677. The branch was rebased to a6b36dfd81123f1e4c9aa6b8c4ef31ffd94802a8 and again onto master d2ecea06, and is now at b9496b5b8d8e28fc4b5d0967a832c9ac02cd1b57. Correction to that sentence as first published: git range-diff does not report the commit strictly unchanged — the rebase merged this branch's import struct in tests/test_pe.py with the import sys master added in between. Both revisions add the same 89 lines and remove the same 3, and no production hunk differs, so every figure above still describes this head.

CI status, head b9496b5b8d8e28fc4b5d0967a832c9ac02cd1b57, read 2026-08-27.

ci / Test (0) was red for a stale sibling, not for anything in this diff and not because the sibling is unmerged. The failing test was tests/analyses/cfg/test_cfgfast_soot.py::TestCfgfastSoot::test_invokespecial_on_an_interface, which belongs to angr and to no file here; it died at construction with Exception: Not a valid binary file: '.../binaries/tests/java/interface_default.jar'. The referenced fixture branch is checked out at its tip rather than merged with master, so its staleness came into the run: angr/binaries#183 sat one commit behind angr/binaries master, and that one commit — de38bc3, merged 2026-08-26T23:33Z — is the one that adds tests/java/interface_default.jar. Querying the pinned revision for the file the test wanted returned 404; querying master returned the blob. That is the discriminator, and it predicts the failure exactly.

The repair belongs on the sibling. feature/pe-loader-fixtures has been rebased onto angr/binaries master, 25318d666b6464c5aa91f701e091a82d80c625dd to 8159566adc4c1ae5128d6f4da0533be399486cb2, with git range-diff reporting the commit unaltered, and run 33039384952 re-run in full — not --failed, because the dependency snapshot is taken in Build, which --failed skips.


The population this closes (measured 2026-08-28)

Private corpus; objects are cited by architecture, container format and sha256 only. This is the same defect the record above measures against a larger PE population, scored here against a different body of objects — the two do not share objects, so the counts are not comparable and neither supersedes the other.

ArchNotFound raised at archinfo/arch.py:arch_from_id accounts for 23,152 objects, and 0% of it is fixed on master: 106 of them, stratified across every architecture family and container in the group, raise ArchNotFound identically at the pinned revisions and at archinfo master f92307b, and arch_from_id is byte-identical between the two.

99.0% of that population is a correct refusal, not a defect. ia64 (9,095), alpha (7,352), vax (5,589), sparc (612), sm83 (197), 65816 (70) and 32-bit s390 (2) have no definition in archinfo and no SLEIGH language in pypcode 4.0.1.dev0 to fall back to: of its 187 languages, ia64, itanium, alpha, vax, sm83, 65816 and hppa match none, and the only sparc languages are sparc:BE:32:default and sparc:BE:64:default while every sparc object sampled here is little-endian — 8 of 8 ELF objects by EI_DATA, and 16 headerless objects by counting the SPARC save prologue in both byte orders (0 big-endian hits in any of them, 2 to 30 little-endian hits each).

The defects inside it are three small cle sub-populations, and this change closes the largest of them: all 124 cil objects. Every one is a PE32+ ReadyToRun image whose COFF Machine is 0xfd1d, and all 124 present that same ident to arch_from_id, because cle/backends/pe/pe.py:100 passes hex(machine_type) when pefile.MACHINE_TYPE has no name for the value; 0xfd1d ^ 0x7b79 == 0x8664. Eight of them were loaded through a worktree at this branch's head b9496b5, with cle resolving to that worktree and archinfo to master: 8 of 8 resolve as AMD64, three sections, entry 0x180000000. One machine value, one override, and a mechanism that keys on nothing else, so the sub-population closes as a unit.

The other two are 107 hppa ar/CaRT objects, which #740 addresses, and 3 LoongArch PE/UEFI objects, which need the PE backend to gain the p-code fallback the ELF backend already has — pypcode does ship Loongarch:LE:64:lp64d, so nothing but that missing path stands in the way, and no open pull request writes it.

Re-keyed 2026-09-04, after a rebase onto cle master 0e77ade3c39a3cee05f65051e57955675e1ac21b. The branch moved from 1b8dcae9990f0cc8d57166560447965747e0655b to 26ad7c645f63c0651d9e28fb2267fb316089ddc1 because angr master bumped its sibling pins from 9.3.4.dev0 to 9.3.5.dev0 on 2026-09-02, and ci / Build installs the branch's own pyproject.toml with uv pip install --no-sources, so a fresh run on the old head died on a .dev version no index publishes. check-stale-pins.py refuses 1b8dcae9 and exits 0 on 26ad7c64. This was a plain replay: no conflict, no hand resolution, no fixup. git range-diff eac0e554..1b8dcae9 0e77ade3..26ad7c64 reports every commit =, and the branch's own added and removed lines are byte-identical across the move, 92 lines on each side. Master did edit cle/backends/pe/pe.py in the range rebased over, at lines 19-26 and 192-201 in new-base coordinates, which do not reach this branch's hunks at 44-50, 61-67, 124-132 and 230-237 even with three lines of slack. So every figure above describes the same patch on a new base. The opening line named b9496b5b8d8e28fc4b5d0967a832c9ac02cd1b57 before this edit, which is one rebase further back than the head this pass moved: the branch had already been replayed once without the record being re-keyed. Measured the same way over the whole chain, the added and removed lines at b9496b5b are byte-identical to those at the new head, 92 on each side, so the record and the branch have never described different patches.

Hosted CI at 26ad7c645f63c0651d9e28fb2267fb316089ddc1, read 2026-09-04T18:00Z: 20 checks -- 19 success, 1 failure, in run 33891252458, attempt 2.

  • ci / Typecheck is the one failure, and it is this branch's own defect rather than a dependency: job 101103292489 reports pyright 1.1.411 raising cle/backends/pe/pe.py from 52 errors to 54 against master, at lines 96 and 98 -- an int subscript into pefile.MACHINE_TYPE and a str | int returned from a function annotated -> str, both introduced by this branch's new machine_type_name. It is not repaired at that head.
  • Correction, 2026-09-04. An earlier version of this paragraph read this same head at 16:24Z as 14 success, 5 failure, 1 cancelled, with four jobs failing on cle.errors.CLEFileNotFoundError for tests/aarch64/langdetect_go.macho and tests/aarch64/relocatable_object.macho, Test windows-2022 cancelled beside them, and the repair stated as a rebase of Add ARM64, ARMNT and ReadyToRun PE fixtures binaries#183. Add ARM64, ARMNT and ReadyToRun PE fixtures binaries#183 now sits on binaries master 003e82a2bfa641530924055695b36cec8af483ab and carries both fixtures, and the whole run was re-run at this unchanged cle head. Every test job passes now; ci / Typecheck is all that is left red.

Re-keyed 2026-09-04, after an amendment at the same base. The branch moved from 26ad7c645f63c0651d9e28fb2267fb316089ddc1 to 304d3e5462a8e86d517121d4ed5a5b5dd0c1a733. This was not a rebase and no rebase language applies to it: the base is the same cle master 0e77ade3c39a3cee05f65051e57955675e1ac21b, the branch's single commit was amended in place with its message and its author identity and date byte-identical, and git diff 26ad7c64 304d3e54 is 3 insertions and 2 deletions in one file. Every one of them is inside machine_type_name in cle/backends/pe/pe.py: the log argument pefile.MACHINE_TYPE[machine] becomes pefile.MACHINE_TYPE.get(machine), and return pefile.MACHINE_TYPE.get(machine, hex(machine)) becomes name = pefile.MACHINE_TYPE.get(machine) followed by return name if isinstance(name, str) else hex(machine). No other file moved and no test changed.

That repairs the ci / Typecheck failure recorded above. Under the rule the hosted job applies -- angr/ci-settings origin/master b23d782e1052da0e512b1a1ef1902d06e0545730, ci-image/scripts/typecheck.py, a per-file pyright error count that fails when a changed file gains one -- pyright 1.1.411 now reports cle/backends/pe/pe.py at 52 errors on the base and 51 on this head, where the previous head raised it to 54; tests/test_pe.py is 4 and 4. The three diagnostics that went away are the two __getitem__ ones from the log subscript and the Type "str | int" is not assignable to return type "str" from the return, and nothing new appeared.

The figures above were measured on the previous head and still describe this one, because the two rewritten expressions are equivalent to the ones they replace on every input the field can hold. pe.FILE_HEADER.Machine is parsed by pefile 2024.8.26 from "H,Machine", so its domain is 0..0xFFFF; over all 65,536 of those values MACHINE_TYPE.get(m, hex(m)) and name = MACHINE_TYPE.get(m); name if isinstance(name, str) else hex(m) agree on every one, 0 differences. pefile.MACHINE_TYPE has 35 integer keys, every value a str and none None, which is the only way the two forms could part. So the 37 ReadyToRun objects still resolve AMD64 and the other 1,520 PE objects still take the unchanged branch.

Hosted CI at 304d3e5462a8e86d517121d4ed5a5b5dd0c1a733, read 2026-09-04T19:25Z: 20 checks, all 20 success -- nothing failing, nothing cancelled, nothing still running. That is run 33908681083, attempt 1, itself concluded success. The count was read twice, with gh pr checks and with a rollup census that puts an unfinished CheckRun -- one whose conclusion is the empty string rather than null -- in the running bucket instead of dropping it; both instruments give 20 of 20.

ci / Typecheck, which is what this amendment was for, passes in 1m4s at job 101141416565. The failure recorded above, at 26ad7c645f63c0651d9e28fb2267fb316089ddc1, is therefore closed, and it was closed by amending the branch's single commit at the same base -- not by a rebase. The hosted job applies ci-image/scripts/typecheck.py from angr/ci-settings origin/master b23d782e1052da0e512b1a1ef1902d06e0545730 with pyright 1.1.411 against base 0e77ade3c39a3cee05f65051e57955675e1ac21b, and its verdict agrees with the per-file counts stated above, which the pre-publication review and the push reproduced independently of each other: cle/backends/pe/pe.py 52 -> 51 and tests/test_pe.py 4 -> 4.

@angr-bot

Copy link
Copy Markdown
Member

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_757

@zardus
zardus force-pushed the feature/fix-cle-readytorun-machine branch 2 times, most recently from ecae05b to eb57694 Compare August 22, 2026 15:30
@zardus
zardus force-pushed the feature/fix-cle-readytorun-machine branch 2 times, most recently from a6b36df to b9496b5 Compare August 27, 2026 04:25
@zardus

zardus commented Aug 28, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Loading the ReadyToRun assembly published for linux-x64, before and after this change. The fixture comes from the angr/binaries pull request linked from the description.

the reproducer
import logging, os
logging.getLogger("cle").setLevel(logging.CRITICAL)
import cle

BIN = os.environ["BINARIES"]   # a checkout of angr/binaries
T = lambda *p: os.path.join(BIN, "tests", *p)

path = T("x86_64", "readytorun_linux_x64.dll")
obj = cle.Loader(path, auto_load_libs=False).main_object
print(f"loaded {os.path.basename(path)}: arch={obj.arch.name} os={obj.os} entry={obj.entry:#x}")

Before — the operating system override in the COFF machine type makes architecture selection raise, so nothing loads:

cle master at d2ecea068794d20b1f14d90eecc1bc4bc4cfa431
EXCEPTION: archinfo.arch.ArchNotFound: Can't find architecture info for architecture 0xfd1d with '' bits and unsure endness
  raised at: File "/home/yans/angr/mega-corpus/repos/archinfo/build/__editable__.archinfo-9.3.3.dev0-py3-none-any/archinfo/arch.py", line 917, in arch_from_id

After — the override is undone and the image loads as AMD64:

with this change, at b9496b5b8d8e28fc4b5d0967a832c9ac02cd1b57
loaded readytorun_linux_x64.dll: arch=AMD64 os=windows entry=0x180000000

@zardus

zardus commented Aug 29, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Rebased onto cle master eac0e554; the branch was CONFLICTING after #800 merged. New head 1b8dcae9.

The conflict was #800, which added EFI_SUBSYSTEMS and image_os() at the same two points in cle/backends/pe/pe.py and a UEFI test at the same point in tests/test_pe.py. All three are kept alongside this branch's additions. The call site in PE.__init__ merged untouched, so self.os = image_os(...) and set_arch(arch_from_id(machine_type_name(self._pe))) now sit next to each other.

The change itself did not move: comparing the branch's own diff before and after the rebase with context discarded gives a byte-identical set of added and removed lines.

Validation at the new head: pytest tests/test_pe.py — 16 passed, run against an angr/binaries checkout with this branch's referenced fixture pull request merged into binaries master (the fixture is not on binaries master yet). The merge-base lint and type comparison that ci / Lint and ci / Typecheck apply to changed files reports no regression at the new base.

@zardus
zardus force-pushed the feature/fix-cle-readytorun-machine branch from b9496b5 to 1b8dcae Compare August 29, 2026 19:11
@zardus
zardus force-pushed the feature/fix-cle-readytorun-machine branch from 1b8dcae to 26ad7c6 Compare September 4, 2026 15:44
…system

A .NET ReadyToRun assembly published for a target other than Windows could not be
loaded at all: arch_from_id raised ArchNotFound for machine type 0xfd1d. The
ReadyToRun compiler exclusive-ors the COFF machine type with a constant naming the
target operating system so that the Windows loader refuses a file that is not for
it, and 0xfd1d is IMAGE_FILE_MACHINE_AMD64 ^ 0x7b79.

Recognise those constants, which the .NET runtime defines as
IMAGE_FILE_MACHINE_NATIVE_OS_OVERRIDE in src/coreclr/inc/pedecoder.h, when the raw
machine type is unknown and the image really is ReadyToRun.
@zardus
zardus force-pushed the feature/fix-cle-readytorun-machine branch from 26ad7c6 to 304d3e5 Compare September 4, 2026 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A .NET ReadyToRun image built for a non-Windows target fails to load: its Machine field carries an OS override

2 participants