Repository navigation
Conversation
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS Validation record for head
Reproducer, using clang 21.1.8 rather than a fixture so it can be rerun from source: On the baseline that last line raises
The bit patterns follow lld-link's Caveats:
Corpus measurement of the open queue, 2026-08-15 — a prerequisite, not an independent recoveryCorrecting the record. The open pull-request queue was scored against 733 objects drawn from a sweep's own failing units (35 error classes, 49 architectures, 16 containers), with each repository's current master as the baseline rather than the revisions the sweep pinned. Each object is loaded with The class here is
Alone, all 24 move from So the corpus effect of this change is real but conditional, and the record should say so: this PR is a prerequisite for the class rather than a change that clears any of it by itself. Loading is a separate claim from analysis, and the loading claim above — that Corpus follow-up, and a dependency worth knowing before merge. A CFGFast sweep over a large mixed-architecture corpus groups its failures by Loading it is not sufficient on its own, though. With this branch alone the CLE labels every COFF/PE object With both applied, the same object goes from So these two are complementary rather than alternatives, and neither alone Re-keyed 2026-08-28. The figures above were measured at |
|
Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_724 |
b1b16c3 to
6551c7a
Compare
6551c7a to
42ac276
Compare
42ac276 to
410339c
Compare
The COFF backend rejected every machine type but I386 and AMD64. CoffParser raised NotImplementedError, which is not a CLEError, and Coff.is_compatible applied the same two-machine filter, so autodetection never reached the backend either and loading an ARM64 or ARMNT object ended in "Unable to find a loader backend". Both are ordinary output of Windows-on-ARM toolchains, and archinfo resolves both to the architectures the PE backend already uses for them. Both are accepted now. The machine check and is_compatible read the same COFF_MACHINE_TO_ARCH_NAME table, and an unsupported machine raises CLECompatibilityError naming the type it rejected instead of NotImplementedError naming nothing. Accepting the header alone does not make the object useful, so this also implements the relocation types those objects carry. ADDR32, ADDR32NB, ADDR64, REL32, SECREL and SECTION are the patch shapes the existing classes already handle. BRANCH26, PAGEBASE_REL21 and PAGEOFFSET_12A/12L patch an immediate field inside a single ARM64 instruction, and MOV32T and BRANCH24T patch the two halfwords of a Thumb-2 instruction. ARMNT code is Thumb-2 only, so function symbols carry the Thumb bit the way CLE already reports Thumb code elsewhere, and so do the extern stubs that stand in for undefined functions. Nothing in the object states this, so without it every function address the backend reports is an ARM address. ADDR32NB and ADDR64 ignored the addend held in the field they patch, unlike DIR32, DIR32NB and REL32 beside them. The .pdata of the x86_64 object in the binaries repository shows the cost: the second ADDR32NB of a RUNTIME_FUNCTION record holds the function's end offset as its addend, so BeginAddress and EndAddress were relocated to the same address. Both apply the addend now, which the ARM64 objects need as well. The tests load the ARM64, ARMNT and R4000 objects the binaries repository now carries, relocate each one through cle.Loader and decode the patched fields back out, rather than assembling a COFF container of their own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
410339c to
9084632
Compare
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS Full COFF load report before and after this change. Row A counts the Before — the cle at the merge base, d2ecea0After — the record spans with this change, 9084632 |
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS How much of a corpus's failure surface this removes, measured rather than Sample. 12,000 objects drawn uniformly at random, from a seeded permutation, Method. Each object is loaded with the catalogue's declared load recipe and Before. 36 / 11,989 = 0.30% of the sample (CI 0.22–0.42), and 7.7% of its After. This head clears all 36, and all 36 go on to reach CFG — the Control. 722 objects that already reached CFG on Three other open changes touch this backend without overlapping this class: The corpus is not redistributable, so its objects are described by architecture, session: sharpen |
The backend maps an object at its own file offsets and gives each section vaddr = PointerToRawData. That offset is only as aligned as the file's packing leaves it, and MSVC packs raw data with no padding between sections, so a section commonly begins where the IMAGE_SCN_ALIGN_* in its own header does not allow. All eight .text$mn sections in tests/x86/fauxware.obj state 16-byte alignment and six of them start at a file offset that is not a multiple of 16. x86 and AMD64 never notice, because their instructions have no alignment requirement. ARM64 and ARMNT, whose machine types #724 adds, do: a .text$mn placed on an odd address holds no instruction anything can decode, every function symbol in it lifts to a zero-length block, and CFGFast recovers nothing from it. Over 66 ARM64 COFF objects, 298 of 880 function symbols went unrecovered and 16 objects recovered none of their own; with the sections placed where their headers ask, all 880 are recovered. A section whose file offset does not satisfy its alignment now gets space of its own past the image with its bytes copied in, which is the treatment a section with no bytes in the file already gets. A header that states no alignment states no requirement and its section is left where it is; the 16 _section_alignment returns for that case is only where to put a section that has to be placed somewhere. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The backend maps an object at its own file offsets and gives each section vaddr = PointerToRawData. That offset is only as aligned as the file's packing leaves it, and MSVC packs raw data with no padding between sections, so a section commonly begins where the IMAGE_SCN_ALIGN_* in its own header does not allow. All eight .text$mn sections in tests/x86/fauxware.obj state 16-byte alignment and six of them start at a file offset that is not a multiple of 16. x86 and AMD64 never notice, because their instructions have no alignment requirement. ARM64 and ARMNT, whose machine types #724 adds, do: a .text$mn placed on an odd address holds no instruction anything can decode, every function symbol in it lifts to a zero-length block, and CFGFast recovers nothing from it. Over 66 ARM64 COFF objects, 298 of 880 function symbols went unrecovered and 16 objects recovered none of their own; with the sections placed where their headers ask, all 880 are recovered. A section whose file offset does not satisfy its alignment now gets space of its own past the image with its bytes copied in, from the same cursor and under the same MAX_IMAGE_SIZE ceiling as the section marked IMAGE_SCN_CNT_UNINITIALIZED_DATA that already goes there. Where that ceiling would stop the move, the section keeps its file offset, which is where its bytes are. A header that states no alignment states no requirement and its section is left where it is; the 16 _section_alignment returns for that case is only where to put a section that has to be placed somewhere. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2 participants
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS
Problem
An ARM64 or ARMNT COFF object fails autodetection, and naming the backend raises something a caller cannot catch as a
CLEErrorand that does not say what it saw:The addend defect underneath it is silent, and reaches x86-64 too. A
.pdataRUNTIME_FUNCTIONis twoADDR32NBrelocations against the same section symbol, told apart only by the addends left in the patched fields, so ontests/x86_64/fauxware.objevery record collapses to a zero-length range:That is the table the unwinder and any exception-handling analysis reads.
Root cause
CoffParser._parsegated the whole backend on two machines, andCoff.is_compatiblerepeated the pair independently:Separately,
CoffRelocationADDR32NB.valuewasreturn self.resolvedby.relative_addrandCoffRelocationADDR64.valuewasreturn self.resolvedby.rebased_addr. Neither reads the field it is about to overwrite, so the producer's addend is discarded.Fix
COFF_MACHINE_TO_ARCH_NAMEbecomes the one machine table_parseandis_compatibleboth consult, gaining ARMNT and ARM64, and an unsupported machine raisesCLECompatibilityErrornaming it.ADDR32NBandADDR64add the resolved address to the field's existing contents. New relocation classes cover the ARM64BRANCH26/PAGEBASE_REL21/PAGEOFFSET_12A/PAGEOFFSET_12Lset and the ARMNTADDR32/MOV32T/BRANCH24Tset; ARMNT code is Thumb-2 only, so its function symbols and extern stubs carry the Thumb bit.The ARM64 relocation types the fixtures do not carry stay unimplemented, and
BRANCH24Traises rather than synthesising a veneer for an out-of-range target.Testing
tests/test_coff.py::TestCoff::test_x86_64_pdata_addendsassertsbegin < endon every.pdatarecord;::test_arm64and::test_armntloadtests/aarch64/coff_reloc_arm64.objandtests/armel/coff_reloc_armnt.objand check each relocation's patched word against the symbol it names;::test_unsupported_machinepins the new error ontests/mips/coff_r4000.obj. All four fixtures are onangr/binariesmaster, so nothing here waits on a fixture change.angr.Projecton these objects additionally needs a Win32 syscall convention angr lacks for AArch64 and ARM; that gap is not addressed here andcle.Loaderis unaffected.Validation: #724 (comment)
session: sharpen