Keep a rebased object out of the null page

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

Loading this ARMEL blob at 0x90000000 places the extern object at zero. find_object_containing(0) then returns that object for an address outside the input image.

Root cause

Loader._find_safe_rebase_addr starts at zero when the mapped address space is narrower than 32 bits or the image reaches its upper half.

Fix

Search above both the image and the loader's configured page_size first, then elsewhere above that null-page guard, and use the low guard region only if those searches cannot fit the object. Keep the mapped address width when calculating the address-space limit.

Testing

tests/test_rebase.py covers ARM and Z80 placement, the exhausted-space fallback, exact one-byte, 4 KiB, and 8 KiB page sizes, an image below the guard, and DOS MZ placement in a 20-bit address space with 16-bit registers. The one-byte regression fails with the removed 4 KiB floor and passes when page_size alone defines the guard.

Fixes #745.

Validation: #765 (comment)

session: sharpen

[angr-bot]

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_765

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head e9a5cf026c0d8e358bf25594c6133f3a19486d2a against baseline 7c5e1a2c1e25524863ed5e778eef63b4d8bc56a1.

The loader now uses self.page_size directly as the null-page guard, with no separate constant or 4 KiB floor. The rebase preserves upstream's mapped_address_bits and the DOS MZ regression for placement above 0x90000 in a 20-bit address space. Fixtures are from angr/binaries at 213d9d4c310eca1be489aefe42ab91d214cd4c6a; no fixture changes are needed.

Regression evidence

• pytest --import-mode=append -q tests/test_rebase.py::test_null_guard_has_no_extra_floor: 1 passed on this head. Restoring the removed max(self.page_size, 0x1000) while keeping the test makes it fail with assert 4096 == 1.
• pytest --import-mode=append -q tests/test_rebase.py tests/test_mz.py tests/test_blob.py: 22 passed through the feature's Nix environment.
• Captured ARM placement output shows the extern object at zero on the baseline and above the image on this head.

Gate and static checks

• ./feature.sh test pr-cle-765: exit 0, PARTIAL PASS (8 of 14 suites). The adopted-scope suites workspace, test-inputs, test-packages, pre-commit, feature-build, mono, pysoot, and cle all passed; the cle suite reported 283 passed, 9 skipped. The six unadopted component suites (archinfo, pypcode, pyvex, angr, angr-rust, and angr-management) did not run. Worktree cleanliness passed with no checkout changed.
• nix/run.sh --feature pr-cle-765 -- python .agents/skills/angr-validate-workspace/scripts/run-ci-diff-checks.py --repository cle: no lint or type regressions across two changed files. cle/loader.py holds pylint 9.93 -> 9.93 and pyright 5 -> 5; tests/test_rebase.py holds pylint 10.00 -> 10.00 and pyright 0 -> 0.

The hosted macOS, Windows, and Pyodide jobs, downstream angr tests, and decompiler snapshots were not run locally; hosted CI must provide that coverage on the new head.

This record replaces the earlier record for 30ce1134096691e9ea16270474530d95a797d554. Earlier corpus/CFG figures, snapshot comparisons, and cross-PR integration results were not rerun and are not carried forward as validation of this candidate. Their replacement is not a claim that the historical measurements were false.

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Extern-object placement for the ARMEL blob tests/armel/i2c_master_read-nucleol152re.bin from angr/binaries at 213d9d4c310eca1be489aefe42ab91d214cd4c6a, loaded at 0x90000000.

Reproducer (BINARIES names the fixture checkout)

import logging
import os
import cle

logging.getLogger("cle").setLevel(logging.CRITICAL)
path = os.path.join(os.environ["BINARIES"], "tests", "armel", "i2c_master_read-nucleol152re.bin")
ld = cle.Loader(path, auto_load_libs=False, main_opts={"backend": "blob", "arch": "ARMEL", "base_addr": 0x90000000})
main, extern = ld.main_object, ld.extern_object
print(f"main object   {main.min_addr:#018x}-{main.max_addr:#018x}")
print(f"extern object {extern.min_addr:#018x}-{extern.max_addr:#018x}")
print(f"find_object_containing(0x0) -> {ld.find_object_containing(0)}")
print(f"find_object_containing(0x4) -> {ld.find_object_containing(4)}")

Before — address zero belongs to the extern object:

cle 7c5e1a2

main object   0x0000000090000000-0x000000009000ff5f
extern object 0x0000000000000000-0x0000000000007fff
find_object_containing(0x0) -> <ExternObject Object cle##externs, maps [0x0:0x7fff]>
find_object_containing(0x4) -> <ExternObject Object cle##externs, maps [0x0:0x7fff]>

After — the extern object is above the image and address zero is unoccupied:

cle e9a5cf0

main object   0x0000000090000000-0x000000009000ff5f
extern object 0x0000000090100000-0x0000000090107fff
find_object_containing(0x0) -> None
find_object_containing(0x4) -> None

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Resolved: the two pull requests this could not be rolled up with

This pull request has been excluded from every mono rollup because it broke two
other green pull requests' tests. Both were assertions pinning the placement this
change moves, not disagreements about behaviour, and both have now been fixed on
the side that owns them.

• Stop container objects from claiming their children's addresses #730 tests/test_overlap.py::test_outer_object_does_not_move_rebased_objects
  asserted rebase(wrapped) == rebase(plain) == 0. The equality is Stop container objects from claiming their children's addresses #730's
  invariant; the 0 was whichever address _find_safe_rebase_addr returned
  before this change. With this branch applied both loaders answer
  0xffffffff80100000, equal as Stop container objects from claiming their children's addresses #730 requires. Fixed in cle 6c31c31.
• Pack and unpack words struct cannot describe #721 tests/test_tls_resiliency.py::TestTlsResiliency::test_tls_24bit_arch
  compared the relocated DTV pointer against thread.dtv_offset, which agrees
  only while the TLS object is mapped at 0. With this branch the thread lands at
  0x500000 and the pointer reads 0x504006 -- correct, and pointing at the
  DTV. Fixed in cle d124129, which compares against
  thread.mapped_base + thread.dtv_offset.

Each fix still fails on its own branch's merge base with the error it was
written for, so neither regression lost its teeth against the defect it covers.

One consequence of this change is worth recording here rather than only on #730.
With the rebase search starting above the image, an outer object mapped at 0 can
only matter in the null-page fallback, which a 64-bit address space never
reaches -- so #730's _free_gaps outer-object skip becomes unobservable through
that path. Measured by removing the skip on the merged tree:

with the outer-object skip (as merged):  plain=0xffffffff80100000 wrapped=0xffffffff80100000 equal=True
without the outer-object skip:           plain=0xffffffff80100000 wrapped=0xffffffff80100000 equal=True

#730's _describe_range_conflict half is unaffected and stays covered by
test_outer_object_does_not_occupy_address_space.

Verification

master + #765 + #730 + #721 at heads d160d975, 6c31c31 and d124129, cle's
whole suite:

257 passed, 9 skipped

session: sharpen

[twizmwazin]

There is already a page size, the null page size is the same size as any other page

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

You're right. I removed the separate null-page constant and the 4 KiB floor; the guard is now exactly self.page_size.

test_null_guard_has_no_extra_floor exhausts the space above a 16-bit image, configures page_size=1 and byte alignment, and asserts that the next object starts at address 1 while address zero stays unoccupied. Restoring the floor makes that test fail because the object starts at 0x1000; the published candidate passes it at address 1. The existing tests still cover 4 KiB and 8 KiB page sizes and the last-resort fallback to address zero.

The focused loader tests pass all 22 cases. I also rebased the branch onto current master; the complete validation record is updated at #765 (comment).