[SizzleUnrlsd]

Refs #227

Summary

A DLL that links the runtime has a copy of the runtime of its own, with its own allocation table, block index and quarantine. None of them was ever freed: the table is built in static storage that nothing destroys, so that the exit-time code of the program and the leak report can still use it (#159, #232). When FreeLibrary unloaded a DLL that allocated and freed through its copy, the table's nodes and buckets, the storage its block index grew into, the quarantine's queue and the blocks the quarantine held all stayed allocated, and the next load built new ones. #240 freed the copy's cache of names; this PR frees the rest that the probe below found.

This PR makes the DLL's copy free that state when FreeLibrary unloads the DLL, in windows/ct_runtime_alloc.cpp:

• Where. In the copy's .CRT$XTY terminator, which the DLL's C runtime runs after the DLL's atexit handlers, the destructors of its global objects, its destructor functions and the terminators of its own code, in .CRT$XTU. The terminators of later sections, which a library linked into the DLL may add, run after it, then the C runtime's teardown, which calls no function of the runtime: see Afterwards.
• When. Only when the loader detaches the DLL (ct_dll_detaching, set by the TLS callback, which runs before the C runtime's detach) and the unload is dynamic (ct_dll_unloading_dynamically()). When the process exits, the copy keeps everything for its leak report; when the DLL ends the process with its own exit(), the terminator runs before the detach and keeps everything too, for the report that the TLS callback makes then (Windows: a DLL's copy of the runtime may never report its leaks at exit #232, fix(runtime): report the leaks of a DLL's copy when the process exits #234).
• What.
  • The blocks in the quarantine, which the program had released, are released for real, and the quarantine's queue is freed: ct_quarantine_release_all(), new in ct_runtime_quarantine.cpp, shared with the POSIX runtime.
  • The block index is emptied back onto the storage it starts with, in the module's image, freeing what it grew into. Its statics move to the file's anonymous namespace for that.
  • The allocation table is destroyed.
• What stays. The blocks that the DLL still holds stay allocated: the copy frees its records of them, not the blocks, which the program may still use.
• As the runtime's own work. The release runs in a runtime scope (CtRuntimeScope), as every other change of the table does. The table and the quarantine release blocks with ct_alloc_lock held, and those releases may reach the runtime's entry points through the copies of the library's inline functions that an instrumented DLL compiles (Windows runtime: its own allocations re-enter it through instrumented inline STL code #197). Outside a scope, the entry point would take the lock again and leave FreeLibrary waiting forever with the loader lock held.
• Afterwards. ct_alloc_table() builds the table under ct_alloc_lock whenever it is not built, instead of once through a function-local static: every use of the table holds that lock. Code of the DLL that runs after the copy's terminator and goes through the copy finds a new table instead of the destroyed one. Building the table allocates with the lock held, and some uses that may build it hold the lock outside a runtime scope: a bounds check (__ct_check_bounds), and the lookups of __ct_realloc and __ct_munmap. The build therefore runs in a runtime scope of its own, whichever use comes first.

Measurement

A probe measured the busy blocks of every heap of the process across loads and unloads of a DLL that allocates and frees through its copy (probe/227-alloc-state, 76bafcf, run 41274637940), with the method of #227's first probe. The four Windows jobs gave the same results.

Variant	Cycles	Kept
A DLL without the runtime	20	nothing
A DLL that links the runtime and allocates nothing	20	nothing that accumulates
64 blocks of 64 bytes allocated and freed per load	20	about 42 KB per cycle
5000 blocks of 16 bytes allocated and freed per load	4	957 to 960 KB per cycle

The blocks kept per cycle match, by their sizes, the state of one copy per load. The probe shows no allocation stacks, so this attribution rests on exact sizes:

• 24,576 bytes: the quarantine's queue, 1,024 entries of 24 bytes; 196,608 bytes once it grew to 8,192 entries.
• 229,376 bytes: the block index grown to 8,192 entries, ct_block_index_storage_size(8192).
• Blocks of 56 bytes, one per freed block plus two: the table's nodes. Blocks of 8,231 and 131,111 bytes: its buckets.
• Blocks of 64 and 16 bytes, and of 512 and 40,000 bytes: the blocks that the DLL freed and the quarantine held, and the arrays that held their pointers.

Blocks of about 12 KB appeared and disappeared without accumulating, as in #227's first probe.

Evidence

Each probe runs the tests of this branch in the four Windows jobs of LLVM versions, LLVM 19, 20, 22 and 23, against other sources. All but the mutant ran the tests of 7541d97.

Run	Branch, head	Sources	Windows jobs
41350156893	probe/227-alloc-state-red, a04e828	main	233/235 in each job: the two accumulation tests fail, with 2,661 blocks (834,601 bytes) kept over 20 loads and, for the index test, 200,121 blocks (19,156,521 bytes), against fewer than 10. The three death tests pass. The quarantine test, whose function main lacks, is left out.
41350145170	probe/227-review-red, 8667488	7c09bf0, the first fix	233/236 in the jobs of LLVM 19, 20 and 22, 234/236 in LLVM 23's: the two defects below.
41350148036	probe/227-poison-red, a203fdb	7c09bf0, the destroyed table's memory filled with 0xA5	233/236 in each job: the same three death tests fail.
41350154003	probe/227-check-bounds-red, 36e0982	707ab8c, the table built again outside a runtime scope	235/236 in each job: only ADllThatChecksAnAccessAfterItsCopyFreedItsStateUnloads fails, with exit status 7 after 20 seconds.
41341735151	probe/227-index-mutant, d5f9b08	this branch without the release of the block index, with the tests of ad71cea	235/236 in each job: only the index test fails, with 21 blocks kept over 20 loads, 20 of them of 229,376 bytes, one per load: what the index grew into.
41350151253	probe/227-poison-green, c5dac0a	this branch, the destroyed table's memory filled with 0xA5	236/236 in each job.
41350135507	this branch, 7541d97	this branch	236/236 in each job; the 15 jobs of LLVM versions pass, and so do Build, clang-format, license-compliance and the commit conventions.
41775275514	this branch, ce70bac	this branch with main merged in at 09df200, which brought #241: the CHANGELOG conflict resolved, both entries kept, and churnAndUnload starting its watchdog with startWatchdog()	245/245 in each job; the 15 jobs of LLVM versions pass, and so do Build, clang-format, license-compliance and the commit conventions.

The two defects of the first fix, 7c09bf0. A review of 7c09bf0 found two defects, which its own tests did not exercise. Its probe reproduces both:

• The release waited for its own lock. ct_release_alloc_state() freed the table and released the quarantine with ct_alloc_lock held, outside a runtime scope. The blocks it released came back to the copy's entry points through the DLL's operator delete, which took the lock again. In each job, ADllWhoseOperatorDeleteFreesThroughItsCopyUnloads exits with 7 after 20 seconds, and so does ADllThatChecksAnAccessAfterItsCopyFreedItsStateUnloads, whose DLL waits in the same release before its late terminator runs.
• Code after the terminator used the destroyed table. The function-local static still returned the table that the terminator had destroyed. A fault there does not show in FreeLibrary: the loader swallows an exception raised while it unloads a DLL, and FreeLibrary returns as usual. So the death tests have the DLL report the end of its late terminator, and fail with 6 without it. ADllThatAllocatesAfterItsCopyFreedItsStateUnloads exits with 6 in the jobs of LLVM 19, 20 and 22, and passes in LLVM 23's: using a destroyed object is undefined, and it did not fault there. With the destroyed table's memory filled with 0xA5, as a debug heap fills freed memory, the test exits with 6 in each job on 7c09bf0, and passes on this branch, whose next use of the table builds a new one in that memory first.

The defect of the rebuild, 707ab8c. The fix of the second defect, 707ab8c, built the new table outside a runtime scope when a bounds check was its first use. The bounds check takes the lock outside a runtime scope, and the new table's blocks come back to the copy's entry points through the DLL's operator new: ADllThatChecksAnAccessAfterItsCopyFreedItsStateUnloads exits with 7 in each job.

The index mutant. It is no defect of any head: 7c09bf0 already freed what the block index grew into. The index test now runs over 20 loads, as the other accumulation tests do, instead of 4, so that one block kept at each load stands out from the few blocks that a DLL without the runtime leaves. The mutant removes the index's release from this branch, every fix kept: only the index test fails, on the 20 blocks of 229,376 bytes that it keeps, one per load.

The watchdog. The child process of each death test ends itself with 7 after 20 seconds, from a watchdog thread. A thread takes the loader lock to start: the child waits until the watchdog runs before it loads the DLL. In a first run of the check-bounds probe, whose child did not wait (run 41341732725), the job of LLVM 23 waited behind the stuck FreeLibrary until its 45-minute timeout cancelled it, while the three others exited with 7.

ct_quarantine_release_all() is shared with the POSIX runtime, and its test ran RED then GREEN on macOS: with an empty body, QuarantineTest.ReleaseAllEmptiesTheQuarantine fails (280 bytes left in the quarantine, a freed record kept); with the fix, the 76 runtime unit tests pass, five times in shuffled order.

Tests

Test	What it checks	On main
DllAllocState.ABlockADllKeepsStaysAllocatedOnceItUnloads	A block that the DLL allocated through its copy and still holds is in the process's heaps once FreeLibrary has unloaded the DLL.	Passes
DllAllocState.ADllThatAllocatesThroughItsCopyKeepsNoHeapBlockAtEachLoad	Over 20 loads that each allocate and free 64 blocks of 64 bytes, the DLL leaves fewer than 10 blocks beyond a DLL without the runtime.	Fails
DllAllocState.ADllWhoseBlockIndexGrewKeepsNoHeapBlockAtEachLoad	Over 20 loads that each allocate and free 5000 blocks, more than the 4096 the block index starts with, the DLL leaves fewer than 10 blocks beyond a DLL without the runtime.	Fails
DllAllocStateDeathTest.ADllWhoseOperatorDeleteFreesThroughItsCopyUnloads	FreeLibrary and the DLL's late terminator return within 20 seconds when the DLL's operators new and delete go through its copy while the copy frees its table and releases its quarantine.	Passes: nothing is released
DllAllocStateDeathTest.ADllThatAllocatesAfterItsCopyFreedItsStateUnloads	FreeLibrary and the late terminator return when the DLL allocates and frees four blocks through its copy from that terminator, which runs after the copy's.	Passes: nothing is destroyed
DllAllocStateDeathTest.ADllThatChecksAnAccessAfterItsCopyFreedItsStateUnloads	FreeLibrary and the late terminator return when the DLL, whose operators new and delete go through its copy, checks an access through the copy from that terminator: the first use of the table since the copy freed it.	Passes: nothing is destroyed
QuarantineTest.ReleaseAllEmptiesTheQuarantine	ct_quarantine_release_all() leaves no byte in the quarantine and no freed record of its blocks, and a block released afterwards goes to the allocator at once.	No function yet

The DLL tests use a new fixture, reload/ct_allocating_library.cpp, whose allocation trace is off. Their failure messages give the blocks kept by size only. The fixture is built a second time with reload/ct_new_delete_through_runtime.cpp, whose operators new and delete go through the copy: the blocks that the runtime allocates and releases for itself then reach its entry points, as through an instrumented DLL's copies of the library's inline functions (#197). When asked, the fixture also works through its copy from a terminator in .CRT$XTYZ, which its C runtime runs after the copy's in .CRT$XTY: it checks an access, then allocates and frees blocks, then calls a function that the test gave it.

The three death tests guard the release and what comes after it, which main does not do. Each child process starts its watchdog, waits until it runs, and loads the DLL. It exits with 0 once FreeLibrary and the DLL's late terminator have returned; with 6 when FreeLibrary returned without the late terminator, as when the loader swallowed a fault there or in the copy's terminator; and with 7 when FreeLibrary has not returned within 20 seconds, as when it waits for a lock that its own thread holds.

Limits

• Code after the copy's terminator. Code of the DLL that runs there and goes through the copy, such as a terminator that a library linked into the DLL puts in a later section, finds a new table. What that code leaves stays allocated once the DLL is gone, and depends on what it does:

  • Any use of the copy that looks the table up, even a bounds check: the new table, empty.
  • A block it allocates and still holds when the DLL is gone: the block's record, a node of 56 bytes in the measurement above.
  • The most blocks it holds at once: the table's buckets, which do not shrink, and, past 4,096 blocks, the storage that the block index grows into; the index holds 4,096 blocks in the module's image.
  • A block it releases: nothing. The quarantine's limit stays 0 once ct_quarantine_release_all() has set it: CT_QUARANTINE_MB is read only while no limit is set, and no other function of the runtime sets one, only tests do. So the block goes to the allocator at once, its freed record dropped, from any thread, and the queue is not allocated again.

  Nothing stays when no code goes through the copy after its terminator, as in every test but the two that ask for it.

• Memory outside the heaps. The probe and the tests see the heaps only, not VirtualAlloc regions such as the shadow memory of a DLL built with it, nor handles.

• Before Windows 10. ct_dll_unloading_dynamically() cannot tell FreeLibrary apart from the process's termination there: nothing is freed, as before.

• Linux and macOS. The POSIX runtime's state when dlclose unloads a shared library that links it is neither measured nor changed, ct_quarantine_release_all() aside, which it does not call.

• __ct_munmap. It drops the freed record of a page with ct_alloc_lock held, outside a runtime scope, and waits for its own lock when the node it frees reaches the runtime's entry points (Windows runtime: its own allocations re-enter it through instrumented inline STL code #197). This PR does not change that path: Windows runtime: __ct_munmap drops a freed record under the table's lock outside a runtime scope #243, reproduced on ad71cea by probe/munmap-forget-lock.