Skip to content

Fix undefined behavior in sound, sith and std - #68

Merged
smlu merged 5 commits into
smlu:developfrom
seelenkuchen:fix/undefined-behavior
Nov 8, 2026
Merged

smlu merged 5 commits into
smlu:developfrom
seelenkuchen:fix/undefined-behavior

Conversation

@seelenkuchen

Copy link
Copy Markdown

Four small fixes for undefined behavior. Three were found by running the engine under AddressSanitizer and UndefinedBehaviorSanitizer on my fork (a native Linux build loading Canyonlands for 1,000 frames). The fourth is an off-by-one found while reading the code.

On the 32-bit builds this repository ships, the first three produce the same results as before. Only the last changes behavior, and only in a debug console command.

Commits

  1. [sound] Keep channel loops from stepping before the array. Three loops decremented the channel pointer before checking the count, so their last step pointed before Sound_apChannels, and to NULL − 1 when there are no channels. The iteration order is the same.
  2. [sith] Skip zeroing the puppet classes when none are allocated. With size 0 the array stays NULL, and memset on NULL is undefined even with a size of 0.
  3. [std] Store and check heap end markers through memcpy. The end marker sits right after the user's bytes, so its address is often unaligned, and dereferencing it as void* or uint32_t* is undefined. It's now written and read as 4 bytes through memcpy, which compiles to the same access on x86. Free already checked 4 bytes.
  4. [sith] Reject cogstatus indexes equal to the cog count. sithCog_CogStatus (the dev-mode cogstatus command) checked index > numCogs, so cogstatus <numCogs> read one element past aCogs. It now checks >=. This adds the first sithCog tests to sithTests, and Docs/Testing.md mentions the new coverage.

Validation

This was built and tested here on this branch's exact changes, on top of a local MinGW-w64 build setup that isn't part of this PR (GCC 13, i686, DX9 Debug, under Wine 10).

  • Passed: sith 41/41, including the 3 new sithCog tests. With the old bound, TestRejectsIndexEqualToNumCogs fails (expected 0, got 1). The other unit suites are unchanged: 12 of 13 pass.
  • Known failures, unchanged by this PR: std 263/265 (a stdConffile %a scan and the __LINE__ value in a Printf message, both specific to GCC/Wine).
  • Commits 1–3 on my fork: the sanitizer run went from 15 reported locations to none, with an identical end state.
  • Not run: MSVC (left to this repo's CI), DX6, Release, guard and QOL profiles.

Draft until CI has run.

Three loops decremented the channel pointer before checking the count, so
the last step pointed before Sound_apChannels (NULL - 1 with no channels).
Same iteration order.

Validation (Linux host, MSVC not run):
- GCC ASan+UBSan: Canyonlands headless, 1,000 frames, no reports (was 15
  locations). End state identical to the Clang native run and to the run
  before this change.
- MinGW i686 Debug and standalone Release build. Unit tests: 13 of 14 pass;
  std fails with the 2 known stdConffile tests. std.System: 61 tests,
  2 known failures, 2 ignored. Regression run: 0.0025, OK.
- Not run: MSVC DX6/DX9 builds, host check.
With size 0 the array stays NULL, and memset on NULL is undefined.

Validation (Linux host, MSVC not run):
- GCC ASan+UBSan: Canyonlands headless, 1,000 frames, no reports (was 15
  locations). End state identical to the Clang native run and to the run
  before this change.
- MinGW i686 Debug and standalone Release build. Unit tests: 13 of 14 pass;
  std fails with the 2 known stdConffile tests. std.System: 61 tests,
  2 known failures, 2 ignored. Regression run: 0.0025, OK.
- Not run: MSVC DX6/DX9 builds, host check.
The marker sits right after the user's bytes, so its address is often
unaligned. It's now always 4 bytes, which is what Free already checked.

Validation (Linux host, MSVC not run):
- GCC ASan+UBSan: Canyonlands headless, 1,000 frames, no reports (was 15
  locations). End state identical to the Clang native run and to the run
  before this change.
- MinGW i686 Debug and standalone Release build. Unit tests: 13 of 14 pass;
  std fails with the 2 known stdConffile tests. std.System: 61 tests,
  2 known failures, 2 ignored. Regression run: 0.0025, OK.
- Not run: MSVC DX6/DX9 builds, host check.
sithCog_CogStatus checked index > numCogs, so index == numCogs read
one element past aCogs. Adds the first sithCog tests to sithTests:
the boundary index, an index past the array, and the last valid
index.

Validation (MinGW-w64 i686 GCC 13, DX9 Debug, under Wine 10):
- Passed: sith 41/41, including the 3 new sithCog tests. With the
  old bound, TestRejectsIndexEqualToNumCogs fails (expected 0,
  was 1).
- Unit suites otherwise unchanged: 12 of 13 pass; std fails the 2
  known stdConffile tests.
- Not run: MSVC, DX6, Release, guard and QOL profiles.
@seelenkuchen
seelenkuchen marked this pull request as ready for review November 2, 2026 22:38
@smlu
smlu merged commit 172becd into smlu:develop Nov 8, 2026
20 checks passed
@smlu

smlu commented Nov 8, 2026

Copy link
Copy Markdown
Owner

Thanks for providing these fixes!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants