Skip to content

dat: validate archive inputs and unit completeness - #3589

Merged
ribbanya merged 3 commits into
doldecomp:masterfrom
itsgrimetime:codex/dat-sample-build-failures
Oct 1, 2026
Merged

ribbanya merged 3 commits into
doldecomp:masterfrom
itsgrimetime:codex/dat-sample-build-failures

Conversation

@itsgrimetime

@itsgrimetime itsgrimetime commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Reject missing or empty DAT archive directories during CMake configuration instead of successfully producing an empty objdiff project. Report the input path and extraction guidance.

Mark a sample unit complete only when every target symbol outside its samples is also inferred in the base, alongside the existing public-symbol, relocation, and walk checks. Check the symbol count first, then each symbol's file location and size, before inspecting the archive walks. A typed symbol start can still leave unexplained trailing data and a missing base definition.

Add a regression check against real generated objects, comparing inferred symbols by name and size. It uses MELEE_DAT_BUILD or the local DAT preset output, skipping the default when absent. No synthetic DATs or generated artifacts are committed. Document both behaviors.

Validation:

  • The real dat preset builds successfully with Clang 22.1.7 and LLD 23.1.1: all 838 archives, 2,201 generated C files, 838 base objects, and 765 objdiff units. The generated project opens in the GUI.
  • The regression check fails before the fix on missing DbCo tables. After rebuilding, 246 incorrect complete flags become false; all 6,327 inferred symbols in the remaining 37 complete units pass.
  • Cargo tests pass against real archives and the generated DWARF ELF. Only the existing root_witnesses test skips because it hardcodes an object under the separate ppc-dwarf preset directory.
  • samples report succeeds with objdiff-cli 3.8.2: 11,945/11,947 samples match, covering 99.9984% of sampled bytes; no extra base sections. The two existing sample mismatches are outside this change's scope.
  • Missing, empty, non-archive-only, and file-valued input paths fail configuration. Existing Ninja builds also fail on reconfiguration.
  • Clippy completes with three existing warnings. The new test's formatting, git diff --check, and commit hooks pass.

Human review is pending.

@itsgrimetime itsgrimetime added tooling ai-assisted Utilizes a LLM to do the heavy lifting labels Sep 30, 2026
@ribbanya

Copy link
Copy Markdown
Collaborator

You might want to hold off on this. I'm in the process of swapping to full cmake-ninja in master...ribbanya:melee:wip/dat.

@ribbanya

Copy link
Copy Markdown
Collaborator

And I don't want to introduce synthetic DATs at this time.

@ribbanya

Copy link
Copy Markdown
Collaborator

@itsgrimetime Hit my session limit and merged in #3590; feel free to continue on with what's still relevant.

@itsgrimetime
itsgrimetime force-pushed the codex/dat-sample-build-failures branch from 9da8d09 to 8b002f2 Compare October 1, 2026 04:54
@itsgrimetime itsgrimetime changed the title dat-cli: fail sample builds when compilation fails dat: reject missing or empty archive directories Oct 1, 2026
@itsgrimetime

Copy link
Copy Markdown
Collaborator Author

@itsgrimetime Hit my session limit and merged in #3590; feel free to continue on with what's still relevant.

Pivoted this one to work off that

@itsgrimetime
itsgrimetime force-pushed the codex/dat-sample-build-failures branch from 8b002f2 to aeef8df Compare October 1, 2026 16:29
@itsgrimetime itsgrimetime changed the title dat: reject missing or empty archive directories dat: validate archive inputs and unit completeness Oct 1, 2026
Comment thread tools/dat-cli/src/cmd/samples.rs Outdated
@itsgrimetime
itsgrimetime marked this pull request as ready for review October 1, 2026 17:04

@ribbanya ribbanya left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, this makes way more sense to look at. Later on we can try to reconstruct the dat files from source and then the completion flag will prompt a checksum between the built dat and the original.

@ribbanya
ribbanya merged commit 3a2a8c0 into doldecomp:master Oct 1, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-assisted Utilizes a LLM to do the heavy lifting tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants