Skip to content

Expand dat coverage and verify types with objdiff samples - #3588

Merged
ribbanya merged 1 commit into
doldecomp:masterfrom
ribbanya:pr/dat
Sep 30, 2026
Merged

ribbanya merged 1 commit into
doldecomp:masterfrom
ribbanya:pr/dat

Conversation

@ribbanya

Copy link
Copy Markdown
Collaborator

See tools/dat-cli/README.md.

The walk now reads every archive, including the packed Pl??AJ.dat animation files, and types 99.5% of their public symbols. Symbols the loaders don't record, because their names are built at runtime, are typed by name patterns in config/GALE01/dat_symbols.txt (fighter animations, the HSD exporter's *_joint/*_matanim_joint/... names, textures and palettes as byte arrays, SISLib text, stage parameters, scenes and Kirby's copy abilities). New annotations cover what C types can't: DAT_NULLTERM lists (also on typedefs, for nested ones like the envelope lists), DAT_SCRIPT command streams (fighter action scripts, with lengths from the game's own table), DAT_EXTENT on pointers, and DAT_TYPE on typedefs.

melee-dat symbols coverage reports how many relocations the walk explains (91.5%) and classifies the rest: gaps where a type is missing or wrong, data trailing a typed object where nothing points, and data nothing references at all. It can group them by archive, by the object they follow or by untyped field, and writes JSON shaped like objdiff's report.json or Markdown tables like this one:

Family Archives Publics Typed Relocations Explained Gap Trailing Unreferenced Reachable
Db 1/1 1 100.00% 854 0.70% 0 848 0 0.70%
Ef 35/36 36 97.22% 16693 0.31% 2 16521 118 0.31%
Gm 46/47 97 98.97% 30755 50.69% 283 14883 0 50.69%
Gr 71/71 4260 98.99% 216551 71.95% 2306 51885 6543 74.20%
If 7/7 18 100.00% 7184 81.92% 0 1299 0 81.92%
Ir 6/6 126 100.00% 4656 65.31% 0 1615 0 65.31%
It 1/1 1 100.00% 43705 89.11% 2 1926 2830 95.28%
Lb 6/6 12 91.67% 186 5.91% 2 173 0 5.91%
Mn 5/5 350 98.29% 33176 93.11% 8 2279 0 93.11%
Nt 4/4 4 100.00% 908 82.82% 0 156 0 82.82%
Pd 1/1 1 100.00% 1 100.00% 0 0 0 100.00%
Pl 6485/6485 6623 100.00% 1306218 97.35% 661 24392 9501 98.07%
Sd 15/15 15 100.00% 7332 100.00% 0 0 0 100.00%
Sm 1/1 1 100.00% 4198 0.24% 0 4188 0 0.24%
Tm 0/1 3 0.00% 0 100.00% 0 0 0 100.00%
Ty 352/352 492 100.00% 95067 96.47% 74 1323 1961 98.50%
Vi 11/11 20 100.00% 5563 26.53% 0 4087 0 26.53%
Total 7047/7050 12060 99.54% 1773047 91.55% 3338 125575 20953 92.64%

melee-dat samples checks the types themselves. It picks one instance of each type, and of each variant its tagged unions choose, then writes the instance's bytes as a target object and C initializers generated from the current types, compiled with the game's MWCC command. objdiff compares them, so a pointer typed as an integer, data in padding or a layout MWCC disagrees with shows up as a mismatch. Everything is generated under build/GALE01/dat, which is an objdiff project. All HSD and SDK samples match; one item type doesn't yet.
image

The DWARF build now compiles aurora's headers in their console layout rather than TARGET_PC, which had made GXBool an int and grown GXTexObj and GXTlutObj. dat_macros.h fills in the GX vertex functions aurora's console branch lacks, until they're upstreamed.

Fixed along the way: stage touch-line callbacks return hitbox data (lbColl_80008D30_arg1), not DynamicsDesc; Kirby's copy table is one entry per FighterKind, so every hats[] read was a fighter off, and each copy now has a struct; ftData_x20.x0 is the shield's root joint; FigaTrack has a byte at +7; HSD_RObjDesc_u lists a pointer member first so C initializers can hold every variant.

See tools/dat-cli/README.md.

The walk now reads every archive, including the packed Pl??AJ.dat
animation files, and types 99.5% of their public symbols. Symbols the
loaders don't record, because their names are built at runtime, are
typed by name patterns in config/GALE01/dat_symbols.txt (fighter
animations, the HSD exporter's *_joint/*_matanim_joint/... names,
textures and palettes as byte arrays, SISLib text, stage parameters,
scenes and Kirby's copy abilities). New annotations cover what C types
can't: DAT_NULLTERM lists (also on typedefs, for nested ones like the
envelope lists), DAT_SCRIPT command streams (fighter action scripts,
with lengths from the game's own table), DAT_EXTENT on pointers, and
DAT_TYPE on typedefs.

`melee-dat symbols coverage` reports how many relocations the walk
explains (91.5%) and classifies the rest: gaps where a type is missing
or wrong, data trailing a typed object where nothing points, and data
nothing references at all. It can group them by archive, by the object
they follow or by untyped field, and writes JSON shaped like objdiff's
report.json.

`melee-dat samples` checks the types themselves. It picks one instance
of each type, and of each variant its tagged unions choose, then writes
the instance's bytes as a target object and C initializers generated
from the current types, compiled with the game's MWCC command. objdiff
compares them, so a pointer typed as an integer, data in padding or a
layout MWCC disagrees with shows up as a mismatch. Everything is
generated under build/GALE01/dat, which is an objdiff project. All HSD
and SDK samples match; one item type doesn't yet.

The DWARF build now compiles aurora's headers in their console layout
rather than TARGET_PC, which had made GXBool an int and grown GXTexObj
and GXTlutObj. dat_macros.h fills in the GX vertex functions aurora's
console branch lacks, until they're upstreamed.

Fixed along the way: stage touch-line callbacks return hitbox data
(lbColl_80008D30_arg1), not DynamicsDesc; Kirby's copy table is one
entry per FighterKind, so every hats[] read was a fighter off, and each
copy now has a struct; ftData_x20.x0 is the shield's root joint;
FigaTrack has a byte at +7; HSD_RObjDesc_u lists a pointer member first
so C initializers can hold every variant.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@ribbanya ribbanya added tooling code-quality ai-assisted Utilizes a LLM to do the heavy lifting portability Improves non-matching builds labels Sep 30, 2026
@ribbanya
ribbanya marked this pull request as ready for review September 30, 2026 13:27
@ribbanya
ribbanya merged commit 5e8cad2 into doldecomp:master Sep 30, 2026
10 checks passed
@ribbanya
ribbanya deleted the pr/dat branch September 30, 2026 13:30
ribbanya added a commit to ribbanya/melee that referenced this pull request Sep 30, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-assisted Utilizes a LLM to do the heavy lifting code-quality portability Improves non-matching builds tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant