Skip to content

refactor(tests): run each test input once through the CLI loader - #269

Merged
zhen8838 merged 18 commits into
tile-ai:mainfrom
zhen8838:refactor/test-corpus
Oct 25, 2026
Merged

zhen8838 merged 18 commits into
tile-ai:mainfrom
zhen8838:refactor/test-corpus

Conversation

@zhen8838

Copy link
Copy Markdown
Collaborator

Why

  • Source tests took 314–509s on main (651s when runtime headers change). Most of that time came from running the same input many times: one scheduled HIR fixture was loaded 12 times, checked 9 times, and analyzed once per family; candidates and finalize also checked and analyzed again inside the library.
  • Tests built DSL programs from strings and from inline decorators, used several private loaders that accepted programs the CLI refuses (no Module, no Target), and parsed fixtures during collection in every xdist worker.

What

  • One CLI front end (checked_program) for analyze, schedule candidates and schedule finalize: parse --dim, load, require bound dims, check. candidates() takes a checked program; finalize() takes the memory AnalysisResult plus the authored function. PassManager records function identities before each pass, so a pass that lowers its input in place is still re-verified.
  • Corpus tests: one case per input, every stage once, all problems reported together (tests/core/_corpus.py). Schedule (121 cases), analysis (84, static table), parser (19 core / 23 nvidia). A case with its own checks is a @case subclass whose fields hold its expectations and whose guard() holds its checks.
  • Every DSL program used by a test is a fixture file in a Module that resolves a Target, loaded only through the CLI loader (load_authored_ir / load_namespace) inside the test body. import_dsl and the test-only loaders are gone; collection executes no fixture file.
  • Batches of "load one program, assert" tests became case tables; op typeinfer/eval/cost cases share one OpCase.
  • The 26 static-assert refusals compile in one nvcc run (28 → 3 runs).
  • Coverage measures lines only through sys.monitoring; CI runs source tests with -n 8.
  • Test files and functions renamed after what they check.

Contract

  • schedule finalize always writes a Module, also when it holds a single PrimFunction (docs/spec/cli.md).
  • A malformed, repeated or multi-extent --dim is refused before the source runs, for all three commands. A specialization error from schedule candidates / schedule finalize is now prefixed with the command name, as analyze already did.
  • finalize(analyzed, *, authored), candidates(module, entry, *, source) and ConvertHIRToTIR(analyzed=, authored=) replace the entry= / dims= forms; ConvertHIRToTIR no longer checks or analyzes (docs/spec/passes.md §7.1).

Risk

  • src/ grows by 11 lines: the shared front end and the finalize guard outweigh the three removed copies.
  • examples/*/tir/ keep the old single-PrimFunction output; they are generated and no test reads them. docs/tutorial/schedule.ipynb was already out of date with schedule.md.
  • Two programs stay inline in tests because moving them changes what they test (core/dsl/test_dsl_namespace.py::sub, nvidia/dsl/test_module_decorator.py::_RefusingRoot). nvidia/fixtures/integration/dynamic_cta_square.py is refused on load; its test is skipped until dynamic CTA lowering exists.

@zhen8838
zhen8838 merged commit 1393b19 into tile-ai:main Oct 25, 2026
2 checks passed
@zhen8838
zhen8838 deleted the refactor/test-corpus branch October 25, 2026 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant