[Moenupa]

Before submitting

• I read the contributor guideline.

Changes

• Image preview dump (--save-preview / SAVE_PREVIEW, default 0): saves images from the first N formatted samples to the default save path so output blobs can be visually inspected
• New OutputActions.do_dump() writes preview files named <row>_<idx>.<ext>
• New dependency: filetype (pyproject.toml)
• Split dataset-loading out of dataio.py into a new module src/prep/api/load.py
• Hoisted import typer to module level in dataio.py (was previously lazy-imported inside two methods)
• I updated the tests.
• I updated documentation.

[Copilot]

🟡 Changes recommended

Preview dumping currently shares output paths in a way that can misreport split availability, and the new crop test depends on a non-repo file and has side effects instead of assertions.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (5)

Previously missed (4) — in code that hasn't changed since the last review.

src/prep/api/dataio.py:263

• do_dump() writes preview files directly under the split output directory (the same path scan_splits() uses to decide whether a split has been processed). If a user runs with --save-preview while saving is skipped (e.g., non-interactive mode), the presence of preview images will make ppls/status reporting incorrectly mark that split as available. Consider writing previews under a separate directory (e.g. <dataset_dir>/_preview/<split>/...) that does not overlap the dataset save path.

        dump_dir = self.default_save_path(
            pipeline, as_parquet=False, id_override=id_override
        )
        dump_dir.mkdir(parents=True, exist_ok=True)

tests/api/test_dataio.py:166

• This test assumes previews are written under the split directory. If do_dump() is changed to write under a dedicated _preview/<split> directory (to avoid confusing status scanning and save paths), update the expected dump location accordingly.

This issue also appears on line 184 of the same file.

        dump_dir = tmp_path / "cls" / "demo" / "train"
        assert sorted(p.name for p in dump_dir.iterdir()) == [

tests/api/test_dataio.py:219

• This assertion assumes previews are written under <fmt>/<id>/<split>/.... If previews are moved under <fmt>/<id>/_preview/<split>/..., update the expected location for the id_override case.

        assert [p.name for p in (tmp_path / "cls" / "alias" / "train").iterdir()] == [
            "000000_00.png"
        ]

src/prep/api/load.py:121

• load_file() uses path.suffix without normalizing case, so files like DATA.CSV will raise Unsupported file format even though they’re valid. Normalizing to lower() avoids surprising failures on case-preserving file systems.

    if path.suffix not in _FORMATS:
        raise ValueError(f"Unsupported file format: {path.suffix!r}")

    return load_dataset(_FORMATS[path.suffix], data_files=[str(path)], split="train")

tests/api/test_dataio.py:185

• This test assumes previews are written under the split directory. If do_dump() is changed to write under a dedicated _preview/<split> directory (to avoid confusing status scanning and save paths), update the expected dump location accordingly.

        dump_dir = tmp_path / "sft" / "demo" / "train"
        assert sorted(p.name for p in dump_dir.iterdir()) == [

• Files reviewed: 12/12 changed files
• Comments generated: 2
• Review effort level: Lite