fix(checkpoint): tolerate corrupt checkpoints.jsonl lines and write it atomically

[posthog]

Problem

• One bad line in checkpoints.jsonl stops all AI checkpoints for the base commit. Each checkpoint fails with IO error: expected value at line 1 column 1, so AI attribution for that commit is lost. Seen on one Windows install (CLI 2.1.2).
• read_all_checkpoints returns an error on the first line that does not parse. status and post-commit read the same file, so they fail too.
• append_checkpoint calls read_all_checkpoints().unwrap_or_default(). When the read fails, it writes back only the new checkpoint and silently drops all earlier checkpoints.
• Probable cause of the bad line: write_all_checkpoints truncates the file with File::create and rewrites it in place. A crash or a concurrent writer during the rewrite can leave junk bytes (for example NUL padding).

Origin

• Error tracking: issue
• First signal: 2026-10-12
• Inbox report: open
• Likely cause: ce859d0
• Task started by: auto-start, after the report was rated P2 and ready to fix

Changes

• Tolerant read: read_all_checkpoints reads raw bytes line by line. A line that does not parse (bad JSON, NUL padding, bad UTF-8) is skipped. A warn log records the file path, line number and parse error.
• Atomic write: write_all_checkpoints writes to a unique temp file (checkpoints.jsonl.<pid>.<n>.tmp), calls sync_all, then renames it over checkpoints.jsonl. If a step fails, it removes the temp file.
• No silent data loss: append_checkpoint now returns the read error with ?. A bad line no longer causes a read error, so only real IO failures get to the caller.
• The next successful write drops the bad line. Only one warning shows per bad line.

Case	Before	After
One bad line, then checkpoint mock_ai	every checkpoint fails	succeeds, AI lines attributed
append_checkpoint with a bad line	earlier checkpoints dropped	earlier checkpoints kept
Crash during rewrite	partial file	old file or new file, never partial

Warning

Each checkpoint write now calls fsync. This can add some latency per checkpoint on slow disks (mainly Windows). The write already rewrote the full file each time, so the change in I/O is small.

Tests

• New tests in tests/integration/repo_storage_unit.rs. All three fail without the fix:
  • test_read_all_checkpoints_skips_corrupt_lines: NUL padding, bad UTF-8 and truncated JSON lines are skipped. Valid lines are kept.
  • test_append_checkpoint_keeps_earlier_checkpoints_past_corrupt_line: earlier checkpoints are kept and the bad line is removed on rewrite.
  • test_ai_checkpoint_succeeds_with_corrupt_checkpoint_line: end to end. checkpoint mock_ai succeeds and the commit has the correct AI attribution.
• cargo clippy --all-targets -- -D warnings and cargo fmt pass.
• repo_storage_unit passes (18/18) in daemon mode.
• Full integration binary in this sandbox: 3052 passed, 81 failed. None of the failures are from this change:
  • 42 were daemon sync timeouts under load. They pass when run again with fewer threads.
  • 18 fail because the sandbox git wrapper blocks git commit (bash_tool_provenance, ignore_unit bare repo, commit_post_stats_benchmark, the doctor self-check).
  • 21 graphite tests need the gt CLI, which the sandbox does not have.
• Lib unit tests: 11 fail for the same blocked git commit reason (notes_api, notes_migrate, ci_context, repository). None of them touch checkpoints.
• Not run: macOS and Windows. CI covers them. The sandbox runs as root, so I ran the tests with CI=true to stop the superuser warning from breaking blame output checks.

Agent context

• I did not try to recover data from a bad line (for example by removing leading NULs). Skipping it is simpler. The other checkpoints in the file still give attribution.
• I did not reuse mdm::utils::write_atomic. It needs the full file in memory and uses a fixed .tmp name, so two concurrent writers could use the same temp file.

---

Created with PostHog Desktop from this inbox report.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}