fix(filereplication): a full-size .part with no final file is not "present"

[pujitha24]

Motivation:
LocalBackend.StatFile falls back to the ".part" staging file when the
final file is absent, so the puller can resume a partial download. The
puller's presence rule (presentAtSize, used by the worker's pre-pull check
and the walks' self-origin check) is "stat succeeded and size equals the
manifest's". A crash after the last byte reached the .part but before the
rename leaves a staging file of exactly the manifest size, which then reads
as "already here": the worker skips the pull, the walks skip it, and nothing
renames it. Reads glob *.parquet, so the file's rows stay missing on that
node. The window is one rename wide and nobody is known to have hit it.

Approach:
Puller.statLocal now confirms the final file exists whenever the backend
reports a staged partial (StagedSize, implemented by LocalBackend only), and
otherwise reports not-found. The entry is then pulled again; WriteReader
truncates the stale .part and writes from zero. tryResumeFromPartial calls
StatFile directly and is unchanged, so resume of short partials is as before.
S3/Azure have no staging and are unaffected. presentAtSize stays the single
shared rule.

Validation:

• Added TestCatchUpPullsFullSizeStagingFile (walk with RepullMissingSelfOrigin)
  and TestWorkerPullsFullSizeStagingFile (worker pre-pull check), seeding a
  manifest-size .part with no final file. Both fail without the change
  (entry skipped as local/present) and pass with it; the fake backend gained
  StagedSize to mirror LocalBackend.
• go build ./... ; go vet and go test ./internal/cluster/... ./internal/storage/
  pass.
• golangci-lint on the package reports 4 errcheck findings, all in files this
  change does not touch. Not run: a real crash/kill against a live cluster.

Report: #963
Signed-off-by: Pujitha Paladugu 10557236+pujitha24@users.noreply.github.com
Assisted-by: claude-sonnet-5-5 (via Claude Code)

Fixes #963

[xe-nvdk]

Thanks, this is a clean fix for #963. I verified it three ways and pushed one fixup commit to your branch (details at the end).

What I checked

Unit tests fail before the fix. With main's puller.go swapped in, both new tests fail with skipped_local: 1, which is exactly the bug. With the fix they pass, including under -race. gofmt and vet are clean.

Live, pre-fix (released 26.09.2). Four-node per-node-storage cluster (3 writers + 1 reader, deploy/docker-compose/enterprise-local). Six flushes written on writer1, six replicas landed on writer2. Stopped writer2, renamed one replica X.parquet → X.parquet.part (1287 bytes, the manifest size, no final file), started it again. The catch-up walk enqueued all 6 entries, the worker skipped the broken one as present, and nothing logged it:

writer2 hour dir: 5 final, 1 .part
SELECT COUNT(*) FROM cpu on writer2: 25   (30 everywhere else)
"File pulled from peer" lines for the broken path: 0

Live, with the fix. Same volume, writer2 recreated on an image built from this branch:

File pulled from peer attempts=1 ... path=.../X.parquet peer=arc-writer1:9100 size_bytes=1287
writer2 hour dir: 6 final, 0 .part
SELECT COUNT(*) FROM cpu on writer2: 30

So the stale staging file is truncated by WriteReader and renamed away, as the description says.

Other StatFile callers don't have the same problem: edgesync/receive.go:261 calls Exists on the final path first, the tiering migrator sizes files a listing already saw, and the cold-side check runs on S3/Azure.

Fixup commit pushed to your branch

• RELEASE_NOTES_2026.09.3.md: created, with the entry for this fix and the credit line. 26.09.3 is the November patch release and this is its first entry.
• statLocal asserts storage.StagingInspector (the named contract LocalBackend implements) instead of an anonymous one-method interface; the fake backend now implements the whole contract. Behaviour is identical.
• A Debug line when a staged partial is found without its final file, so a re-pull is explainable from the log. (The present-skip at puller.go:1193 is silent at every level, which is how filereplication: a full-size .part left by a crash before the rename reads as present and is never finalised #963 stayed invisible; pre-existing, out of scope here.)
• LocalBackend.StatFile's doc comment still described the .part fallback as serving the pre-pull check's full-vs-partial decision, which is exactly what this PR retires. It now says the fallback serves the resume path and that presence confirms the final file.
• Two tests for the branch the diff left uncovered: a .part beside a present final file must still read as present, through the walk and through the worker, with no fetch. They pass before and after the fix; they pin the Exists == true branch so it can't be simplified away.
• Nit: the walk test asserted the worker's skipped_local counter; the walk's own is catchup_skipped_local. Switched so the assertion pins the path the test names.

I'll merge once CI is green on the new head.