Repository navigation
Conversation
propagateItem's deferred f.Close() assigned to a local err that was never part of the return values, so a failed close of the parent's lock file went unnoticed while the walk carried on and the parent was left with a possibly stale treesize/tmtime. Naming the results makes the close error reach Propagate, which logs it and aborts the walk. The async propagator had the same assignment inside a void function, where it could never be observed. Log the close error there instead, which is what the surrounding comment promises.
The posix and decomposed drivers use the pkg fork of decomposedfs, whose propagators threw away the error from unlock(), which closes the node's .lock file (xattrs, messagepack and hybrid backends). A failing close on NFS was therefore invisible. Log it instead of returning it: the lock file is created empty and never written, so no data is at stake, and changing propagation semantics is out of proportion to what is lost. Already-closed errors are filtered out because the async propagator releases the lock early and the deferred unlock then runs a second time.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four places in the two copies of decomposedfs throw away the error returned when
releasing a parent node's lock during treetime/treesize propagation. This makes
them observable.
Why
Propagation opens the parent's lock file, and both copies have code that tries
to report a failing close/release — but the reporting is inert:
pkg/storage/utils/decomposedfs(legacy copy:ocisdriver, datagateway/ocs)tree/propagator/sync.go—propagateItemhad unnamed results, so the deferrederr = cerrassigned to the localerrdeclared byf, err := lockedfile.OpenFile(...)after the return values had already been fixed. The error never reached the caller,
so the comment ("always log error if closing node fails") and the
if err == nilguard were aspirational. Naming the results makes the close error reach
Propagate,which logs it and returns it. (
f, err :=had to becomef, err =because namedresults live in the function body scope, leaving no new variable for
:=.)tree/propagator/async.go— the same assignment sits inpropagate, which returnsnothing, so it could never be observed. Replaced with the log the sibling early-release
path already emits (
Failed to close node and release lock).pkg/storage/pkg/decomposedfs(maintained fork:posix,decomposed,decomposeds3)tree/propagator/sync.goandtree/propagator/async.go—defer func() { _ = unlock() }()discards the error from closing the
.mlockfile (xattrs / messagepack / hybrid backends).tree/propagator/async.goearly release —_ = unlock()likewise discarded it. Theos.ErrClosedfilter is load-bearing here: the async propagator releases the lock early,so the deferred unlock always runs a second time and sees
ErrClosedon the happy path.Behaviour change
returns an error from
Propagate, where previously the walk continued silently. Expect moresurfaced errors in the operations that trigger propagation; the underlying cause (I/O /
metadata failure) was already there, just invisible.
Why the pkg fork logs instead of returning
The lock file is created empty and never written, so there are no dirty pages to lose — a
failing close signals an I/O error on the handle, not lost data. Aborting propagation (and
failing the upload/copy that triggered it) would be out of proportion, so the error is logged.
The utils copy keeps returning it because that code path was already written to do so.