Repository navigation
fix(posix): stop on-disk deletes and moves from resetting tree sizes to 0 - #849
Open
NickWalters wants to merge 1 commit into
Open
NickWalters wants to merge 1 commit into
NickWalters wants to merge 1 commit into
Conversation
…irty ones With onlyDirty set, WarmupIDCache skipped a clean directory but still recorded a size of 0 for it, and then wrote that 0 as its tree size and left it out of its parent's sum. Deleting a directory on disk triggers this walk on the parent, so every clean sibling lost its tree size. A moved clean directory is the walk root itself and was reset to 0 the same way. Count the stored tree size of a clean directory for its parents and leave its own value alone. A clean directory without a valid stored tree size is walked instead.
NickWalters
force-pushed
the
fix/posix-warmup-clean-dir-treesize
branch
from
October 6, 2026 04:39
b0c5cc8 to
1359ff4
Compare
Author
|
Hi @aduffeck, could you approve the CI run for this one? It fixes tree sizes, and with them quota checks, dropping to 0 after a folder is deleted or moved on disk. I hit it on 8.1.0. Could you also add the |
This was referenced Oct 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #854. Part of opencloud-eu/opencloud#3109, together with #853.
Problem
On a posixfs storage with the watcher on, deleting or moving a folder directly on disk can reset the tree size of other folders to 0. The space usage drops with it.
CheckQuotareads that same number, so a space with a quota then accepts uploads past its quota until OpenCloud restarts.This affects setups where other tools write to the storage, which is the main reason to run posixfs. It needs nothing unusual, only
STORAGE_USERS_POSIX_WATCH_FS.I hit it on OpenCloud 8.1.0. I deleted a test folder in my personal space with
rm -rf. A sibling folder with 52 GB in it dropped to a tree size of 0, and the space root went from 52 GB to 911 bytes. A restart put both back, because the startup scan walks everything.How to reproduce
rm -rf.Moving a folder on disk (
mv a b) resets the moved folder to 0 in the same way.Cause
With
onlyDirtyset,WarmupIDCacheskips a clean folder, but it records a size of 0 for it first. After the walk, it writes that 0 as the folder's tree size and leaves the folder out of its parent's sum.The watcher runs this walk on the parent after a folder is deleted on disk. A folder moved on disk is not marked dirty, so the walks that follow the move start at a clean root and reset that folder as well.
Fix
A clean folder now adds its stored tree size to its parents and keeps its own value. If a clean folder has no valid tree size, it is walked as before. A clean root writes nothing.
Only the
onlyDirtywalks that the watcher runs are affected. The startup scan and moves through the API passonlyDirty=false, so they behave as before.Tests
Five new specs. All of them fail on main and pass with this change:
warmup_test.gocallsWarmupIDCachedirectly: a clean folder next to a dirty one, a clean folder with an invalid tree size, and a clean root.tree_test.gogoes through the watcher: it deletes a sibling of a clean folder, and it moves a clean folder on disk. Both specs run a full warmup first, because a folder is only clean after a walk has passed over it. The existing specs never reach that state, which is probably why this wasn't caught before.go test -race ./pkg/storage/fs/posix/...passes locally (the tree package five times in a row), and golangci-lint is clean on the package.Notes
WarmupIDCache, so whichever of the two merges second needs a small rebase. Both sides only add lines. I merged them together and the tests pass. I'm happy to rebase this one.