Skip to content

fix(posix): stop on-disk deletes and moves from resetting tree sizes to 0 - #849

Open
NickWalters wants to merge 1 commit into
opencloud-eu:mainfrom
NickWalters:fix/posix-warmup-clean-dir-treesize
Open

NickWalters wants to merge 1 commit into
opencloud-eu:mainfrom
NickWalters:fix/posix-warmup-clean-dir-treesize

Conversation

@NickWalters

@NickWalters NickWalters commented Oct 6, 2026 •

Copy link
Copy Markdown

Fixes #854. Part of opencloud-eu/opencloud#3109, together with #853.

Problem

On a posixfs storage with the watcher on, deleting or moving a folder directly on disk can reset the tree size of other folders to 0. The space usage drops with it. CheckQuota reads that same number, so a space with a quota then accepts uploads past its quota until OpenCloud restarts.

This affects setups where other tools write to the storage, which is the main reason to run posixfs. It needs nothing unusual, only STORAGE_USERS_POSIX_WATCH_FS.

I hit it on OpenCloud 8.1.0. I deleted a test folder in my personal space with rm -rf. A sibling folder with 52 GB in it dropped to a tree size of 0, and the space root went from 52 GB to 911 bytes. A restart put both back, because the startup scan walks everything.

How to reproduce

  1. On a posixfs space with the watcher on, create two folders with a file in each.
  2. Restart OpenCloud. The startup scan marks every folder as clean.
  3. Delete one folder on disk with rm -rf.
  4. The other folder now has a tree size of 0, and the space root has lost its size too.

Moving a folder on disk (mv a b) resets the moved folder to 0 in the same way.

Cause

With onlyDirty set, WarmupIDCache skips a clean folder, but it records a size of 0 for it first. After the walk, it writes that 0 as the folder's tree size and leaves the folder out of its parent's sum.

The watcher runs this walk on the parent after a folder is deleted on disk. A folder moved on disk is not marked dirty, so the walks that follow the move start at a clean root and reset that folder as well.

Fix

A clean folder now adds its stored tree size to its parents and keeps its own value. If a clean folder has no valid tree size, it is walked as before. A clean root writes nothing.

Only the onlyDirty walks that the watcher runs are affected. The startup scan and moves through the API pass onlyDirty=false, so they behave as before.

Tests

Five new specs. All of them fail on main and pass with this change:

  • warmup_test.go calls WarmupIDCache directly: a clean folder next to a dirty one, a clean folder with an invalid tree size, and a clean root.
  • tree_test.go goes through the watcher: it deletes a sibling of a clean folder, and it moves a clean folder on disk. Both specs run a full warmup first, because a folder is only clean after a walk has passed over it. The existing specs never reach that state, which is probably why this wasn't caught before.

go test -race ./pkg/storage/fs/posix/... passes locally (the tree package five times in a row), and golangci-lint is clean on the package.

Notes

…irty ones

With onlyDirty set, WarmupIDCache skipped a clean directory but still
recorded a size of 0 for it, and then wrote that 0 as its tree size and
left it out of its parent's sum. Deleting a directory on disk triggers
this walk on the parent, so every clean sibling lost its tree size. A
moved clean directory is the walk root itself and was reset to 0 the
same way.

Count the stored tree size of a clean directory for its parents and leave
its own value alone. A clean directory without a valid stored tree size
is walked instead.
@NickWalters
NickWalters force-pushed the fix/posix-warmup-clean-dir-treesize branch from b0c5cc8 to 1359ff4 Compare October 6, 2026 04:39
@NickWalters NickWalters changed the title fix(posix): keep the tree size of clean directories when warming up dirty ones fix(posix): stop on-disk deletes and moves from resetting tree sizes to 0 Oct 6, 2026
@NickWalters

Copy link
Copy Markdown
Author

Hi @aduffeck, could you approve the CI run for this one? It fixes tree sizes, and with them quota checks, dropping to 0 after a folder is deleted or moved on disk. I hit it on 8.1.0.

Could you also add the Type:Bug label so it shows up in the release notes? #796 and #797 were merged without a label, and neither is listed in the v2.51.0 notes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

posix: deleting or moving a folder on disk resets other folders' tree size to 0

2 participants