Repository navigation
Conversation
scheduler-tick and the auto-backup observer started one detached worker per package, all at once, and pruned right after starting them. A job for all apps therefore ran every backup in parallel, and pruning counted the backups still being written, so a failed run could leave one good backup fewer than keep. - Each due job now starts one backup-batch (or restore-batch) op for its targets; the auto-backup observer starts one backup-batch. - Scheduled and auto batches carry keep; backup_batch prunes an app to keep finished backups only after that app's backup succeeded. - prune_pkg counts and removes only backups with a manifest.json. - scheduler-tick no longer reports pruned refs (pruning happens later, in the worker, and is still audited). Signed-off-by: ash-iir <254348575+ash-iir@users.noreply.github.com>
5 of 6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backend #4.
scheduler-tickcalleddetachonce per target package in a tight loop, so a job for all apps started every backup at once (each a root worker with tar/zstd/dumpsys), which is the overloadbackup-batchexists to avoid. It then ranprune_pkgright away, which counted the backup dirs still being written; if a new backup then failed, its trap removed it and the package was left with keep-1 good backups. The auto-backup observer did the same.backup-batchwith{packages:[…], user, mode, dest, items, sched:true, keep}; restore jobs startrestore-batchwith{refs:[…]}(a pinned.refis listed once).rankeeps one entry per package, all with the job's opid. Theprunedfield is gone from the output, since pruning now happens later in the worker (and is audited there byprune).backup-batchfor all changed apps, withauto:true, keep.keep, each app whose backup succeeded is pruned throughhelper.sh prune(same code and audit as manual prune). A failed app is not pruned. Manual batches have nokeepand are unchanged.manifest.json, so an in-progress or failed backup neither pushes a good one out nor gets deleted (also applies to manual prune).Batches already run apps one at a time with the per-app timeout and thermal pacing, so scheduled runs get those too.
Testing
sh -n,mksh -n,busybox sh -non both;shellcheck -s sh: worker gets one more SC2016 note (a jq$pin single quotes, same as the others in the file), helper has two fewer.helper.shscheduler-tickandprune, run under mksh R59, busybox 1.37 ash and dash against a sandboxed state dir with a stub worker: a 3-app backup job and a restore job (2 apps with backups, 1 without) give exactly two worker launches, onebackup-batchwith the 3 packages andkeep:2, onerestore-batchwith the 2 refs;prunewith keep 2 over 4 finished + 1 unfinished dir deletes the 2 oldest finished ones and leaves the unfinished one.backup_batchwith stubbed items: prune runs only after successful apps, not withoutkeep, not with a non-numerickeep.