Skip to content

scheduler: run each job as one batch op and prune after success - #20

Draft
ash-iir wants to merge 1 commit into
mainfrom
fix-sched-batch
Draft

ash-iir wants to merge 1 commit into
mainfrom
fix-sched-batch

Conversation

@ash-iir

@ash-iir ash-iir commented Nov 19, 2026

Copy link
Copy Markdown
Owner

Backend #4. scheduler-tick called detach once per target package in a tight loop, so a job for all apps started every backup at once (each a root worker with tar/zstd/dumpsys), which is the overload backup-batch exists to avoid. It then ran prune_pkg right away, which counted the backup dirs still being written; if a new backup then failed, its trap removed it and the package was left with keep-1 good backups. The auto-backup observer did the same.

  • scheduler-tick: one op per due job. Backup jobs start backup-batch with {packages:[…], user, mode, dest, items, sched:true, keep}; restore jobs start restore-batch with {refs:[…]} (a pinned .ref is listed once). ran keeps one entry per package, all with the job's opid. The pruned field is gone from the output, since pruning now happens later in the worker (and is audited there by prune).
  • auto-backup observer: one backup-batch for all changed apps, with auto:true, keep.
  • backup_batch (worker): when the args carry a numeric keep, each app whose backup succeeded is pruned through helper.sh prune (same code and audit as manual prune). A failed app is not pruned. Manual batches have no keep and are unchanged.
  • prune_pkg: counts and deletes only dirs with manifest.json, so an in-progress or failed backup neither pushes a good one out nor gets deleted (also applies to manual prune).

Batches already run apps one at a time with the per-app timeout and thermal pacing, so scheduled runs get those too.

Testing

  • sh -n, mksh -n, busybox sh -n on both; shellcheck -s sh: worker gets one more SC2016 note (a jq $p in single quotes, same as the others in the file), helper has two fewer.
  • The real helper.sh scheduler-tick and prune, run under mksh R59, busybox 1.37 ash and dash against a sandboxed state dir with a stub worker: a 3-app backup job and a restore job (2 apps with backups, 1 without) give exactly two worker launches, one backup-batch with the 3 packages and keep:2, one restore-batch with the 2 refs; prune with keep 2 over 4 finished + 1 unfinished dir deletes the 2 oldest finished ones and leaves the unfinished one.
  • The real backup_batch with stubbed items: prune runs only after successful apps, not without keep, not with a non-numeric keep.
  • Not tested on a rooted device.

scheduler-tick and the auto-backup observer started one detached worker
per package, all at once, and pruned right after starting them. A job
for all apps therefore ran every backup in parallel, and pruning counted
the backups still being written, so a failed run could leave one good
backup fewer than keep.

- Each due job now starts one backup-batch (or restore-batch) op for
  its targets; the auto-backup observer starts one backup-batch.
- Scheduled and auto batches carry keep; backup_batch prunes an app to
  keep finished backups only after that app's backup succeeded.
- prune_pkg counts and removes only backups with a manifest.json.
- scheduler-tick no longer reports pruned refs (pruning happens later,
  in the worker, and is still audited).

Signed-off-by: ash-iir <254348575+ash-iir@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant