Skip to content

feat(linbo-remote): prune old per-run logs, rotate the run record - #185

Merged
HappyBasher merged 1 commit into
7.4from
feat/linbo-remote-log-retention
Oct 1, 2026
Merged

HappyBasher merged 1 commit into
7.4from
feat/linbo-remote-log-retention

Conversation

@HappyBasher

Copy link
Copy Markdown
Member

Part 3 of #179. Stacked on #184 (which is stacked on #183); retarget to main once those are merged.

  • pruneRunLogs() deletes per-run logs (<host>_linbo-remote_<timestamp>.log) older than 90 days (RUN_LOG_RETENTION_DAYS) when a new run starts. Best effort, only touches files matching the per-run name.
  • logrotate can't do this: per-run logs have unique names, so it would never delete the small ones, and its size 50k rule would rotate a large one into an empty file of the same name. The existing stanza therefore becomes *[!0-9].log, which leaves them out (all other logs are still matched, checked with logrotate -d on a live server).
  • New stanza rotates linbo-remote_runs.jsonl monthly, 12 kept, compressed.
  • Verified on a live server: a 200 day old per-run log was removed by the next run, an equally old rsync-*.log stayed.

Refs #179

🤖 Generated with Claude Code

Per-run logs have unique names, so logrotate never rotates (and thus
never deletes) the small ones, and its size rule would rotate a big one
into an empty file of the same name. Keep them out of the existing
*.log stanza with *[!0-9].log and let linbo-remote delete per-run logs
older than 90 days when a new run starts (pruneRunLogs()).

Rotate the run record linbo-remote_runs.jsonl monthly, keeping 12.

Part 3 of #179.

Signed-off-by: thomas@linuxmuster.net
Assisted-by: Claude
@HappyBasher
HappyBasher force-pushed the feat/linbo-remote-log-retention branch from 4d79828 to dd53957 Compare October 1, 2026 10:20
@HappyBasher
HappyBasher force-pushed the feat/linbo-remote-run-record branch from 352dbcc to 6f13ee0 Compare October 1, 2026 10:20
@HappyBasher
HappyBasher changed the base branch from feat/linbo-remote-run-record to 7.4 October 1, 2026 10:21
@HappyBasher
HappyBasher merged commit b853c4c into 7.4 Oct 1, 2026
9 checks passed
@TomlDev

TomlDev commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Thanks for this. A review of the code, with the points a consumer (linuxmuster-api, webui7, edulution) runs into:

Line references are to the files on the PR branches (linbo_remote_lib.py = src/serverfs/usr/share/linuxmuster/linbo/linbo_remote_lib.py).

1. *[!0-9].log in logrotate.d/linuxmuster-linbo7:4 also stops rotating every other log whose name has a digit before .log, not just the per-run ones. Today none of the shipped names qualify, but a future foo2.log silently never rotates (no error, the file grows).
Suggestion: put the per-run logs in a subdirectory ($LINBOLOGDIR/linbo-remote/), which keeps the old *.log stanza unchanged and lets pruneRunLogs match on the directory instead of a regex; or keep the glob and add a spec that fails when a new *.log writer in the repo ends in a digit.

2. Record retention (12 months) is longer than log retention (90 days) (RUN_LOG_RETENTION_DAYS, rotate 12 in the jsonl stanza), so most log entries in linbo-remote_runs.jsonl.*.gz point to files that no longer exist. Either state it in the README (consumers must treat a missing log as normal) or align the two (rotate 3).

3. Legacy <host>.linbo-remote files are never cleaned up (no logrotate match, not covered by _RUN_LOG_RE). Add a one-time removal in the package postinst, or leave them and document it; with #183 point 1 they would otherwise stay readable but stale.

4. Rotated naming for consumers. Without dateext/delaycompress the rotated files are linbo-remote_runs.jsonl.1.gz ... .12.gz and renumber every month. Add this to the README ("read linbo-remote_runs.jsonl plus .N.gz in ascending N; do not cache by file name").

Checked and fine (no action): per-line appends are serialized by flock and >> (30 parallel runs gave 30 intact lines), ensure_ascii JSON plus shlex.quote keeps quotes, backslashes, newlines and unicode intact in the record, no copytruncate needed because the file is reopened per write, create 644 root root is readable by linuxmuster-api (service has no User=, runs as root), _RUN_LOG_RE does not match .1.gz, _linbo.log, rsync-*.log or the jsonl.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants