Skip to content

linbo-remote: keep a record of finished runs #179

Open

Description

@TomlDev

Follow-up to linuxmuster/linuxmuster-api#43 (item 2). The maintainer noted there that a listing of finished linbo-remote runs has to be implemented in LINBO first. Verified against main at 40ecef0.

For each host, sendCmds() in linbo_remote_cli.py starts a tmux session named <hostname>.linbo-remote and pipes its pane to $LINBOLOGDIR/<hostname>.linbo-remote with cat >. The remote script built by renderRemoteScript() ends with exit $RC. After the run:

  • only the tmux session records that the run happened, and the session is gone once the script exits
  • the commands, start time, end time and exit status are not written anywhere
  • the next run on the same host overwrites the log, because cat > truncates it

As a result, no consumer (linuxmuster-api, webui, edulution-ui) can list the runs that finished. A run that starts and finishes between two polls of tmux list-sessions is never seen.

Asked for: one record per run, persisted by linbo-remote, for example a JSON line appended to a file in $LINBOLOGDIR with:

  • hostname
  • commands and mode (direct -c / onboot -p)
  • start and end time
  • exit status ($RC)
  • the path of that run's log

It would also help to write each run's log to its own file (for example with a timestamp in the name) instead of overwriting the per-host file. linuxmuster-api could then expose GET /v1/linbo/sync/runs.

Activity

  1. HappyBasher commented on Oct 1, 2026

    @HappyBasher
    Member

    Thanks for the detailed write-up, agreed on both points. Proposal for the per-run log name:

    <hostname>_linbo-remote_<timestamp>.log

    with timestamp = %Y%m%d%H%M%S (seconds included). With minute resolution two runs on the same host within one minute would write to the same file again, which is exactly the truncation this issue is about. The name also does not collide with the *_linbo.log glob used by update-linbofs, and it follows the existing <host>_<file> convention from rsync-pre-download.sh.

    Notes on the implementation:

    • The end time and exit status are only known inside the remote script from renderRemoteScript(), and the tmux session is gone once it exits. So the script itself should append the JSON line, ideally via trap ... EXIT so that an early abort also leaves a record. sendCmds() passes in the metadata known up front (hostname, commands, mode, start time, log path).
    • tmuxSessionName() currently doubles as the log basename. That has to be decoupled from the log name.
    • Retention (pruning old per-run logs) is better handled in a separate follow-up.

    I would split this into two small PRs: first the per-run log file, then the run record (JSONL).

  2. HappyBasher commented on Oct 1, 2026

    @HappyBasher
    Member

    Implementation is up as three small, stacked PRs (each one is based on the previous branch; retarget to main as they get merged):

    1. feat(linbo-remote): write one logfile per run #183 - one logfile per run: <hostname>_linbo-remote_<YYYYmmddHHMMSS>.log, so a new run no longer truncates the previous log.
    2. feat(linbo-remote): append a JSON record per finished run #184 - run record: the remote script appends one JSON line per run to $LINBOLOGDIR/linbo-remote_runs.jsonl (hostname, mode, commands, dry_run, start, end, rc, log) from an EXIT trap. An aborted run (killed tmux session) is recorded with its 128+signal status (e.g. 129).
    3. feat(linbo-remote): prune old per-run logs, rotate the run record #185 - retention: per-run logs older than 90 days are pruned when a new run starts; the run record is rotated monthly via logrotate.

    All three were verified against a live server and client (success, failing command, killed session, pruning).

    Not covered: onboot (-p) runs. The server only queues them and never learns their end time or status, so there is nothing to record on its side. If a "queued" record for those is wanted, that would be a separate change.

    With this, linuxmuster-api can list runs from linbo-remote_runs.jsonl (plus the rotated .1.gz files for older history).

  3. TomlDev commented on Oct 1, 2026

    @TomlDev
    ContributorAuthor

    Thanks for the three PRs. What a consumer (linuxmuster-api GET /v1/linbo/sync/runs, edulution-ui) needs from the record, beyond what #184 writes:

    • run_id: one id per linbo-remote invocation, shared by all host lines of that invocation, so a room/group run can be shown as one run with a per-host outcome.
    • user: who started it (SUDO_USER/USER); linuxmuster-api can additionally fill in the API user itself.
    • Timestamps in ISO 8601 with a colon offset (+02:00); %z yields +0200, which strict parsers reject.
    • A "v" schema field and a documented rc meaning (0 for a backgrounded start/reboot only means dispatched).
    • Running and lost runs are not in the file (record written at exit only). A start line or documenting the tmux join would remove the guess.
    • Offline hosts that were skipped are not recorded.
    • The stable <host>.linbo-remote name is still read by linuxmuster-api, webui7 and the satellite (see review on feat(linbo-remote): write one logfile per run #183).

    Planned API shape on the linuxmuster-api side (#46): GET /v1/linbo/sync/runs?host=<name>&since=<ISO8601>&limit=<n>&offset=<n>, newest first, school-filtered like /sessions, reading linbo-remote_runs.jsonl and the rotated .N.gz until since is covered; GET /v1/linbo/sync/runs/{run_id} and GET /v1/linbo/sync/runs/{run_id}/hosts/{hostname}/log for the per-run log (404 once pruned). Onboot (-p) runs stay out of the record, so the pending-onboot list (#44) remains the only source for them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions