Skip to content

log.Fatalf in lifecycle/storage adapters kills the entire benchmark harness on a single trial failure #10

Description

@magic-peach

What's happening

log.Fatalf (hard os.Exit(1)) is used across all 4 lifecycle/storage adapter files:

  • internal/runtime/lifecycle/cli_adaptor.go
  • internal/runtime/lifecycle/adapter.go
  • internal/runtime/storage/cli_adaptor.go
  • internal/runtime/storage/adapter.go

For example, CLIDeleteTask/CLICleanupTask call log.Fatalf if the delete command fails -- which is exactly the scenario when CreateTask never succeeded in the first place.

Why it matters

This compounds with the Cleanup-skipping issue: a single failed trial in a multi-trial benchmark plan currently kills the entire harness process, discarding every remaining trial in the run rather than just failing the one trial. For a tool whose purpose is running long, unattended benchmark sweeps, this is a real reliability problem.

Suggested direction

Replace log.Fatalf with returned errors throughout these files, so a failure in one trial's cleanup/delete path can be handled (logged, trial marked failed) without terminating the whole process. Given the number of call sites across 4 files, this probably makes sense as a few small PRs (e.g. one per file) rather than one large change.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions