Skip to content

Fleet Tests Run on Stable/Main Despite Being Experimental — Missing Feature Gate + Platform Guard #2257

Closed

Description

@Trecek

Problem

Fleet is a develop-only experimental feature, but test_orphan_l3_reaping (and potentially other fleet tests) ran on the macOS-15 CI runner when PR #945 targeted stable. Two gaps:

  1. Feature gate gap: Fleet tests are not excluded by feature flag when running on stable/main branches. As an experimental feature, fleet components should not be tested (or accessible) outside develop.

  2. Platform guard gap: Fleet tests that depend on Linux-specific infrastructure (/proc via read_starttime_ticks, read_boot_id) lack sys.platform != "linux" skip markers.

Expected Behavior

  • Fleet tests should be deselected on stable/main CI runs via feature flag / experiment type gating
  • Fleet tests that depend on /proc-based identity (_linux_proc.py) should carry @pytest.mark.skipif(sys.platform != "linux", ...) as a secondary guard
  • The fleet feature should not be accessible (components, tools, CLI subcommands) when running on stable/main

Affected Test Files

At minimum: tests/fleet/test_fleet_e2e.py::test_orphan_l3_reaping

A broader audit of tests/fleet/ should identify all tests that:

  • Use read_starttime_ticks() or read_boot_id() (Linux-only)
  • Test fleet reaping, liveness, or dispatch lifecycle that touches /proc

Existing Precedents

Other Linux-only test modules are properly gated:

  • tests/execution/test_linux_tracing.py — module-level pytestmark = [pytest.mark.skipif(sys.platform != "linux")]
  • tests/execution/test_session_log_integration.py — same pattern
  • tests/fleet/test_liveness.py — uses pytest.skip("Not on Linux") inside individual tests

Context

  • Discovered via PR test #945 macOS-15 CI failure
  • Investigation report: .autoskillit/temp/investigate/investigation_macos_ci_failures_pr945_2026-05-08_160000.md
  • Companion issue for the reaper's /proc dependency: see fleet reaper identity check issue

Activity

  1. added
    in-progressIssue is being actively worked on
    stagedImplementation staged and waiting for promotion to main
    and removed
    in-progressIssue is being actively worked on
    on May 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    stagedImplementation staged and waiting for promotion to main

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions