Skip to content

Make native QA sweeps non-conflicting and sub-100ms per immediate outcome #200

Description

@pathscale

Objective

Reduce ps-qa execution overhead so a full app/component sweep starts once and drives non-conflicting checks rapidly. One immediate UI outcome should generally complete in under 100 ms after host setup, with single-frame cases approaching 10 ms when the application responds that quickly.

Requirements

  • Report host/process startup and attach time separately from check execution.
  • Reuse one prepared host for compatible checks and surfaces; do not relaunch per outcome.
  • Partition state-mutating, isolated, manual-only, and truly conflicting checks explicitly instead of serializing everything by default.
  • Replace fixed sleeps/settle delays with event-, frame-, mutation-, or rendered-outcome readiness where possible.
  • Pipeline inspection reads and actions when ordering permits without hiding races.
  • Keep deterministic cleanup and restore shared state between groups.
  • Preserve strict outcome semantics; speed is not permission to turn checks into click-dispatched or node-present assertions.

Metrics and acceptance

  • Emit startup, preparation, action-to-outcome, cleanup, median, p95, and slowest checks.
  • Immediate local controls: median below 100 ms after setup; identify every slower case and whether the application or harness owns the delay.
  • Single-frame local cases should be capable of approximately 10–20 ms.
  • Full component/app sweep remains non-conflicting and fits the established 1–2 minute ceiling, with a concrete lower target derived from measured app response.
  • CI may use an explicit multiplier for overloaded runners, but local thresholds remain strict.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions