Skip to content

fix(benchmarks): require AIPerf 0.12 for traffic bursts - #767

Open
afourniernv wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
afourniernv:codex/switch-1517-aiperf-traffic-burst
Open

afourniernv wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
afourniernv:codex/switch-1517-aiperf-traffic-burst

Conversation

@afourniernv

@afourniernv afourniernv commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

What

Update the benchmark version guard, install hints, documentation, and fixtures from AIPerf 0.11.0 to 0.12.0.

Why

SWITCH-1517

The traffic-burst profile already emits --request-rate-series, but the pinned AIPerf 0.11.0 rejects that option before sending a request. AIPerf 0.12.0 supports the existing command and schedule format.

Notes for reviewers

The behavior change is the shared supported-version constant. The other edits keep the install guidance and test fixtures aligned. There are no runtime or public API changes.

Live checks against the synthetic backend:

  • 0.11.0 reproduced the unsupported-option failure and sent zero requests.
  • 0.12.0 ran the unchanged traffic-burst command with 167 requests, zero errors, and all report formats generated.
  • Fixed-rate, concurrency-knee, and two-run aggregation controls also passed.

Local validation:

  • uv run pytest tests/test_aiperf_runner.py tests/test_routing_performance_report.py -q
  • uv run pytest tests/ --ignore=tests/e2e --ignore=tests/test_nemo_gym_run.py -q
  • uv run ruff check .
  • uv run mypy switchyard
  • cd docs && make publish

Summary by CodeRabbit

  • Documentation
    • Updated installation guidance to require AIPerf version 0.12.0.
  • Bug Fixes
    • Updated AIPerf version validation and export aggregation to support version 0.12.0.
  • Tests
    • Updated version-validation and export-aggregation coverage for the new AIPerf requirement.

Signed-off-by: Alex Fournier <afournier@nvidia.com>
@afourniernv
afourniernv requested a review from a team as a code owner September 17, 2026 22:41
@coderabbitai

coderabbitai Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6d2e4263-27f5-4cba-8777-06fb8271108f

📥 Commits

Reviewing files that changed from the base of the PR and between 4645dad and 3ca86f9.

📒 Files selected for processing (5)
  • docs/operations/soak_test.md
  • scripts/aiperf_runner.py
  • scripts/benchmark_routing_algorithms.py
  • scripts/run_local_soak_test.py
  • tests/test_aiperf_runner.py

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.


Walkthrough

Changes

AIPerf version update

Layer / File(s) Summary
Version contract and validation
scripts/aiperf_runner.py, tests/test_aiperf_runner.py
The supported version changes to 0.12.0. Tests reject 0.11.0 and use 0.12.0 for export aggregation.
Installation guidance
docs/operations/soak_test.md, scripts/benchmark_routing_algorithms.py, scripts/run_local_soak_test.py
Installation instructions now specify AIPerf 0.12.0.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~5 minutes

Merge Risk: ⚪ Minimal · up to 3ca86

The version update is consistently reflected across validation, installation guidance, documentation, and fixtures, so the change is ready to merge.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 4 files. (1 skipped: 1… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: benchmark traffic bursts now require AIPerf 0.12.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 4 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI

A rabbit checks the version line
Twelve-point-zero now aligns
Old eleven hops away
Tests keep watch through night and day
Fresh setup notes lead the way

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant