Repository navigation
Show trace latency and runner wall time in guided reports - #334
Merged
Merged
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
AgentLoop replay self-testThis required repository check evaluates generated synthetic demo traces. It tests the replay and CI machinery; it does not show that this PR improves application performance. AgentLoop CI Report
Performance Gates
Trace inputsThis comparison includes synthetic data. It does not establish application performance gains from a pull request.
Metric Impact
Top Findings
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The guided retrieval pilot measured a large improvement inside the trace and a much smaller improvement across the runner process. The report exposed only the trace duration, which made the two scopes easy to confuse.
Changes
Validation
.venv/Scripts/python.exe -m pytest tests/test_improvement.py tests/test_improvement_init.py tests/test_guided_validation.py -q --basetemp D:/agentloop/runs/guided-runner-timing-focused --tb=short— 56 passed, 1 skipped. The Windows smoke and onboarding test initially decoded redirected UTF-8 with the system code page. Both subprocess readers now explicitly use UTF-8..venv/Scripts/python.exe -m pytest tests/test_improvement.py::test_timing_summary_reads_legacy_receipts_and_keeps_runner_time_out_of_gates tests/test_improvement.py::test_cli_runs_the_normal_entry_points_and_returns_quality_failure tests/test_improvement_init.py::test_complete_offline_onboarding_with_manual_edit_and_rejected_candidate -q --basetemp D:/agentloop/runs/guided-timing-final-focused -W error::pytest.PytestUnhandledThreadExceptionWarning— 3 passed after the UTF-8 fix and report wording changes..venv/Scripts/python.exe runs/bugfix_quality_checks.py— all applicable pre-commit checks passed. A concurrent wheel build briefly created a temporary source tree during one link check. Running the check after the build resolved it.uv build --out-dir runs/guided-timing-final-dist --quietanduv tool run --from twine twine check runs/guided-timing-final-dist/agentloop_profiler-0.2.3-py3-none-any.whl runs/guided-timing-final-dist/agentloop_profiler-0.2.3.tar.gz— both artifacts passed..venv/Scripts/python.exe -m mkdocs build --strict --site-dir D:/agentloop/runs/guided-timing-site— passed.runs/guided-timing-validation/venv/Scripts/python.exe -I scripts/smoke_installed_wheel.py --expected-version 0.2.3passed from a fresh environment with the locally built wheel.venv/Scripts/python.exe -I replay_pilot.pyandvenv/Scripts/python.exe -I check_final_reports.pyfromD:/agentloop/runs/guided-timing-validation. Browser rendering was checked withuv run --isolated --no-project --with playwright==1.65.0 python runs/guided-timing-validation/check_html.py..venv/Scripts/python.exe -m pytest -q --basetemp D:/agentloop/runs/guided-timing-full --durations=5— 2,808 passed and 109 skipped. This run collected tests before the UTF-8 test-reader fix, so it retained two reader warnings. The final focused rerun treated those warnings as errors and passed.4af75b0bcc2bc2a42ad10bc81592f14e67cfc02apassed all 23 runnable hosted checks. GitHub Pages deployment runs on main. The automatic CodeRabbit review was skipped under its existing eligibility policy and supplied no approval.Limits
The offline retrieval fixture measures an owned filesystem workload on one host. Its speedup describes that workload. External developer onboarding remains unmeasured. Runner wall time covers the child process boundary and excludes capture-side parsing, validation and report generation.
Substantial implementation and validation assistance was provided by Codex.