Repository navigation
fix(measurements): align rgb_frames sampling with output ticks (#733) - #734
shobhitagnihotri69 wants to merge 1 commit into
Conversation
…an-Robotics#733) Add ':round=up' to the fps filter in rgb_frames so that the emitted frame is the one visible at-or-before each output tick, matching the expectation of source_log_times_for_sampled_frames and mediapipe_hands.
[Medium impact] The change affects sampled video measurements in a bounded workflow. The PR appears safe to merge, with two non-blocking improvements to the regression test. Findings
|
| ) as frames: | ||
| shown = [read_frame_index(f) for f in frames] | ||
|
|
||
| expected = [round(i * 30 / sample_fps) for i in range(len(shown))] |
There was a problem hiding this comment.
The expected list uses len(shown), so an empty result or a correct but shortened prefix passes. This lets a change that drops sampled frames escape the regression test. Build the expected list from the fixture's known three-second duration and sample_fps instead.
| expected = [round(i * 30 / sample_fps) for i in range(len(shown))] | |
| expected = [round(i * 30 / sample_fps) for i in range(int(3 * sample_fps))] |
| ) as frames: | ||
| shown = [read_frame_index(f) for f in frames] | ||
|
|
||
| expected = [round(i * 30 / sample_fps) for i in range(len(shown))] |
There was a problem hiding this comment.
Between-frame sampling stays untested
Every tested rate divides 30 exactly, hiding the difference between round() and the required at-or-before rule. At 4 fps, the first sample after zero falls at source position 7.5: the expected frame is 7, but this expression gives 8. Add a non-dividing rate and use the floor-based rule from source_log_times_for_sampled_frames to protect this case.
kstonekuan
left a comment
There was a problem hiding this comment.
Could you change Fixes #733 to Refs #733? This fixes the offset, but the second point in the issue (the last good picture repeated across a lost keyframe) is still open, and merging would close it.
Summary
rgb_frames(frames_per_second=...)now configures FFmpeg'sfpsfilter withround=up, ensuring that emitted frames correspond to the picture visible at or before each sampling tick rather than pictures recorded after it.Why
FFmpeg's default
fpsrounding isround=near, which selects frames up to half a sample period after the output tick (e.g., 14 source frames late at 1 fps on 30 fps video). Downstream callers likemediapipe_handsattribute sampled frames viasource_log_times_for_sampled_frames, which relies onround=up(the frame visible at-or-before each tick). Specifyingfps={frames_per_second:g}:round=upaligns the extracted RGB pixel content with the assigned log times across all sample rates.Refs #733
Validation
test_rgb_frames_sampling_aligns_with_output_ticksintests/test_ffmpeg.pytesting sample rates 1.0, 3.0, 10.0, and 30.0 fps against a barcode-numbered video: all 4 passed.uv run pytest tests/test_ffmpeg.py tests/test_mediapipe_hands.py tests/test_video.py tests/test_source_sampling.py tests/test_module_boundaries.py— 191 passed, 11 skipped.uv run ruff check --fix,uv run ruff format, anduv run ty check— all passed without errors.Checklist
uv run ruff check --fix,uv run ruff format, anduv run ty check.Need help on this PR? Tag
@codesmith-botwith what you need.