Repository navigation
fix(video): bypass frame extraction for raw LongVideoBench prompts - #1767
Open
rudycelekli wants to merge 1 commit into
Open
rudycelekli wants to merge 1 commit into
rudycelekli wants to merge 1 commit into
Conversation
Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LongVideoBench accepts nframe=0/fps=-1 for direct video input, but build_prompt() always enters frame extraction. Without sampling this raises UnboundLocalError before a frame is fetched; even sampled configurations unnecessarily open the decoder for raw-video models. Move frame extraction into the frame-model branch so video-model prompts use the original file directly.
Validation: a physical four-frame local MP4, the shipping dataset constructor with its local prepare_dataset extension, and actual BaseModel media validation/prompt consumption. Both raw-video cases fail on the base; sampled-frame and seven neighboring image controls pass. All10 cases pass after the fix. Raw-video prompts perform zero reader/frame calls; cached sampled-frame prompts retain metadata reads and actual decoded RGB image pixels. All nine pinned pre-commit hooks pass on materialized source/metadata.
The offline reader seam follows the primary Decord v0.6.0 len/FPS protocol using actual FFprobe metadata and FFmpeg decoding; this does not claim native Decord integration. No model/provider calls, downloaded datasets, subtitle-policy changes, optional lock-path integration, GPU runs, or benchmark accuracy claims. AI assistance: prepared with Codex and verified with owned native fixtures.
Hosted CI update: lint, API scoring and the completed Qwen OCR job pass. The API text check is red while remaining jobs are running/queued. GitHub's failure annotation reports that its self-hosted runner lost communication with the server; no execution logs are available. The underlying runner/machine condition is not verified, and no source-baseline reproduction is claimed. This is not fully green hosted CI. Text job.