[Benchmark] VideoOdyssey-V

[ssiq]

Summary

This PR adds support for the VideoOdyssey-V benchmark, including:

• Automatic annotation and video download from Hugging Face.
• Official fixed-frame video sampling.
• 64-frame and 128-frame dataset configurations.
• Official prompt construction.
• Exact-match MCQ evaluation.
• Breakdown metrics by video category, question type, video duration, and certificate length.

Evaluation

A full 128-frame evaluation was run on all 1,618 VideoOdyssey-V questions using Qwen3.5-27B in thinking mode.

HF_HOME=cache/huggingface \
LMUData=cache/LMUData \
.venv/bin/python run.py \
  --data VideoOdyssey-V \
  --model qwen3.5-27b \
  --mode all \
  --api-mode \
  --stream \
  --reuse \
  --base-url "$URL" \
  --key "$KEY" \
  --temperature 1.0 \
  --top-p 0.95 \
  --top-k 20 \
  --max-tokens 65536 \
  --extra-body '{"presence_penalty":1.5,"enable_thinking":true}' \
  --api-nproc 64

Results

Result	Overall
Official VideoOdyssey result	44.60
This VLMEvalKit run	45.24

The VLMEvalKit result is 732 / 1,618 = 45.2410%.

The official score is reported for Qwen3.5-27B with 128 sampled frames on the VideoOdyssey project page and in the VideoOdyssey paper.