[Benchmark] Add MomentSeeker for long-video moment retrieval

[aliasgerovs]

What this adds

This PR adds MomentSeeker (https://arxiv.org/abs/2502.12558), a benchmark for finding moments in long videos (over 500 seconds on average). Each query asks for the time interval(s) in the video that match it, and there are three kinds of query:

Query type	Queries	What the model gets
Text (TMR)	1,000	a text query
Image (IMR)	400	a text query plus an image
Video (VMR)	400	a text query plus a short reference clip

The queries cover nine tasks, from describing an event to OCR and spatial relations. All three query types are supported.

Data: https://huggingface.co/datasets/avery00/MomentSeeker (CC BY-NC-SA 4.0, research use only).

How it works

• The annotations, the videos (a split archive of about 92 GB) and the query images and clips (1.6 GB) are downloaded from Hugging Face the first time and kept in LMUData/MomentSeeker.
• Dataset names: MomentSeeker (all 1,800 queries) and MomentSeeker-TMR, MomentSeeker-IMR, MomentSeeker-VMR, each with _1fps, _2fps, _64frame and _128frame settings.
• The prompts are the ones from the paper's appendix, including the line that gives the video length and the timestamps of the sampled frames. The query image is passed as "Image 1" and the reference clip as "Video 2".
• Scoring follows the paper. R@1: the first interval in the answer has IoU above 0.3 with one of the ground-truth moments. mAP@5: the average precision over the first five intervals, where each ground-truth moment can be matched only once. I also report R@1 at IoU 0.5 and 0.7 and the mIoU, overall, per query type and per task.

The MomentSeeker repo only ships evaluation code for retrieval models, not for generative models, so the metrics are written from the definitions in the paper and covered by unit tests (tests/test_momentseeker_metrics.py).

Results

Qwen3-VL-8B-Instruct with the same model settings as #1715, on MomentSeeker_1fps:

Query type	Queries	R@1	mAP@5
Text (TMR)	1,000	15.9	16.9
Image (IMR)	400	9.3	9.3
Video (VMR)	400	7.3	7.4
All	1,800	12.5	13.1

For reference, the paper's results table gives R@1 / mAP@5 of 8.1 / 8.0 for Qwen2.5-VL-7B and 17.2 / 16.9 for Qwen2.5-VL-72B (with 768 frames), so this 8B model lands between them. The image and video queries are clearly harder than the text ones. 59 answers were an empty list (no matching moment) and count as zero.

Checks

• The repo pre-commit hooks (flake8, isort, yapf and the others) pass on the changed files.
• pytest tests/test_momentseeker_metrics.py: 4 passed.
• pytest tests/test_temporal_grounding.py: 14 passed (13 from [Benchmark] Add TimeLens-Bench for video temporal grounding #1715 plus the multi-span IoU test).
• pytest tests/test_dataset_alias.py: 29 passed (the new datasets are in the shortcut manifest).

[aliasgerovs]

Now that #1715 is merged, I rebased this onto the latest main as well. The PR now has a single commit with only the MomentSeeker changes.