A benchmark for information-dense video

Understand the frames
others skip.

DIVE-Bench tests video-language models when evidence is not confined to a few keyframes—from long-form educational content to precise high-motion trajectories.

Read the paper Code forthcoming Get dataset

Haichao Zhang1, Wenhao Chai2, Shwai He3, Ang Li3, Yun Fu1

1Northeastern University   2Princeton University   3University of Maryland

00:0032:14
Dense evidence over time A signal varies across a long timeline, with sampling markers retaining evidence throughout the video. evidence is distributed—not episodic
Dense evidence
requires temporal coverage GRT
reuses what stays static
317long-form source videos
634educational QA items
1,000high-motion preview items
—evaluated LPM methods
01 / Benchmark

Two tracks. One missing capability.

Most video benchmarks reward sparse glimpses. DIVE-Bench isolates the harder case: details accumulate over long timelines, or change faster than sparse sampling can preserve.

A

Educational dense video · LPM

Read, listen, and reason across long-form video.

Questions target visible text and spoken subtitles distributed across educational videos. The public split contains 634 evaluation items, with two prompts per source video.

  • OCR
  • Subtitles
  • Long-form
  • Open-ended QA
Primary rankingOpen MOS ↑
B

High-motion dense video · preview

Track fine motion at frame-level precision.

Models predict a hand trajectory over a semantic 3×3 image grid. The current result snapshot evaluates 1,000 examples from the 3,243-example test split; public mirror access will accompany the evaluation-code release.

  • Trajectory
  • 3×3 grid
  • Transitions
  • High FPS
Primary rankingGrid Accuracy ↑
DIVE-Bench overview showing dense video question answering and Gated Residual Tokenization
Figure 1. DIVE-Bench couples information-dense evaluation with a token-efficient model pathway. Source: project paper.
02 / Method

Spend computation on what changed.

Gated Residual Tokenization (GRT) treats video more like a codec: preserve a full visual reference, then selectively recompute residual content while consolidating repeated scene semantics.

Inter-tokenization

Motion-aware gating reuses selected patch projections for visual regions that remain stable from frame to frame.

Intra-tokenization

Semantic-scene merging reduces redundant context after visual encoding while retaining dynamic evidence.

Measured efficiency

Recompute ratio measures patch-projection reuse; sampling density (fps) is sampled frames divided by source duration, not end-to-end throughput.

03 / Leaderboard

Public results, inspectable by metric.

Switch tracks, search models, filter sources, and sort any metric. A dash means that the current public artifact did not report that field. For the LPM default rank, methods without Open MOS are placed after all scored methods, then ordered by Token F1.

Result snapshot17 Aug 2026
DIVE-Bench Educational Dense Video leaderboard

Showing — methods

Default rank: Open MOS (reported first; missing last), then Token F1

How to read the table

Metric glossary

There is no hidden composite score. Each track exposes its task metric and supporting quality or efficiency measures.

Result provenance. This static snapshot was generated from 20 evaluation artifacts on 17 August 2026. LPM Open MOS uses Qwen/Qwen3-VL-32B-Instruct as the open text judge. Re-run details and exact model arguments will ship with the forthcoming evaluation-code release.

04 / Protocol preview

Evaluation recipe, pending code release.

The integration follows LMMS-Eval conventions and uses an explicit eight-frame budget with deterministic generation. Commands are shown for protocol inspection; the evaluation repository and High-Motion mirror are not public yet.

setup.sh

# Evaluation code release in progress.
# The pinned installation recipe will include:
conda create -n dive python=3.10 -y
conda activate dive
python -m pip install qwen-vl-utils
hf auth login

Protocol notes

  1. 01 Code repository and High-Motion mirror: release forthcoming.
  2. 02 Keep model-specific frame budgets explicit in --model_args.
  3. 03 Save sample logs and aggregate summaries for auditable submissions.

Release status
Paper and LPM data are available; pinned code and the High-Motion mirror are being prepared.

05 / Cite

Build on DIVE-Bench.

If the benchmark or GRT is useful in your research, please cite the project paper.

@article{zhang2025dive,
  title   = {Dense Video Understanding with
             Gated Residual Tokenization},
  author  = {Zhang, Haichao and Chai, Wenhao and
             He, Shwai and Li, Ang and Fu, Yun},
  journal = {arXiv preprint arXiv:2509.14199},
  year    = {2025}
}