 spanning 8–32 input frames.
Model
#Text
LLM Params
Frames
Short
Medium
Long
Overall
Gain
Proprietary LVLMs
GPT-4o (OpenAI, 2024)
–
–
384
80.0
70.3
65.3
71.9
–
Gemini-1.5-Pro (Team et al., 2024)
–
–
0.5 fps
81.7
74.3
67.4
75.0
–
Open-Source LVLMs
Video-LLaVA (Lin et al., 2024a)
–
7B
8
44.6
38.3
35.8
39.6
–
Video-LLaVA + EC-RAG
2.0K
7B
8
50.1
44.6
43.4
46.0
+6.4
LLaVA-NeXT-Video (Zhang et al., 2024d)
–
7B
16
49.4
43.0
36.7
43.0
–
LLaVA-NeXT-Video + EC-RAG
2.0K
7B
16
55.8
53.2
52.5
53.8
+10.8
Long-LLaVA (Song et al., 2024)
–
7B
32
60.3
51.4
44.1
52.0
–
Long-LLaVA + EC-RAG
1.9K
7B
32
67.6
60.3
60.0
62.6
+10.6
LLaVA-Video (Zhang et al., 2024e)
–
7B
32
75.7
59.6
52.1
62.4
–
LLaVA-Video + Video-RAG (Luo et al., 2024)
2.0K
7B
32
70.9
63.2
59.0
64.3
+1.9
LLaVA-Video + TV-RAG (Cao et al., 2025)
2.0K
7B
32
72.4
64.1
59.8
65.4
+3.0
LLaVA-Video + EC-RAG
2.0K
7B
32
77.8
64.3
60.4
67.5
+5.1
LLaVA-Video (Zhang et al., 2024e)
–
72B
32
78.0
63.7
59.6
67.1
–
LLaVA-Video + EC-RAG
2.1K
72B
32
81.9
72.4
72.7
75.7
+8.6
4. Experiments
4.1. Datasets
We assess EC-RAG on three widely adopted long-video benchmarks that span diverse durations, domains, and reasoning demands.
Video-MME (Fu et al., 2025) offers a systematic testbed for evaluating multimodal comprehension over real-world video content.
Its 900 videos are grouped by length into short (<< 2 min), medium (4–15 min), and long (30–60 min) subsets, yielding 2,700 multiple-choice questions across 12 task types—ranging from object recognition and counting to temporal reasoning and OCR—and 30 sub-categories drawn from six broad domains including knowledge, sports, and film.
This breadth makes Video-MME particularly suitable for stress-testing whether a retrieval-augmented pipeline can supply the right modality of evidence for each question type.
MLVU (Zhou et al., 2024) assembles nine evaluation tasks from videos of 3 min to 2 h (mean ≈\approx 12 min), requiring models to sustain reasoning over extended temporal ranges
LongVideoBench (Wu et al., 2024) focuses on multimodal retrieval and compositional reasoning within lengthy footage.
It provides 6,678 human-written multiple-choice questions spanning 17 thematic categories, testing a model’s ability to pinpoint and integrate evidence scattered across long video timelines.
4.2. Implementation Details
All experiments were conducted on NVIDIA RTX 4090 and A800 80 GB GPUs.
We adopt LLaVA-Video-7B (Zhang et al., 2024e) as the primary LVLM backbone, with three additional 7B architectures and the 72B variant evaluated in Table 1 to verify cross-architecture and cross-scale generalizability.
Detection requests produced during query decoupling are post-filtered with spaCy to retain only concrete, CLIP-responsive entities.
Both the CLIP similarity threshold and the FAISS (Johnson et al., 2019) retrieval threshold are fixed at τ=0.3\tau=0.3, with IndexFlatIP as the similarity backend.
Videos are segmented into 30-second windows for event chain construction.
Speech is transcribed by Whisper-Large (Radford et al., 2023), on-screen text is recognized by EasyOCR (JaidedAI, 2023), open-vocabulary detection is handled by APE (Shen et al., 2024), and dense text retrieval is performed by Contriever (Izacard et al., 2021).
Our ablation suite is built around LLaVA-Video-7B at 32 frames, whose moderate context window makes it a practical testbed for probing the effect of individual components and hyper-parameters.
Video-MME.
To verify that EC-RAG generalizes across architectures, model scales, and frame budgets, we integrate it into four open-source LVLMs spanning 8–32 input frames, as reported in Table 1. EC-RAG yields consistent improvements for every backbone, with gains most pronounced on long videos where uniform sampling alone cannot cover the query-relevant content. The benefit is especially evident on frame-constrained backbones, confirming that event-chain retrieval compensates effectively for sparse visual input. The pipeline injects approximately 2 K tokens of 