 tuning with synthetic data.
arXiv preprint arXiv:2410.02713.
Cited by: Table 5,
Appendix A,
Figure 5,
Table 7,
Table 7,
Table 7,
Appendix C,
Table 1,
Table 1,
§4.2,
Table 2,
Table 3.
Zhou et al. (2024)
J. Zhou, Y. Shu, B. Zhao, B. Wu, S. Xiao, X. Yang, Y. Xiong, B. Zhang, T. Huang, and Z. Liu
Mlvu: a comprehensive benchmark for multi-task long video understanding.
arXiv preprint arXiv:2406.04264 2 (5), pp. 6.
Cited by: §1,
§4.1,
Table 2.
Supplementary Material
Appendix A Additional Ablation Studies
Frame Sampling Budget.
We investigate the sensitivity of EC-RAG to the number of sampled frames by evaluating LLaVA-Video-7B (Zhang et al., 2024e) under three frame budgets: 8, 16, and 32 on Video-MME (Fu et al., 2025).
As shown in Figure 5, EC-RAG consistently outperforms the baseline at every frame budget across all duration categories.
Notably, the baseline shows diminishing returns as the frame count increases—particularly on long videos, where doubling from 16 to 32 frames yields only marginal improvement, suggesting that simply adding more visual input cannot compensate for the lack of temporal and textual context.
In contrast, EC-RAG maintains substantial gains at all budgets, with the largest absolute improvement observed at 16 frames, indicating that event-chain retrieval is especially effective when the visual input is moderately sparse and the model most needs supplementary evidence to bridge temporal gaps.
Retrieval Threshold.
Table 5 investigates the effect of the retrieval similarity threshold τ\tau on both efficiency and accuracy. Lower thresholds (τ=0.0\tau{=}0.0–0.10.1) retain nearly all retrieved text, injecting up to 4.4 K supplementary tokens and pushing inference time to over 46 s per question, yet only achieve 64.1–64.7% overall. As τ\tau increases, the token budget and latency decrease steadily; at τ=0.3\tau{=}0.3, EC-RAG processes 2.4 K tokens in 18 s while reaching the peak overall accuracy of 65.1%, with the best short-video (73.4%) and long-video (61.2%) scores. This non-monotonic pattern indicates that unfiltered retrieval introduces noisy evidence that can mislead the model. Beyond τ=0.3\tau{=}0.3, accuracy degrades as useful context is progressively pruned: τ=0.5\tau{=}0.5 drops to 60.8% and τ=1.0\tau{=}1.0—which effectively discards all retrieved text (0.0 K tokens)—falls to 58.9%. We therefore adopt τ=0.3\tau{=}0.3 as the default, which reduces latency by roughly 65% compared to unfiltered retrieval while maximizing accuracy.
Cross-Backbone RAG Comparison.
Table 6 compares EC-RAG with Video-RAG (Luo et al., 2024) and TV-RAG (Cao et al., 2025) under two additional backbone settings. On Video-LLaVA-7B (Lin et al., 2024a) with 8 frames, all three RAG methods substantially improve over the baseline (39.6%), with EC-RAG achieving the highest overall accuracy of 46.0%, outperforming TV-RAG by +0.5 and Video-RAG by +1.0. The advantage is most pronounced on long videos (+0.8 over TV-RAG), where event-chain retrieval provides more temporally precise context than frame-level or segment-level approaches. On LongVA-7B (Zhang et al., 2024b) with 32 frames, the same trend holds: EC-RAG reaches 62.8% overall, surpassing TV-RAG (61.7%) and Video-RAG (60.1%). EC-RAG consistently leads across all duration categories under both backbones, confirming that structured event-level retrieval generalizes across different LVLMs and frame budgets.
Table 5. Performance with different retrieval thresholds on Video-MME (Fu et al., 2025) with LLaVA-Video-7B (Zhang et al., 2024e).
τ\tau
#Token
Time
Short
Medium
Long
Overall
0.0
4.4K
52s
72.4
61.6
60.0
64.7
0.1
4.0K
46s
71.8
60.5
59.9
64.1
0.2
3.5K
23s
70.3
60.4
57.8
62.8
0.3
2.4K
18s
73.4
60.7
61.2
65.1
0.4
1.6K
13s
70.1
60.6
57.5
62.7
0.5
1.2K
12s
68.9
58.2
55.3
60.8
1.0
0.0K
10s
66.5
56.4
53.7
58.9
Table 6. Comparison with other RAG methods across different backbones on Video-MME (Fu et al., 2025).
Model
Params
Frames
Short
Medium
Long
Overall
X=Video-LLaVA (Lin et al., 2024a)
7B