EC-RAG: Event Chain Retrieval-Augmented Generation for Long Video Understanding
Report GitHub Issue
×
Title:
Content selection saved. Describe the issue below:
Description:
Submit without GitHub
Submit in GitHub
arXiv is now an independent nonprofit!
Learn more
×
Back to arXiv
Why HTML?
Report Issue
Back to Abstract
Download PDF
Abstract.
1 Introduction
2 Related Work
2.1 Video Understanding with Large Language Models
2.2 LLM-based Agents for Video Understanding
2.3 Retrieval-Augmented Generation for Video
3 Method
3.1 Query Decoupling
3.2 CLIP-Variance Guided Multimodal Extraction
3.3 Temporal Event Chain Construction
3.4 Evidence-augmented Multimodal Reasoning
4 Experiments
4.1 Datasets
4.2 Implementation Details
4.3 Ablation Studies
4.4 Qualitative Analysis
5 Conclusion
References
A Additional Ablation Studies
B Task-Type Breakdown
C More Qualitative Results
License: arXiv.org perpetual non-exclusive license
arXiv:2610.08674v1 [cs.CV] 06 Oct 2026
EC-RAG: Event Chain Retrieval-Augmented Generation for Long Video Understanding
Yuhao Qin
email: qinyuhaoswu@163.com
Affiliation: Northwestern Polytechnical University, Xi’an, China
,
Junbo Wang
email: jbwang@nwpu.edu.cn
Affiliation: Northwestern Polytechnical University, Xi’an, China
,
Yuke Li
email: liyuke@nwpu.edu.cn
Affiliation: Northwestern Polytechnical University, Xi’an, China
and
Yining Zhu
email: yiningzhu@nwpu.edu.cn
Affiliation: Northwestern Polytechnical University, Xi’an, China
© none
Abstract.
Current large video-language models (LVLMs) still face challenges when dealing with long videos, mainly because frames are often processed independently, making it difficult to capture temporal dependencies across events. Although retrieval-augmented approaches have been introduced to provide additional context, most of them operate at the frame or snippet level, which limits their ability to model how events evolve over time and relate to each other. In this paper, we propose Event Chain Retrieval-Augmented Generation (EC-RAG), a training-free framework that organizes video content into an explicit event chain before question answering. Instead of retrieving isolated frames or text segments, EC-RAG first partitions the video into semantically coherent segments, represents each segment using multi-modal signals, and then links them into a structured chain that preserves temporal order and captures inter-event relationships. Given a query, the system identifies relevant events within this chain and gathers supporting evidence from the associated modalities. Our approach offers several practical advantages: (i) event-level abstraction that better reflects how video content is naturally structured, enabling more reliable localization compared to frame-level retrieval; (ii) structured multi-modal fusion that aggregates speech, text, and visual cues at the event level, allowing complementary information to be more effectively utilized during reasoning; and (iii) plug-and-play compatibility with existing LVLM backbones, requiring no additional training or reliance on proprietary models. Experiments on Video-MME, MLVU, and LongVideoBench show that this event-centric design consistently outperforms frame-level retrieval baselines, highlighting the importance of modeling temporal structure for long-video understanding.
Keywords:
Long Video Understanding, Retrieval-Augmented Generation, Event Chain Reasoning
1. Introduction
With the rapid advancement of Large Video-Language Models (LVLMs), significant progress has been made in video understanding tasks (Liu et al., 2023; Chen et al., 2024a; Liu et al., 2026; Zhang et al., 2023; Zhang et al., 2024c; Lin et al., 2024b). However, when processing long videos spanning tens of minutes to hours, current LVLMs face critical challenges: the limited context window restricts the number of frames that can be processed, while simply increasing sampled frames leads to information redundancy and reasoning degradation.
Recent studies have sought to exten