t al., 2026; Zhang et al., 2023; Zhang et al., 2024c; Lin et al., 2024b). However, when processing long videos spanning tens of minutes to hours, current LVLMs face critical challenges: the limited context window restricts the number of frames that can be processed, while simply increasing sampled frames leads to information redundancy and reasoning degradation.
Recent studies have sought to extend the context length of LVLMs for long video understanding, such as LongLLaVA (Song et al., 2024) scales token capacity to ingest more frames. However, this brute-force expansion proves brittle—Video-MME (Fu et al., 2025) benchmark shows accuracy drops when additional frames are supplied beyond a threshold. This outcome suggests that simply increasing the number of sampled frames not only leads to information redundancy but also imposes additional challenges for the model to handle complex reasoning.
Figure 1. Advantages of our EC-RAG. EC-RAG provides an event-centric, training-free pipeline with structured temporal understanding that is easily compatible with any LVLM.
A parallel research direction employs Retrieval-Augmented Generation (RAG) to supplement video queries with externally retrieved information. Early efforts in this direction (Liu et al., 2024a; Zhang et al., 2024d) demonstrate that augmenting visual inputs with auxiliary textual information can effectively enhance video understanding, particularly for content that is difficult to capture through visual frames alone. These methods typically extract three types of auxiliary texts from videos: automatic speech recognition (ASR) for transcribing spoken dialogue and narration, optical character recognition (OCR) for capturing on-screen text such as titles and labels, and object detection (DET) for identifying visual entities and their spatial arrangements. Building upon this foundation, Video-RAG (Luo et al., 2024) proposes visually-aligned auxiliary texts from OCR, ASR, and DET, achieving notable improvements with minimal computational overhead. TV-RAG (Cao et al., 2025) further introduces temporal-window retrieval with entropy-based keyframe selection to handle semantic drift in long videos. More recently, AdaVideoRAG (Xue et al., 2025) employs adaptive difficulty routing and graph databases for complex multi-hop reasoning. Despite these advances, existing video RAG methods share a fundamental limitation: they retrieve at the frame level or text segment level, treating video as a collection of independent units rather than a coherent narrative of interconnected events. Long videos inherently contain sequences of causally connected events—a lecture progresses through topics, a movie unfolds through scenes, a cooking video follows recipe steps. Without modeling this event structure, current methods produce fragmented retrieval results that fail to support questions requiring temporal reasoning.
Based on this observation, we focus on the following question: Can we design a video RAG framework that explicitly captures temporal event dynamics and inter-event relationships for coherent long video understanding?
To this end, we propose Event Chain Retrieval-Augmented Generation (EC-RAG), a training-free framework that organizes video content into explicit event chains before question answering. Instead of retrieving isolated frames or text snippets, EC-RAG first partitions the video into semantically coherent temporal segments, aggregates multi-modal signals (ASR, OCR, DET) within each segment to form events, and links them into a structured chain that preserves temporal order. For multi-modal extraction, we introduce CLIP-Variance guided keyframe selection that combines query-semantic relevance with visual informativeness, ensuring selected frames are both relevant and information-rich. Given a user query, EC-RAG performs event-level localization to identify the most relevant events along with their temporal context, and gathers supporting evidence from associated modalities fo