fferent tasks, including video-to-text summarization. To create meaningful summaries of long-form videos,
models must identify and correlate key events and behaviors that occur over a
long period of time, which requires effectively capturing and
processing long-range temporal dependencies (Li et al. 2026).
Despite recent improvements, the performance of VideoLLMs on long-form videos is challenged by the following issues:
•
Information compression: long videos are frequently compressed through sparse frame sampling, temporal pooling, or fixed-duration chunking, forcing models to reason over aggressively condensed visual representations (Wang et al. 2025). While often sufficient for short clips, these strategies become increasingly problematic for long-horizon and dense-event videos.
•
Context saturation: LLMs are prone to hallucination and performance drops when their context window is largely filled (Zhang et al. 2024). Since the position of multimodal information also matters, this is typically exacerbated by the lost-in-the-middle issue.
•
Temporal coherence degradation: on videos spanning multiple scenes, subjects, or actions, VideoLLMs struggle to preserve the temporal continuity of the event structure and the semantic saliency of event descriptions. Long-form video-to-text summarization thus requires reasoning over temporally distributed events while maintaining chronological consistency and exhaustive coverage of the underlying concepts (Niu et al. 2025).
•
Limited semantic richness of the event descriptions: summaries typically mention the key events occurring in the video, but the semantic concepts in the textual descriptions are not necessarily aligned with human expectations (Cagliero et al. 2025).
To tackle the above issues, in this paper we present the first adaptation of the Recursive Language Models (RLMs) (Zhang et al. 2026) towards long-form video-to-text summarization.
RLM is a recently proposed inference paradigm for textual LLMs,
where long textual context is treated as an external object inside persistent Read-Eval-Print Loop (REPL) environments, allowing a root-LLM to
recursively invoke sub-VLLMs or external tools over portions of the input through programmatic decomposition.
Although they have proven to be effective in document summarization (Zhang et al. 2026), the existing framework is limited to the textual modality, overlooking the unique challenges of video-to-text summarization.
We present Recursive Video Language Summarization Models (RVLSMs), a recursive framework for long-form video-to-text summarization that extends the RLM paradigm to multimodal video reasoning. Importantly, the proposed approach is agnostic to the pretrained LLM used. RVLSM treats the input video as an external multimodal object and recursively summarizes semantically coherent video segments through VideoLLM sub-calls and external tools (see Figure 1). Unlike static multimodal augmentation pipelines, RVLSM employs a heuristic routing mechanism to select the appropriate analysis configuration for each chunk based on previously generated summaries, enabling sequential context-aware evidence acquisition across the video timeline.
Specifically, RVLSM employs (1) Semantic Temporal Decomposition (STD), which chunks videos according to scene transitions and additional contextual features to preserve temporal coherence, and (2) Adaptive Multimodal Tool Calling (AMTC), which calls auxiliary external tools for object detection, action recognition, automatic speech recognition and optical character recognition
through a programmable Root-LLM planner to ensure semantic richness of the summary.
RVLSM addresses information compression and context saturation via the RLM paradigm, temporal coherence degradation using the STD component, and lack of semantic richness of the summary using the AMTC component.
Using RVLSM, we address the following Research Questions: RQ1) Is RLM beneficial for LLM-based video-to-text summarization of long-form vid