nner to ensure semantic richness of the summary.
RVLSM addresses information compression and context saturation via the RLM paradigm, temporal coherence degradation using the STD component, and lack of semantic richness of the summary using the AMTC component.
Using RVLSM, we address the following Research Questions: RQ1) Is RLM beneficial for LLM-based video-to-text summarization of long-form videos compared to non-recursive zero-shot baselines? (see the results in Table 1). RQ2) What are the effects of the main RVLSMs’s components? (see the ablation study in Figure 3 and Table 1). RQ3) Are the RVLSMs’s summaries more closely aligned with human-generated ones in terms of presence of different NLP entities? (see the results in Table 2).
The main paper contributions are summarized below:
•
We introduce the Recursive Video Language Summarization Model (RVLSM), a new recursive REPL-based framework for long-form video-to-text summarization agnostic to the pretrained LLMs used.
•
On two video benchmarks,
we demonstrate that the RVLSM consistently improves over zero-shot VideoLLM baselines across multiple model settings and performance metrics , and is competitive with a state-of-the-art fine-tuned model for video-to-text summarization (e.g., SBERT F1 on MixtyVSum: RVLSM 0.64 vs. Baseline non-recursive 0.57 vs. V2Xum Baseline 0.44).
•
We show RVLSM’s summaries align more closely with those detected in the ground-truth summaries than those detected by baseline and state-of-the-art methods (e.g., F1 MicroAverage on MixtyTVSum: RVLSM 0.14 vs. Baseline non-recursive 0.07 vs. V2Xum Baseline 0.03), confirming the saliency of the event descriptions the model produces.
Related Work
Video-to-text Summarization
Video-to-text summarization has traditionally focused on extractive methods that select key frames or representative clips from long videos (Gong et al. 2014; Zhang et al. 2016; Lin et al. 2022). More recent approaches have introduced reinforcement learning for learning compact video representations (Zhou et al. 2018; Wang et al. 2024). With the emergence of video LLMs, recent methods have shifted toward abstractive video-to-text summarization, directly generating natural language descriptions from visual and multimodal inputs (Mu et al. 2024; Cagliero et al. 2025; Lee et al. 2025; Hua et al. 2025). Despite strong performance on short videos, current VideoLLMs remain constrained by limited temporal context windows and typically rely on sparse frame sampling or fixed temporal compression strategies.
Long-form Video Understanding
Recent work has studied how to extend VideoLLMs capabilities to longer temporal horizons
through memory modules, hierarchical architectures, sparse attention mechanisms,
or temporal compression strategies (Wu et al. 2024; Song et al. 2024; Wang et al. 2026). Hierarchical summarization pipelines commonly split long videos into fixed-duration segments before independently summarizing each chunk (Pennec et al. 2026).
Although effective in reducing memory usage, fixed chunking may fragment semantically
coherent events across temporal boundaries. RVLSM, instead, performs scene-aware
temporal decomposition through dynamic scene-based chunking, enabling recursive
reasoning over semantically coherent video segments.
Position of the Present Work
To the best of our knowledge, this is the first attempt to apply RLMs to long-context inputs in the video modality, targeting video-to-text summarization because its programmatic and recursive nature can overcome the aforementioned limitations.
Recursive Video Language Summarization Model
Given a base VideoLLM MM with limited temporal context capacity, a Recursive Video Language Summarization Model (RVLSM) is an inference-time scaffold around MM that treats the input video as part of an external environment while preserving the ability to densely process its temporal structure through recursive sub-calls. Given an input video
V={f1,f2,…,fT},V=\{f_{1},f_{2},\dots,f_{T}\},
where fi