Long-form Video-to-Text Summarization using Recursive Language Models
Report GitHub Issue
×
Title:
Content selection saved. Describe the issue below:
Description:
Submit without GitHub
Submit in GitHub
arXiv is now an independent nonprofit!
Learn more
×
Back to arXiv
Why HTML?
Report Issue
Back to Abstract
Download PDF
Abstract
Introduction
Related Work
Video-to-text Summarization
Long-form Video Understanding
Position of the Present Work
Recursive Video Language Summarization Model
Semantic Temporal Decomposition
REPL Environment and Recursive Video Queries
Adaptive Multimodal Tool Calling
Hierarchical Summary Aggregation
Evaluation Setup
Hardware
Datasets
Performance Metrics
Models
Auxiliary Modules
Main Results and Discussion
Peformance comparison (RQ1): Is RLM beneficial for LLM-based video-to-text summarization of long-form videos compared to non-recursive zero-shot baselines?
Choice of the Root-LLM: Proprietary vs. Opensource
Ablation study (RQ2): what are
the effects of the main RVLSMs’s components?
Semantic alignment (RQ3): are RVLSMs’s summaries closely aligned with human-generated ones?
LLM-as-a-Judge
Qualitative Analysis
Statistical Significance Tests
Conclusions and future work
Limitations
Routing Mechanism.
Dataset Availability.
Chunking Quality.
Annotation Sensitivity.
References
License: CC BY-NC-SA 4.0
arXiv:2610.26431v1 [cs.CV] 19 Oct 2026
Long-form Video-to-Text Summarization using Recursive Language Models
Alessia Intini
Lorenzo Vaiani
Enrico Chen
Luca Cagliero
Abstract
While VideoLLMs have made significant progress in understanding short video benchmarks, they often struggle to summarize long-form ones. This is mainly due to performance degradation caused by long-context prompts, the inherent difficulty of preserving temporal continuity and a lack of semantically rich concepts in event descriptions.
To tackle these issues, this paper introduces the first recursive Video Language Summarization Model (RVLSM) designed for video processing and tailored to long-form video-to-text summarization. RVLSM recursively decomposes videos into semantically coherent segments and delegates chunk-level reasoning to VideoLLM sub-models and external tools.
It employs (1) Semantic Temporal Decomposition (STD), which preserves temporal coherence and continuity
through a scene-aware chunking strategy that dynamically partitions videos according to semantic boundaries, and
(2) Adaptive Multimodal Tool Calling (AMTC), which ensures semantic richness of the summary via a routing mechanism that selectively invokes auxiliary modules such as action recognition, optical character recognition, automatic speech recognition, and object detection.
Experiments on the VideoXum and MixTyVSum benchmarks show that RVLSM consistently improves over zero-shot VideoLLM baselines across multiple model settings and performance metrics, and is competitive with a state-of-the-art fine-tuned model for video-to-text summarization. Importantly, the named entities in the RVLSM’s summaries align more closely with those detected in the ground-truth summaries than those detected by baseline and state-of-the-art methods, confirming the saliency of the event descriptions it generates.
1Politecnico di Torino
{name.surname}@polito.it
Introduction
Figure 1: Sketch of the RVLSM paradigm. The Root-LLM decomposes the long video, recursively invokes Sub-LLMs and external tools (action recognition, object detection, etc.), and composes the summary from the intermediate outcomes.
Recent advances in VideoLLMs have significantly improved their understanding capabilities across different tasks, including video-to-text summarization. To create meaningful summaries of long-form videos,
models must identify and correlate key events and behaviors that occur over a
long period of time, which requires effectively capturing and
processing long-range temporal dependencies (Li et al. 2026).
Despite recent improvements, the performance of VideoLLMs on long-form videos is challenged 