nk summary si−1s_{i-1} is matched. A normalized keyword-matching score is then computed for each category by dividing the raw number of matches by the size of the corresponding keyword set, so that categories with larger keyword sets are not favored.
The selector defaults to VideoQuery when no signal is detected, to EnrichedVideoQuery when no category dominates, and to the corresponding SingleModalityQuery otherwise.
Hierarchical Summary Aggregation
After all chunks have been recursively summarized, the Root-LLM aggregates the local
summaries into a single global summary. Let
𝒮={s1,s2,…,sN}\mathcal{S}=\{s_{1},s_{2},\dots,s_{N}\}
be the ordered set of chunk summaries produced by the recursive VideoLLM calls.
The Root-LLM receives the complete sequence 𝒮\mathcal{S} and synthesizes a final
abstractive summary:
S=ℳr​o​o​t​(s1,s2,…,sN).S=\mathcal{M}_{root}(s_{1},s_{2},\dots,s_{N}).
The Root-LLM is directed to maintain chronological order and consistency in the final summary.
Evaluation Setup
Hardware
All experiments were run on a machine equipped with an Intel® CoreTM
i9-10980XE CPU, 1×1\times NVIDIA®
A40 48GB GPU, 128128 GB of RAM running Ubuntu 22.0422.04 LTS.
The overall cost is approximately 1,400 GPU hours,
with several runs across various combinations of dataset, model, and setting.
On average, each run consumes approximately 9.09.0M input tokens
and ∼200{\sim}200K output tokens, resulting in an average API cost
of $5.58\$5.58 per run across all tested models and datasets.
Datasets
We evaluate RVLSM on two long video-to-text summarization benchmarks: VideoXum (Lin et al. 2024) and MixTyVSum (Cagliero et al. 2025). VideoXum focuses on holistic narrative summarization of open-domain videos, while MixTyVSum emphasizes dense event-centric summarization with temporal and spatial annotations. Following our long-video setting, we restrict VideoXum to videos longer than 230 seconds and we removed MSVD(Hendria 2025) videos from MixTyVSum, because they are all shorter than 20 seconds. Additional details are provided in the Supplementary Document.
Performance Metrics
We compare the generated summaries with the ground truth using the F1-Score values of the following established metrics: ROUGE-1, ROUGE-2, ROUGE-L (Lin 2004) and SBERT (Reimers and Gurevych 2019), computed as the cosine similarity between the Sentence-BERT embeddings of the generated and reference summaries. We further report METEOR (Banerjee and Lavie 2005), BLEU-4 (Papineni et al. 2002), and CIDEr (Oliveira dos Santos et al. 2021), three metrics commonly adopted for video captioning and summarization evaluation.
For the Named Entity Recognition analysis, we report Precision, Recall, and F1 score. More details in the Supplementary Document.
Aggregated values are computed as the micro-average over TP, FP, and FN across entity types.
Models
Following the RLM paradigm (Zhang et al. 2026), our pipeline distinguishes between two types of models. The Root-LLM is a coding-oriented LLM acting as the main orchestrator: it decomposes the input,
manages intermediate variables, and dispatches sub-calls within the REPL
environment. The sub-LLM is, in our context, a multimodal Video LLM invoked recursively
by the root-LLM to process individual video segments and return structured descriptions.
We experimented with two root-LLMs (GPT-5.4 (Singh et al. 2026) and
Qwen3-Coder-480B-A35B (Qwen Team 2025)) and four sub-LLMs (Molmo2-8B (Clark et al. 2026), Qwen3-VL-8B-Instruct, Qwen/Qwen3-VL-30B-Instruct (Bai et al. 2025), and Qwen/Qwen3-Omni-30B-A3B-Instruct (Xu et al. 2025)).
Further details are given in the Supplementary Document.
Auxiliary Modules
For Named Entity Recognition we use SpaCy (Honnibal et al. 2020) (en_core_web_sm model).
For scene-aware STD we use PySceneDetect (Castellano 2014).
Object-level semantic signals are extracted using YOLOv8 (Jocher and Qiu 2023) pretrained on the COCO dataset, applied every 10 frames.
Action-level semantic evidence is obtained through MMAction2 (MM