ting locations, enabling video-free question answering.
To evaluate the effectiveness of MIT3, we conduct experiments on LVBench (Wang et al., 2025a), LongVideoBench (Wu et al., 2024), and Video-MME (Fu et al., 2025), spanning long-video and general video question answering across diverse durations.
On LVBench, MIT3 improves the matched NKR-style baseline by 6.8 points with Qwen3-VL-8B and by 5.7 points with Qwen3-VL-32B, demonstrating the value of causal localization and visual representation matching under identical textual supervision.
The gains extend to LongVideoBench-Long and Video-MME, where MIT3 achieves 63.3 and 66.7 accuracy, respectively, confirming the value of the proposed memory-writing mechanism across extreme-duration and general video understanding settings.
Ablation studies isolate the contributions of memory localization and visual distillation.
Our main contributions are:
•
We study the layer-wise video-memory distribution in the language model and find that layers with high causal memory utility exhibit a concentration pattern across videos.
•
We introduce MIT3, a test-time memory-writing framework that combines causal localization with visual memory distillation for video-free question answering.
•
We demonstrate data-efficient video internalization across LVBench, LongVideoBench, and Video-MME with substantially less synthesized supervision.
2 Related Work
2.1 Video Understanding
Video question answering requires models to ground language in visual entities, actions, temporal order, and causal relations.
Early benchmarks progressively expanded this reasoning demand from short animated clips to more complex web videos and causal-temporal questions(Tapaswi et al., 2016; Lei et al., 2018; Xiao et al., 2021).
Modern video-language models connect a visual encoder to an LLM through learned projection or cross-modal modules and use instruction tuning to support question answering (Maaz et al., 2024; Zhang et al., 2023; Lin et al., 2024; Jin et al., 2024).
They answer by integrating sampled frames or visual tokens into the language-model context, making the quality and cost of reasoning depend on the visual information available for each question.
2.2 Long-Video Context Modeling
Long-video question answering further requires locating and combining evidence scattered over minutes or hours.
Recent benchmarks evaluate long-range retrieval, temporal reasoning, and event comprehension in this setting (Mangalam et al., 2023; Fu et al., 2025; Wang et al., 2025a; Wu et al., 2024).
Existing methods extend multimodal context capacity (Chen et al., 2025; Zhang et al., 2024), compress observations into sparse memories (Song et al., 2024; He et al., 2024), reduce temporal and knowledge redundancy (Wang et al., 2024; Ren et al., 2024), or retrieve relevant evidence from external multimodal indices (Ren et al., 2026).
Although these approaches improve access to long-range evidence, answering each question still requires video tokens, cached representations, or retrieval.
They therefore optimize access to video evidence rather than internalizing a video’s knowledge for subsequent text-only question answering.
A complementary direction internalizes a video’s knowledge into video-specific parameters, enabling video-free, text-only question answering after one-time per-video adaptation.
NKR (Guan et al., 2026) synthesizes multi-scale descriptions and question-answer pairs and trains a video-specific adapter (Hu et al., 2021), demonstrating the feasibility of parameterized video memory.
However, its internalization process primarily relies on synthesized text and adopts a fixed adaptation configuration, without explicitly determining where video knowledge should be written within the backbone or how visual evidence should shape the resulting memory.
2.3 Test-Time Training
Test-time training adapts models on test inputs with auxiliary self-supervised objectives to improve robustness under distribution shifts (Sun et al., 2020; Liu e