 the matched NKR-style baseline across all supervision budgets and both backbone sizes, indicating that the proposed memory-writing strategy improves the utilization of synthesized supervision rather than relying on a larger training set.
Notably, the performance of MIT3 approaches saturation once the supervision reaches a moderate scale, and further increasing the number of generated examples yields only marginal gains beyond the 5.5k setting.
This trend suggests that the benefit of MIT3 is driven primarily by how video knowledge is localized and distilled into the model, rather than by continued scaling of synthetic supervision.
In other words, causal localization and visual memory distillation allow the model to extract more useful video-specific knowledge from a compact set of training examples.
Figure 4: Data efficiency on LVBench under matched per-video supervision budgets.
5 Conclusion
We introduced MIT3, a per-video test-time training framework that converts each long video into compact adaptive memory for write-once, query-many understanding. By answering subsequent questions through text-only inference, MIT3 eliminates repeated visual encoding or retrieval while retaining video-specific knowledge in lightweight parameters. Causal localization identifies where this knowledge can be written most effectively, and visual memory distillation trains the adapter to reproduce the representation change induced by visual evidence. Across LVBench, LongVideoBench, and Video-MME, MIT3 consistently improves controlled text-only internalization using roughly one-eighth of NKR’s synthesized supervision. Its per-video state is 45.6×45.6\times smaller than persistent KV reuse with Qwen3-VL-32B, enabling scalable access to growing videos.
References
Bai et al. (2025)
Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al.
Qwen3-vl technical report.
arXiv preprint arXiv:2511.21631, 2025.
Chen et al. (2025)
Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, et al.
Longvila: Scaling long-context visual language models for long videos.
In International Conference on Learning Representations, volume 2025, pp. 18227–18246, 2025.
Fu et al. (2025)
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al.
Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis.
In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24108–24118. IEEE, 2025.
Gandelsman et al. (2022)
Yossi Gandelsman, Yu Sun, Xinlei Chen, and Alexei Efros.
Test-time training with masked autoencoders.
Advances in Neural Information Processing Systems, 35:29374–29385, 2022.
Guan et al. (2026)
Yuchen Guan, Xiao Li, Zongyu Guo, Xiaoyi Zhang, Xiulian Peng, Chun Yuan, and Yan Lu.
From content to knowledge: Lightning fast long-video understanding with neural knowledge representations.
arXiv preprint arXiv:2606.11913, 2026.
He et al. (2024)
Bo He, Hengduo Li, Young Kyun Jang, Menglin Jia, Xuefei Cao, Ashish Shah, Abhinav Shrivastava, and Ser-Nam Lim.
Ma-lmm: Memory-augmented large multimodal model for long-term video understanding.
In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13504–13514. IEEE, 2024.
Hu et al. (2021)
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.
Lora: Low-rank adaptation of large language models.
arXiv preprint arXiv:2106.09685, 2021.
Jin et al. (2024)
Peng Jin, Ryuichi Takanobu, Wancai Zhang, Xiaochun Cao, and Li Yuan.
Chat-univi: Unified visual representation empowers large language models with image and video understanding.
In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13700–13710. IEEE, 2024.
Lei et al. (2018)
Jie Lei, Licheng Yu, Mohit Bansa