enote the temporally grounded local QA examples used for visual memory distillation.
Their sample-averaged visual loss is
ℒvisual=1|𝒟visual|​∑i∈𝒟visualℒivisual.\mathcal{L}_{\mathrm{visual}}=\frac{1}{|\mathcal{D}_{\mathrm{visual}}|}\sum_{i\in\mathcal{D}_{\mathrm{visual}}}\mathcal{L}_{i}^{\mathrm{visual}}.
(14)
The complete test-time memory optimization objective is
ℒMIT3=ℒNTP+ℒSFT+α​ℒvisual,\mathcal{L}_{\mathrm{MIT}^{3}}=\mathcal{L}_{\mathrm{NTP}}+\mathcal{L}_{\mathrm{SFT}}+\alpha\mathcal{L}_{\mathrm{visual}},
(15)
where each term is averaged over its corresponding training subset and α\alpha controls the visual-distillation weight.
After this one-time optimization, the resulting adapter θV\theta_{V} serves as the video-specific memory.
At inference, the frozen backbone receives only the question QQ with θV\theta_{V} enabled, requiring no video input, visual tokens, or retrieval.
The same adapter can answer subsequent questions about VV without reprocessing the video.
Table 1: Accuracy and inference efficiency on LVBench and LongVideoBench. Time/query denotes inference time per question. †\dagger denotes controlled NKR-style baseline using the same data as MIT3.
Method
Input
Time/query
(s)
Mem.
(GB)
LVBench
LongVideoBench
All
>1>1h
All
Long
VideoLLaMA3-7B
<768<768 fr.
64.5
1.2
45.3
–
–
–
AdaReTaKe-7B
<2048<2048 fr.
45
0.9
51.2
48.4
–
55.5
VideoRAG
N/A
90
0.5
49.2
47.8
–
–
VCA
N/A
–
–
41.3
–
–
–
DVD-o3
N/A
180
0.3
74.2
–
–
–
NKR-8B
N/A
0.25
0
39.5
–
–
–
NKR-14B
N/A
0.33
0
48.8
48.6
–
52.9
Qwen3-VL-8B backbone, with 2 fps
Direct
<768<768 fr.
31.1
15.1
45.6
42.1
60.2
48.0
NKR-style†
N/A
0.25
0
36.7
36.5
54.5
43.3
MIT3 (Ours)
N/A
0.25
0
43.5
43.2
58.5
47.1
Qwen3-VL-32B backbone, with 2 fps
Direct
<768<768 fr.
103.4
26.9
52.5
48.1
65.7
55.1
NKR-style†
N/A
0.46
0
44.9
45.3
57.4
47.3
MIT3 (Ours)
N/A
0.46
0
50.6
50.1
63.3
55.9
4 Experiments
4.1 Experimental Setup
Datasets and Baselines
We evaluate MIT3 on LVBench (Wang et al., 2025a), LongVideoBench (Wu et al., 2024), and Video-MME (Fu et al., 2025). For LVBench and LongVideoBench, we report results on both the full benchmarks and their long-video subsets, excluding subtitle and audio inputs. Baselines include published long-video methods (Zhang et al., 2025; Wang et al., 2025b; Ren et al., 2026; Yang et al., 2025; Zhang et al., 2026; Guan et al., 2026), direct Qwen3-VL-8B/32B inference (Bai et al., 2025), and a controlled NKR-style baseline. Since NKR’s per-video training data are unavailable, we reproduce its data-generation procedure and train both NKR-style and MIT3 on the same generated supervision. The NKR-style baseline optimizes only ℒNTP+ℒSFT\mathcal{L}_{\mathrm{NTP}}+\mathcal{L}_{\mathrm{SFT}}, while MIT3 additionally uses video-specific layer selection and visual memory distillation.
Implementation Details
Following video-NKR (Guan et al., 2026), we use LoRA with rank 256 and scaling factor 512, selecting the top 18 and 32 layers for Qwen3-VL-8B and Qwen3-VL-32B, respectively. For a one-hour video, our pipeline generates approximately 5.5k training examples, roughly one-eighth of those used by NKR. We also verify that no evaluation questions appear in adapter training or causal-localization calibration. Full dataset, data-generation, and optimization details are provided in Appendix C.
4.2 Main Results
Table 1 compares MIT3 with published long-video systems, direct video-conditioned inference, and the matched NKR-style baseline.
Under identical video-specific supervision, MIT3 consistently outperforms the NKR-style baseline across both backbone sizes and all four evaluation settings, demonstrating the effectiveness of video-specific layer selection and visual memory distillation for parameterized video memory.
Despite performing video-free, text-only inference after one-time internalization, MIT3 remains close to direct video-conditioned inference on the full benchmarks, while requiring no video frames, visual tokens, retrieval, or video-dependent KV cache at query time.
Mor