tilities:
𝒮V=TopKlRl​(V).\mathcal{S}_{V}=\operatorname*{TopK}_{l}\;R_{l}(V).
(9)
The utility scores are computed once with the frozen model and without gradients before adapter optimization.
LoRA modules are inserted only into the attention sublayers of model layers in 𝒮V\mathcal{S}_{V}, while the remaining layers and all backbone parameters stay frozen.
This converts adapter placement from a fixed architectural configuration into a video-specific memory-writing decision.
3.4 Visual Memory Distillation
The third stage writes visual evidence into the selected memory locations by distilling how video conditioning redirects question representations.
Synthesized descriptions provide explicit semantic supervision, while video-conditioned hidden states supply complementary signals about appearance, spatial configuration, and transient events.
For each temporally grounded local QA example, the frozen model processes the relevant video segment and question as teacher, while the adapter-enabled student receives the same question without video input.
Visual memory distillation trains the normalized adapter-induced changes to reproduce the normalized video-induced representation changes at the selected model layers 𝒮V\mathcal{S}_{V}.
This transfers the internal computational effect of the relevant video segment into text-only memory without retaining frames or visual tokens.
Teacher and student deltas.
At each selected model layer l∈𝒮Vl\in\mathcal{S}_{V}, the frozen teacher provides the video-induced target defined in Equation 4:
Δ​Hqi,lteacher=Hqi,lV​Q−Hqi,lQ=Δ​Hqi,lV.\Delta H_{q_{i},l}^{\mathrm{teacher}}=H_{q_{i},l}^{VQ}-H_{q_{i},l}^{Q}=\Delta H_{q_{i},l}^{V}.
(10)
The text-only student processes QiQ_{i} with the video-specific adapter enabled.
Let Hqi,lLoRAH_{q_{i},l}^{\mathrm{LoRA}} denote its hidden states at the same aligned question-token positions.
The adapter-induced change is
Δ​Hqi,lstudent=Hqi,lLoRA−Hqi,lQ.\Delta H_{q_{i},l}^{\mathrm{student}}=H_{q_{i},l}^{\mathrm{LoRA}}-H_{q_{i},l}^{Q}.
(11)
Both deltas are measured relative to the same question-only state Hqi,lQH_{q_{i},l}^{Q}.
This shared reference isolates the transformations contributed by the video and the adapter, making their representation changes directly comparable.
Representation alignment.
We align the teacher and student deltas using cosine distance.
For model layer l∈𝒮Vl\in\mathcal{S}_{V}, the token-averaged loss is
ℒi,lvisual=1|qi|​∑t∈qi(1−⟨Δ​hi,l,tstudent,Δ​hi,l,tteacher⟩‖Δ​hi,l,tstudent‖2​‖Δ​hi,l,tteacher‖2+ϵ),\mathcal{L}_{i,l}^{\mathrm{visual}}=\frac{1}{|q_{i}|}\sum_{t\in q_{i}}\left(1-\frac{\left\langle\Delta h_{i,l,t}^{\mathrm{student}},\Delta h_{i,l,t}^{\mathrm{teacher}}\right\rangle}{\left\|\Delta h_{i,l,t}^{\mathrm{student}}\right\|_{2}\left\|\Delta h_{i,l,t}^{\mathrm{teacher}}\right\|_{2}+\epsilon}\right),
(12)
where ϵ\epsilon provides numerical stability.
The per-example visual loss averages this reproduction objective over the selected model layers:
ℒivisual=1|𝒮V|​∑l∈𝒮Vℒi,lvisual.\mathcal{L}_{i}^{\mathrm{visual}}=\frac{1}{|\mathcal{S}_{V}|}\sum_{l\in\mathcal{S}_{V}}\mathcal{L}_{i,l}^{\mathrm{visual}}.
(13)
This loss is evaluated for the temporally grounded local QA examples used for visual distillation, while description and global-QA examples provide text-only NTP and SFT supervision.
3.5 Test-Time Memory Optimization
For each video, we freeze the base VLM and optimize only LoRA parameters in attention sublayers of the causally selected model layers 𝒮V\mathcal{S}_{V}.
Let 𝒟visual⊆𝒟localQA\mathcal{D}_{\mathrm{visual}}\subseteq\mathcal{D}_{\mathrm{localQA}} denote the temporally grounded local QA examples used for visual memory distillation.
Their sample-averaged visual loss is
ℒvisual=1|𝒟visual|​∑i∈𝒟visualℒivisual.\mathcal{L}_{\mathrm{visual}}=\frac{1}{|\mathcal{D}_{\mathrm{visual}}|}\sum_{i\in\mathcal{D}_{\mathrm{visual}}}\mathcal{L}_{i}^{\mathrm{visual}}.
(14)
The complete test-time memory optimization objective is
ℒMIT3=ℒNTP+ℒSFT+α​ℒvisual,\mathca