                                                                                              Visual         Textual        Output
                                                                                              Mixed          Reasoning
                                       IE    TR   AR                              100
                                                                                                       8%
               80                      MSR   KU                                                                                  14%




                                                             % of wrong answers
               70                                                                  80                                            11%
                                                                                                                    44%
               60
Accuracy (%)


                                                                                   60
               50                                                                                      73%

               40                                                                  40
                                                                                        89%
                                                                                                                                 75%
               30                                                                                                   42%


               20                                                                  20

               10                                                                                      10%          11%
                                                                                    0
                0                                                                       IE         MSR               TR           KU
                    32K         64K               128K                                                 Memory type
                           Context length
                                                             (b) Wrong-answer error decomposition at 128K con-
(a) Per-type accuracy vs. input length. Solid: LVLM          text across the four answerable memory types, split
average (27 models, n=789); dashed: agent average            by failure modality (visual, textual, mixed, reasoning,
(7 systems, n=195). Bands: 95% CI.                           output). Category definitions appear in Table 19.


flips the predicted state. MSR is the hardest, as cross-session aggregation over three to eight sessions
defeats every evaluated system: only Kimi-K2.5 (44.06%) and Gemini-3.1-Pro (32.17%) clear 30%
by margin, exposing this as the shared capability ceiling of M EM L ENS. Within types, IE-Entity is
consistently harder than single-hop IE-PrevInfo, and MSR-Arithmetic is the most difficult subtype
overall (Appendix C.3).

No model dominates all memory abilities. No single model fam-                                                      IE MSR TR KU AR
ily dominates across all types (Figure 3). GLM-4.6V [70] leads                                            GPT-5.4 69.5 28.2 39.2 47.4 97.8
TR but collapses on KU, while Qwen3.5 [69] inverts the pattern.                                 Gemini-3.1-Pro 57.3 32.2 40.9 49.1 97.8
Kimi-K2.5 [68] is relatively strongest on MSR at 32K, though this
                                                                                                       Kimi-K2.5 54.8 44.1 53.1 50.9 97.8
advantage fades at longer contexts. Gemini-3.1-Pro is the only
                                                                                                   Qwen3.5 (2) 72.4 29.7 45.4 45.7 87.8
model simultaneously competitive on IE, KU, and MSR at 128K.
                                                                                                        GLM-4.6V 59.3 20.3 53.1 44.0 93.3
Memory agents exhibit an inverted profile: Memory-T1 reaches
                                                                                                            Mem0 13.1 25.0 50.0 17.2 77.3
high TR accuracy through BM25 date matching but falls well below
direct LVLMs on IE, substituting keyword retrieval for the visual                                MemAgent-7B 17.2 25.0 61.5 40.6 13.6

grounding that IE demands.                                                                             Memory-T1 18.0 25.0 63.5 13.8 9.1

                                                                                                        M3-Agent 19.0 22.2 30.6 6.2 13.6

4.3             Analysis
                                                                                                Figure 3: Memory-ability spe-
Current memory pipelines lose faithfulness to original visual cialization across representative
                                                                         LVLMs and memory agents.
evidence. The gap between agents and LVLMs is largest on the
visually grounded types (IE and KU). Despite their different input formats (§4.1), both text-only
and multimodal pipelines compress evidence visual information into a fixed memory representation
at storage time, leaving the original image pixels inaccessible at query time. Retrieval over these
compressed encodings is also less reliable than retrieval over the raw conversation text. The loss
is sharpest on IE and KU because captions and embeddings retain only the gist of an image and
discard fine-grained visual cues, such as counts, attributes, and spatial relations, that these two
types specifically probe. The same gap appears when blip-captions replace raw images for text-only
agents, indicating that the bottleneck lies in lossy cross-modality storage rather than in the answering
model. Closing this gap requires memory architectures that preserve image-level evidence rather than
caption-based compression.

Post-training on memory agent backbones weakens abstention. Memory agents fall far below
their direct-inference counterparts on AR. We evaluate two agents that keep the backbone frozen,
Mem0 (77.27%) and MemOS (68.18%), and five that further finetune the backbone via RL or LoRA


                                                         8
