                                             M EM L ENS: Benchmarking Multimodal Long-Term
                                                Memory in Large Vision-Language Models


                                                       Xiyu Ren1 Zhaowei Wang1 Yiming Du2 Zhongwei Xie1
                                                 Chi Liu1 Xinlin Yang1 Haoyue Feng1 Wenjun Pan1 Tianshi Zheng1
                                                Baixuan Xu1 Zhengnan Li3 Yangqiu Song1 Ginny Wong4 Simon See4
arXiv:2605.14906v1 [cs.CV] 14 May 2026




                                                                        1
                                                                     CSE Deparment, HKUST 2 CUHK
                                                            3
                                                           OmniMemory (Shenzhen) Intelligent Technology Co., Ltd.
                                                     4
                                                       NVIDIA AI Technology Center (NVAITC), NVIDIA, Santa Clara, USA
                                                                {xrenaf, zwanggy, yqsong}@cse.ust.hk
                                                            ydu@se.cuhk.edu.hk lzhengnan389@gmail.com



                                                                                      Abstract

                                                  Memory is essential for large vision-language models (LVLMs) to handle long,
                                                  multimodal interactions, with two method directions providing this capability: long-
                                                  context LVLMs and memory-augmented agents. However, no existing benchmark
                                                  conducts a systematic comparison of the two on questions that genuinely require
                                                  multimodal evidence. To close this gap, we introduce M EM L ENS, a comprehensive
                                                  benchmark for memory in multimodal multi-session conversations, comprising
                                                  789 questions across five memory abilities (information extraction, multi-session
                                                  reasoning, temporal reasoning, knowledge update, and answer refusal) at four
                                                  standard context lengths (32K–256K tokens) under a cross-modal token-counting
                                                  scheme. An image-ablation study confirms that solving M EM L ENS requires vi-
                                                  sual evidence: removing evidence images drops two frontier LVLMs below 2%
                                                  accuracy on the 80.4% of questions whose evidence includes images. Evaluating
                                                  27 LVLMs and 7 memory-augmented agents, we find that long-context LVLMs
                                                  achieve high short-context accuracy through direct visual grounding but degrade
                                                  as conversations grow, whereas memory agents are length-stable but lose visual fi-
                                                  delity under storage-time compression. Multi-session reasoning caps most systems
                                                  below 30%, and neither approach alone solves the task. These results motivate hy-
                                                  brid architectures that combine long-context attention with structured multimodal
                                                  retrieval. Our code is available at https://github.com/xrenaf/MEMLENS.



                                         1    Introduction

                                         Memory is essential for enabling large vision-language models (LVLMs) [1, 2] to maintain con-
                                         sistency across extended multimodal interactions and to continuously incorporate new information
                                         over time [3–5]. When LVLMs are deployed as agents, the inputs they handle accumulate incre-
                                         mentally rather than arriving all at once. As interaction histories expand, the agent must recall past
                                         content and reason over the growing context to remain consistent with prior turns. At the same time,
                                         it must integrate new facts as they arrive and revise outdated information to keep its knowledge
                                         current. Two directions have emerged to provide this capability, namely long-context LVLMs and
                                         memory-augmented agents. Long-context LVLMs enlarge the native context window so that the
                                         complete dialogue history, including interleaved images, can be processed directly by the model [5–7].
                                         Memory-augmented agents, building on retrieval-augmented generation, instead compress, index,
                                         and selectively retrieve past content from an external store [8–10].
