Uni-Mem: Learning a Long-Horizon Agentic Memory Manager with Unified Online Policy Optimization
Skip to main content
Report GitHub Issue
×
Title:
Content selection saved. Describe the issue below:
Description:
Submit without GitHub
Submit in GitHub
Why HTML?
Report Issue
Back to Abstract
Download PDF
Contents
Reading mode
Colour theme
Abstract
1 Introduction
2 Related Work
Long-term Memory Agents.
Reinforcement Learning for Memory Agents.
3 Preliminaries
3.1 Problem Formulation
3.2 Unified Memory Management Agent
4 Method
4.1 Event-Wise Credit Assignment
Evidence-to-event localization.
Evidence-indexed memory scoring.
Event-level credit.
Update-robust evidence attribution.
4.2 Unified Manager Policy Optimization
Supervised cold start.
Stage-Decoupled GRPO.
5 Experiments
5.1 Experiment Setup
5.2 Main Results
5.3 Ablation Study
5.4 Further Analysis
5.5 Case Study
References
A Positioning Uni-Mem with Related Methods
Memory manager as an agent.
Why an agentic manager helps.
A.1 Differences from Mem-T
B Implementation Details
B.1 Memory Representation and Provenance Tracking
B.2 Tool Specifications and Execution Protocol
Construction events.
Retrieval events.
Execution rules.
C Training Details
C.1 Supervised Cold-Start Training
Source data construction.
SFT demonstration generation.
Optimization and checkpoint selection.
C.2 Event-Wise Credit Computation
C.3 RL Training Hyperparameters and Computational Resources
Answer and judge models during training.
D Details of Baselines.
RAG.
A-Mem.
Mem0.
MemoryOS.
MEM1.
MemAgent.
Memory-R1.
Memory-R2.
Mem-T.
E Experimental Setup and Evaluation Protocol
E.1 Memory-R2 Re-evaluation and Judge-Prompt Sensitivity
Comparability of reported baseline results.
E.2 Single-Stage GRPO and Ablation Configurations
M1M_{1}: Standard GRPO.
M2M_{2}: Single-stage GRPO with ECA.
F Supplementary Experiments
F.1 Gains from the Harness versus the RL Algorithm
F.2 Training Curves
F.3 Detailed Results on Training and Manager Scale
F.4 Detailed Analysis of Alternating Training Schedules
F.5 Retrieval Budget and Token Efficiency
G Prompt Templates
G.1 Memory Manager Prompt
G.2 LLM-as-a-Judge (JJ) Prompt
G.3 Answer Prompt
License: arXiv.org perpetual non-exclusive license
arXiv:2610.29671v1 [cs.CL] 21 Oct 2026
Uni-Mem: Learning a Long-Horizon Agentic Memory Manager with Unified Online Policy Optimization
Zhiqian Xia
††thanks: Equal contribution. Work done during internship.
Affiliation: Southeast University
Affiliation: Huawei Technologies, Noah’s Ark Lab
Jielong Tang11footnotemark:
1
Affiliation: Sun Yat-sen University
Affiliation: Huawei Technologies, Noah’s Ark Lab
Ling-An Zeng
Affiliation: Huawei Technologies, Noah’s Ark Lab
Shixiong Kai
Affiliation: Huawei Technologies, Noah’s Ark Lab
Rong Yao
Affiliation: Huawei Technologies, Noah’s Ark Lab
Hao Kong
Affiliation: Huawei Technologies, Noah’s Ark Lab
Xiao Dong
Affiliation: Shandong Normal University
Wei Wang
Affiliation: Sun Yat-sen University
Mingxuan Yuan
Affiliation: Huawei Technologies, Noah’s Ark Lab
Siyu Xia
Affiliation: Southeast University
Haifeng Xia
††thanks: Corresponding author.
Affiliation: Sun Yat-sen University
Abstract
Long-term agent memory is shifting from predefined workflows toward learnable policies for memory construction and retrieval. Existing learning-based methods either optimize construction alone or combine retrieval-side online reinforcement learning (RL) with offline construction supervision, limiting co-adaptation between the two stages. In this paper, we explore end-to-end online RL for a unified memory manager (Uni-Mem) that learns both capabilities under a shared policy. However, joint optimization with existing RL algorithms such as GRPO faces confounded group comparisons and sparse feedback over long memory-management trajectories. To this end, we propose Stage-Decoupled GRPO, which alternates construction and retrieval optimization while controlling variation from the other stage within each rollout group, yielding stage-specific learning signals. We further introduce Event-Wise Credit Assignment (ECA), which localizes feedback to responsible events and propagates event-level advantages to their constituent operations.
Experiments on LoCoMo and LongMemEval show that Uni-Mem outperforms the strongest RL-based memory, demonstrating the effectiveness of jointly learning memory construction and retrieval via Stage-Decoupled GRPO and ECA.
1 Introduction
Figure 1: Comparison of training paradigms in different methods. Construction-only RL keeps retrieval fixed, while asymmetric joint learning combines offline construction imitation with online retrieval RL. Uni-Mem jointly optimizes both stages under a shared policy through alternating online RL and event-wise credit.
Long-term memory enables Large Language Model (LLM) agents to reuse information across interactions that exceed their context windows (Wang et al., 2023; Yu et al., 2026b). Early memory systems, such as MemGPT (Packer et al., 2023), Mem0 (Chhikara et al., 2025), and A-Mem (Xu et al., 2025), rely on manually specified prompts and predefined workflows to organize and access past information. Despite their effectiveness, these systems remain constrained by frozen LLMs and rigid human priors, limiting the generalization of memory construction and retrieval to dynamic agent environments (Xiong et al., 2026).
To bridge this gap, recent works formulate memory management11
1
In this paper, memory management encompasses construction (creation, update, and deletion) and retrieval. as a learnable policy optimized via reinforcement learning (RL).
Methods such as Memory-R2 (Yan et al., 2026a) and Mem-α\alpha (Wang et al., 2025) keep retrieval fixed and train construction with RL.
Such construction-only RL learns how to build memory without jointly learning how to retrieve it adaptively, limiting memory utility in downstream tasks. To jointly learn both capabilities within a shared-parameter memory manager, approaches such as Mem-T (Yue et al., 2026) combine online agentic RL of retrieval with offline imitation of hindsight-selected construction actions. Yet this asymmetric joint learning can introduce a training lag: online retrieval RL continually updates the shared policy, whereas offline construction supervision remains anchored to earlier policy and memory states, hindering coordinated adaptation across the two stages.
This motivates a central research question:
How can a shared-parameter agentic memory manager, which operates through multi-turn tool interaction, jointly learn long-horizon construction and retrieval via end-to-end online RL?
Achieving this goal poses two main challenges. (1) Confounded group comparisons.
Existing group-relative RL (e.g. GRPO Shao et al. (2024)) derives advantages from reward differences among sampled rollouts. For a unified memory manager, each rollout couples construction trajectories with a retrieval continuation, so its final reward reflects decisions from both stages. When both stages vary across rollouts, trajectory-level group-relative advantages can conflate their contributions, obscuring the learning signal for each stage.
(2) Long-horizon credit attribution.
The manager policy performs hundreds of memory operations (tool calls) per rollout, yet receives the outcome reward only at answer time. Such sparse feedback neither localizes the query-relevant portions of a rollout nor reveals whether the required information was stored or successfully retrieved. Moreover, memory updates further complicate attribution by overwriting memory items produced by earlier decisions. Thus, feedback must be attributed to the relevant stage and operations within them.
In this work, we propose Uni-Mem, a unified memory manager that jointly learns agentic construction and retrieval through end-to-end online RL. Fig. 1 compares the training paradigms of existing methods with ours. To address confounded group comparisons, we introduce Stage-Decoupled GRPO, which decouples the two stages contributions in group-relative advantages. Specifically, we alternate construction and retrieval optimization while controlling variation from the other stage within each rollout group. For construction, we bypass learned retrieval: since each memory record retains the identifier of its source chunk, we directly gather the records linked to a query’s annotated supporting evidence (used only during training) and score them with a frozen answer model. This evidence-indexed score reflects what was stored rather than how it was retrieved. For retrieval, all rollouts in a group share an identical memory state resampled from the current policy, so reward differences reflect retrieval alone. Alternating the two phases yields stage-specific signals, enabling both capabilities to co-evolve.
To address long-horizon credit attribution, we introduce Event-Wise Credit Assignment (ECA), which organizes multiple memory operations into events22
2
In this paper, we distinguish two types of events: construction event and retrieval event, denoted in Sec. 3.1, triggered by an incoming dialogue chunk (construction event) or a query (retrieval event).
ECA uses a query’s supporting evidence to localize the responsible construction events, and reuses evidence-indexed scoring to penalize an unsuccessful retrieval event only when sufficient evidence has been stored. For update-robust evidence attribution, ECA tracks events along the same memory update chain and assigns them the corresponding credit. Since the event stream is fixed by the input data, rollouts share aligned event positions. ECA thus normalizes each event’s feedback against same-position events across the rollout group, rather than at the trajectory level, and broadcasts the resulting advantage to all memory tool calls within that event. Our contributions are summarized as follows:
•
We propose Uni-Mem, a unified memory manager that jointly optimizes agentic construction and retrieval via end-to-end online RL, with Stage-Decoupled GRPO separating the two stages’ contributions to group-relative signals.
•
We introduce Event-Wise Credit Assignment, which localizes outcome feedback to responsible events and computes advantages over position-aligned events across rollouts.
•
Extensive experiments on LoCoMo and LongMemEval demonstrate that Uni-Mem outperforms the strongest RL-based memory manager, supporting the effectiveness of jointly learning memory construction and retrieval.
2 Related Work
Long-term Memory Agents.
Long-term memory enables LLM agents to retain and reuse information beyond their active context.
Retrieval-augmented generation (RAG) provides a basic mechanism for accessing external information (Lewis et al., 2020), while early agent-memory systems consolidate interactions into persistent records and reflections (Park et al., 2023; Zhong et al., 2024).
Subsequent work emphasizes hierarchical storage and consolidation (Packer et al., 2023; Kang et al., 2025; Fang et al., 2026), or structured and associative organization (Chhikara et al., 2025; Xu et al., 2025; Rasmussen et al., 2025; Gutiérrez et al., 2025; Yuan et al., 2026).
Agentic retrieval further enables iterative evidence gathering from retained histories (Yan et al., 2025).
Complementing these architectural advances, Uni-Mem learns a shared policy for multi-turn memory construction and retrieval via adaptively tool calling, allowing both capabilities to adapt jointly to downstream tasks.
Reinforcement Learning for Memory Agents.
RL-based memory methods optimize compressed working states, as in MEM1 (Zhou et al., 2026) and MemAgent (Yu et al., 2026a), external-memory construction (Wang et al., 2025), or retrieval decisions (Tan et al., 2025; Hsu et al., 2026). AgeMem (Yu et al., 2026b) coordinates long-term memory and short-term context.
Memory-R1 (Yan et al., 2026b) trains memory construction and a separate answer agent, whereas Memory-R2 (Yan et al., 2026a) extends this training to long horizons with local and global group-relative credit.
Denser construction supervision is also explored through intermediate rewards and evidence-based attribution (Ma et al., 2026; Shen et al., 2026).
Beyond individual components, Mem-T (Yue et al., 2026) combines online retrieval RL with offline imitation of construction actions selected using hindsight feedback. Different from existing approaches, Uni-Mem explores an end-to-end online RL of a shared-parameter manager policy while addressing the resulting challenges of confounded group comparisons and credit assignment with Decoupled GRPO and event-wise attribution. Appendix A further positions our work with representative methods’ interaction patterns and training paradigm, including a detailed technical comparison with Mem-T.
Figure 2: Overview of Uni-Mem. (a) Uni-Mem unifies memory construction and retrieval through a shared tool-using agent, continually processing an ordered stream of events. (b) Event-Wise Credit Assignment attributes sparse outcome rewards to relevant events. (c) Event-level feedback is propagated to the corresponding memory operations (tool calls). (d) Stage-Decoupled GRPO alternates between construction and retrieval optimization.
3 Preliminaries
3.1 Problem Formulation
Our goal is to train a memory manager agent to maintain a
persistent external memory bank ℳ\mathcal{M} and retrieve
relevant memories for downstream question answering.
We formulate this task as the continual processing of an
ordered stream of events.
Let X=(x1,…,xT)X=(x_{1},\ldots,x_{T}) and Q=(q1,…,qK)Q=(q_{1},\ldots,q_{K}) denote
the sequences of dialogue chunks and queries, respectively.
A construction event etcone_{t}^{\mathrm{con}} is triggered
by a dialogue chunk xtx_{t} and requires the agent to decide
whether and how to revise the memory bank.
A retrieval event ekrete_{k}^{\mathrm{ret}} is triggered
by a query qkq_{k} and requires the agent to gather supporting
evidence 𝒟k⊆ℳ\mathcal{D}_{k}\subseteq\mathcal{M}.
The arrival order of these inputs defines the event stream:
ℰ=(e1,…,eT+K).\mathcal{E}=(e_{1},\ldots,e_{T+K}).
(1)
In our datasets, all queries arrive after the dialogue, so
retrieval operates on the bank built from XX.
For each query, a frozen external answer model gϕg_{\phi}
responds using the retrieved evidence:
yk=gϕ​(qk,𝒟k).y_{k}=g_{\phi}(q_{k},\mathcal{D}_{k}).
(2)
We aim to jointly optimize the agent’s construction and
retrieval behavior under a shared policy to improve
downstream answer accuracy.
3.2 Unified Memory Management Agent
We instantiate the memory manager as a tool-using agent
governed by a shared policy πθ\pi_{\theta}, as illustrated
in Fig. 2 (a).
The memory bank stores records of the form
mi=(i,ci,contenti)m_{i}=(i,c_{i},\mathrm{content}_{i}), where ii is a unique
record ID, ci∈{1,…,T}c_{i}\in\{1,\ldots,T\} is the index of the dialogue chunk
from which the record was created,
and contenti\mathrm{content}_{i} is the stored textual content.
Both event types support multiple rounds of tool interaction.
At each step, the agent selects a tool call based on the
event type, the triggering input, and the preceding tool
calls and observations within the event.
It then executes the call and uses the returned result
to guide its next decision.
During construction, the agent can use SEARCH,
CREATE, UPDATE, and DELETE
to inspect and revise the bank.
During retrieval, it uses SEARCH to gather evidence,
potentially conducting successive searches based on
earlier results.
For both event types, the agent terminates by calling
FINISH, which also submits the selected evidence
𝒟k\mathcal{D}_{k} for retrieval events. Details of the memory
representation and tool specifications are provided in
Appendices B.1
and B.2, respectively.
4 Method
In this section, we present the training framework of
Uni-Mem, as shown in Fig. 2. Online RL alternates between a construction phase
and a retrieval phase (Section 4.2). We first describe
Event-Wise Credit Assignment (Section 4.1), which
assigns rewards to events in each phase, and then the optimization
procedure, which combines an SFT cold start with Stage-Decoupled GRPO.
4.1 Event-Wise Credit Assignment
In long-horizon memory management, the outcome reward arrives only
after long sequences of tool calls, making it hard to
identify which tool calls contributed. We therefore
introduce Event-Wise Credit Assignment (ECA), which localizes
each query’s reward to the events responsible for it.
Let yk⋆y_{k}^{\star} denote the reference answer to query qkq_{k}. A fixed
LLM judge 𝒥ψ\mathcal{J}_{\psi} evaluates the generated response yky_{k}
and assigns the task reward
Rk=𝒥ψ​(qk,yk,yk⋆)∈{0,1}.R_{k}=\mathcal{J}_{\psi}(q_{k},y_{k},y_{k}^{\star})\in\{0,1\}.
(3)
Evidence-to-event localization.
Let Ek⊆{1,…,T}E_{k}\subseteq\{1,\ldots,T\} denote the indices of the
dialogue chunks that contain the annotated evidence for qkq_{k}, which
are used only during training. The construction events related to
qkq_{k} are
ℰkcon={etcon∣t∈Ek}.\mathcal{E}_{k}^{\mathrm{con}}=\{e_{t}^{\mathrm{con}}\mid t\in E_{k}\}.
(4)
Evidence-indexed memory scoring.
An incorrect answer may result from insufficient stored information
or from unsuccessful retrieval. To distinguish the two, for a memory
bank ℬ\mathcal{B}, Index⁡(Ek,ℬ)\operatorname{Index}(E_{k},\mathcal{B}) selects
the current versions of the records written in
ℰkcon\mathcal{E}_{k}^{\mathrm{con}}, and the frozen answer model answers
qkq_{k} from these records:
y~k​(ℬ)\displaystyle\widetilde{y}_{k}(\mathcal{B})
=gϕ​(qk,Index⁡(Ek,ℬ)),\displaystyle=g_{\phi}\!\left(q_{k},\operatorname{Index}(E_{k},\mathcal{B})\right),
(5)
𝒮k​(ℬ)\displaystyle\mathcal{S}_{k}(\mathcal{B})
=𝒥ψ​(qk,y~k​(ℬ),yk⋆)∈{0,1}.\displaystyle=\mathcal{J}_{\psi}\!\left(q_{k},\widetilde{y}_{k}(\mathcal{B}),y_{k}^{\star}\right)\in\{0,1\}.
Thus, 𝒮k​(ℬ)\mathcal{S}_{k}(\mathcal{B}) serves as a proxy for whether the stored
information suffices to answer qkq_{k}, without involving the learned
retrieval policy.
Event-level credit.
In the construction phase, rollout nn produces a final bank
ℬ(n)\mathcal{B}^{(n)}, and each such event accumulates the
scores of its related queries:
R¯con,e(n)=∑k:e∈ℰkcon𝒮k(ℬ(n)).\bar{R}_{\mathrm{con},e}^{(n)}=\sum_{k:\,e\in\mathcal{E}_{k}^{\mathrm{con}}}\mathcal{S}_{k}(\mathcal{B}^{(n)}).
(6)
Note that events that are not related to any query in a rollout receive no
return and are excluded from the group at their position.
In the retrieval phase, all rollouts share a bank
ℳ¯\overline{\mathcal{M}}, and the retrieval event triggered by qkq_{k}
receives
R¯ret,ek(n)={Rk(n),Rk(n)=1∨𝒮k​(ℳ¯)=1,⊥,otherwise,\bar{R}_{\mathrm{ret},e_{k}}^{(n)}=\begin{cases}R_{k}^{(n)},&R_{k}^{(n)}\!=\!1\lor\mathcal{S}_{k}(\overline{\mathcal{M}})\!=\!1,\\
\bot,&\text{otherwise},\end{cases}
(7)
where ⊥\bot marks an excluded event. A failed retrieval is thus
penalized only when the shared bank supports a correct answer;
otherwise, the failure is attributed to construction, and the event
contributes neither to group statistics nor to policy gradients.
The resulting returns are converted into event-level advantages in
Section 4.2.
Update-robust evidence attribution.
Records written in ℰkcon\mathcal{E}_{k}^{\mathrm{con}} may later be
updated or deleted by events outside ℰkcon\mathcal{E}_{k}^{\mathrm{con}},
which then receive no credit. We therefore track the update lineage
of these records during training and form a rollout-specific credit
set ℰ~kcon,(n)⊇ℰkcon\widetilde{\mathcal{E}}_{k}^{\mathrm{con},(n)}\supseteq\mathcal{E}_{k}^{\mathrm{con}}
that additionally contains such events, which share the same score
𝒮k​(ℬ(n))\mathcal{S}_{k}(\mathcal{B}^{(n)}). With this attribution,
ℰ~kcon,(n)\widetilde{\mathcal{E}}_{k}^{\mathrm{con},(n)} replaces
ℰkcon\mathcal{E}_{k}^{\mathrm{con}} in Eq. (6),
while memory selection in Eq. (5) still uses
the original EkE_{k}. See Appendix C.2.
4.2 Unified Manager Policy Optimization
In this section, we describe the optimization procedure for
our Uni-Mem. We first cold-start the shared
policy through supervised fine-tuning (SFT) on demonstration
data, then further optimize it with Stage-Decoupled GRPO, as shown in Fig. 2(d).
Supervised cold start.
To establish basic construction and retrieval capabilities, we
initialize the shared policy through supervised fine-tuning on memory
management demonstrations.
Details of the demonstration data and training objective
are provided in Appendix C.1.
Stage-Decoupled GRPO.
Directly applying GRPO to end-to-end rollouts can conflate
construction and retrieval contributions when both stages
vary, obscuring the learning signal for each stage.
To address this, we introduce Stage-Decoupled GRPO, which
alternates construction and retrieval optimization while
controlling variation from the other stage within each
rollout group. Construction rollouts are evaluated without
learned retrieval, while retrieval rollouts share a fixed
memory bank. Both phases update the same shared manager
πθ\pi_{\theta}.
At each iteration, a snapshot πθold\pi_{\theta_{\mathrm{old}}}
of the current manager samples NN sibling rollouts per group
for the scheduled phase. Construction rollouts process the
same dialogues from empty banks; retrieval rollouts for each
query share a bank ℳ¯\overline{\mathcal{M}} that this snapshot
constructs once by processing the dialogue before retrieval.
Algorithm 1 assigns
event-level returns R¯p,e(n)\bar{R}_{p,e}^{(n)}, which are used to update
πθ\pi_{\theta}.
Algorithm 1 Phase-Specific Event Reward Assignment
1:
Phase pp; NN sampled rollouts with their memory banks;
query annotations {qk,Ek,yk⋆}k=1K\{q_{k},E_{k},y_{k}^{\star}\}_{k=1}^{K}
2:
Event returns {R¯p,e(n)}\{\bar{R}_{p,e}^{(n)}\}, with ⊥\bot marking
excluded events
3:
if p=conp=\mathrm{con} then
4:
for n=1,…,Nn=1,\ldots,N do
5:
Form ℰ~kcon,(n)\widetilde{\mathcal{E}}_{k}^{\mathrm{con},(n)} from
Eq. (4) and the update
lineage, for all kk
6:
sk(n)←𝒮k​(ℬ(n))s_{k}^{(n)}\leftarrow\mathcal{S}_{k}(\mathcal{B}^{(n)}),
for all kk
7:
R¯con,e(n)←∑k:e∈ℰ~kcon,(n)sk(n)\bar{R}_{\mathrm{con},e}^{(n)}\leftarrow\sum_{k:\,e\in\widetilde{\mathcal{E}}_{k}^{\mathrm{con},(n)}}s_{k}^{(n)},
for related events ee
8:
end for
9:
else
10:
for k=1,…,Kk=1,\ldots,K do
11:
Evaluate task rewards {Rk(n)}n=1N\{R_{k}^{(n)}\}_{n=1}^{N}
12:
if ∃n:Rk(n)=0\exists n:R_{k}^{(n)}=0 then
13:
Evaluate 𝒮k​(ℳ¯)\mathcal{S}_{k}(\overline{\mathcal{M}})
on the shared bank
14:
end if
15:
Assign R¯ret,ek(n)\bar{R}_{\mathrm{ret},e_{k}}^{(n)} by
Eq. (7), for all nn
16:
end for
17:
end if
18:
return {R¯p,e(n)}\{\bar{R}_{p,e}^{(n)}\}
Using the assigned returns, we group corresponding events across
rollouts by their triggering dialogue-chunk or query ID. Since all
rollouts of a training sample process the same stream of dialogue
chunks and queries, events triggered by the same input are aligned
across rollouts, even though the tool calls performed within
them may differ. Differences in the preceding memory states are
treated as part of the sampled trajectory, as in standard GRPO.
Construction events without a return and retrieval events marked
⊥\bot in Eq. (7) are excluded from both group
statistics and policy optimization. Groups with fewer than two valid
returns or zero variance are discarded.
For each retained group in phase
p∈{con,ret}p\in\{\mathrm{con},\mathrm{ret}\}, we compute
A^p,e(n)=R¯p,e(n)−μp,eσp,e+δ,\widehat{A}_{p,e}^{(n)}=\frac{\bar{R}_{p,e}^{(n)}-\mu_{p,e}}{\sigma_{p,e}+\delta},
(8)
where μp,e\mu_{p,e} and σp,e\sigma_{p,e} are the mean and
standard deviation of the valid returns,
and δ>0\delta>0 is a numerical stabilizer. Under event-level credit
assignment, all tool calls within an event share its advantage, as illustrated in Fig. 2(c).
Following GRPO, every token of a retained tool call is
optimized with the clipped surrogate, using the advantage of its
event:
𝒥p(θ)=𝔼n[1|τp(n)|∑o∈τp(n)min(ρoA^p,e⁡(o)(n),clip(ρo,−ϵ,+ϵ)A^p,e⁡(o)(n))],\begin{split}\mathcal{J}_{p}(\theta)=\mathbb{E}_{n}\Big[&\tfrac{1}{|\tau_{p}^{(n)}|}\textstyle\sum_{o\in\tau_{p}^{(n)}}\min\big(\rho_{o}\widehat{A}_{p,e(o)}^{(n)},\\
&\operatorname{clip}(\rho_{o},1\!-\!\epsilon,1\!+\!\epsilon)\,\widehat{A}_{p,e(o)}^{(n)}\big)\Big],\end{split}
(9)
where τp(n)\tau_{p}^{(n)} contains the tokens of the tool calls in rollout
nn retained in phase pp, ρo\rho_{o} is the token-level likelihood ratio
between πθ\pi_{\theta} and πθold\pi_{\theta_{\mathrm{old}}}, and e⁡(o)e(o) is
the event containing token oo. Rollouts without retained tokens are
skipped; no KL penalty is used.
The two phases alternate according to
𝜿=(κcon,κret)\boldsymbol{\kappa}=(\kappa_{\mathrm{con}},\kappa_{\mathrm{ret}}):
κcon\kappa_{\mathrm{con}} consecutive construction updates are followed
by κret\kappa_{\mathrm{ret}} retrieval updates (denoted CC and RR in
Figure 4). The RL objective is
𝒥RL​(θ)=λcon​𝒥con​(θ)+λret​𝒥ret​(θ),\mathcal{J}_{\mathrm{RL}}(\theta)=\lambda_{\mathrm{con}}\mathcal{J}_{\mathrm{con}}(\theta)+\lambda_{\mathrm{ret}}\mathcal{J}_{\mathrm{ret}}(\theta),
(10)
where λcon=1\lambda_{\mathrm{con}}=1 and λret=0\lambda_{\mathrm{ret}}=0
during construction updates, and vice versa during retrieval updates.
5 Experiments
5.1 Experiment Setup
Datasets and Evaluation Metrics.
We evaluate our framework on LoCoMo Maharana et al. (2024) and LongMemEval Wu et al. (2025).
For a fair comparison, we follow the training data configuration of Memory-R2 Yan et al. (2026a), partitioning LoCoMo into RL training, validation, and test sets with a 2:1:7 split. We use LongMemEval for out-of-distribution evaluation. We report LLM-as-a-Judge scores (JJ) using the official Mem0 (Chhikara et al., 2025) judge prompt.
Baselines. We compare Uni-Mem against two groups of baselines.
(I) Training-free methods include RAG for conversation retrieval, as well as A-Mem (Xu et al., 2025), Mem0 (Chhikara et al., 2025), and MemoryOS (Kang et al., 2025).
(II) Training-based methods include MEM1 (Zhou et al., 2026), MemAgent (Yu et al., 2026a), Memory-R1 (Yan et al., 2026b), and Memory-R2 (Yan et al., 2026a), and Mem-T (Yue et al., 2026). These methods typically use RL to optimize memory phase. For a fair comparison, we re-evaluate the Memory-R2 (OSS) variant, which uses GPT-OSS-120B OpenAI (2025) as its answer model. Details of baselines are in Appendix D
Implementation Details.
We use Qwen3-4B-Instruct Yang et al. (2025) as the backbone of our memory manager. GPT-OSS-120B serves as the external answer model, and GPT-4o-mini OpenAI (2024) serves as the judge model. Additional training details are provided in Appendix C.
5.2 Main Results
Model
Multi
Open
Single
Temp.
Overall
Training-free Methods
RAG
15.06
19.28
12.79
5.43
12.17
A-MEM
40.78
31.32
46.90
28.68
40.78
Mem0
61.92
48.19
65.20
38.76
53.30
MemoryOS
48.12
38.55
63.14
24.81
51.26
Training-based methods
MEM1
41.29
43.10
57.22
44.73
50.78
MemAgent
70.65
62.07
78.61
56.96
71.52
Memory-R1
62.34
40.96
67.81
49.61
61.51
Memory-R2†
69.15
61.29
78.61
57.38
71.26
Memory-R2 (OSS)†
74.13
70.97
81.15
74.68
77.87
Mem-T†
68.16
58.06
81.15
76.79
76.49
Uni-Mem(w/o sft)
81.09
75.81
88.79
62.87
80.99
Uni-Mem
87.06
79.03
91.51
86.08
88.80
Table 1:
Performance comparison on LoCoMo (LLM-as-a-Judge, %). Bold and underlined results indicate the best and second-best, respectively. † marks our re-evaluation of the officially released checkpoints under the same protocol as Uni-Mem. Other baseline results are taken from Yan et al. (2026a) under their original protocol, as reference only (Appendix E.1).
Superiority over existing baselines.
As shown in Table 1, Uni-Mem achieves the best performance across all four question types, with an overall JJ score of 88.80%.
It surpasses the strongest baseline, Memory-R2 (OSS), by 10.93 percentage points and Mem-T by 12.31 points, with gains of 12.93 and 9.29 points over the strongest competing results on multi-hop and temporal questions, respectively.
Additionally, Uni-Mem still achieves 80.99% without SFT, outperforming the strongest baseline by 3.12%.
These results support the effectiveness of jointly optimizing memory construction and retrieval through Stage-Decoupled GRPO and Event-Wise Credit Assignment, which provide stage-specific learning signals and localized supervision for memory operations.
Out-of-Distribution Generalization.
To assess whether the memory-management capabilities learned
on LoCoMo transfer to a different benchmark, we evaluate
Uni-Mem on LongMemEval-Oracle and
LongMemEval-S (Wu et al., 2025).
Figure 3 presents the category-level results. On LongMemEval-Oracle, Uni-Mem substantially outperforms all baselines on single-session-preference and multi-session questions. It also achieves notable gains in knowledge update and temporal reasoning over Mem-T, a prior unified memory manager. On the larger-scale LongMemEval-S, Uni-Mem consistently outperforms Memory-R2 and its variants across all six task categories. These results demonstrate the transferability of its learned memory-management capabilities beyond the LoCoMo training distribution.
Figure 3: OOD accuracy (JJ, %) on LongMemEval-Oracle
and LongMemEval-S. SS: single-session; asst.: assistant;
pref.: preference. Both panels use a 0–100% scale.
Mem-T results are shown only for Oracle.
Model
Multi
Open
Single
Temp.
Overall
Qwen3-4B-Instruct
M0M_{0}
78.61
70.97
84.04
57.81
76.58
M1M_{1}
81.09
69.35
85.57
54.01
76.95
M2M_{2}
82.59
72.58
86.59
55.27
78.24
M3M_{3}
84.08
66.13
87.10
65.82
80.72
M4M_{4}
81.09
75.81
88.79
62.87
80.99
Qwen3-4B-Instruct after SFT
M0M_{0}
82.09
69.35
88.12
77.22
83.56
M1M_{1}
84.08
70.97
88.62
75.95
84.02
M2M_{2}
84.08
74.19
90.66
83.97
87.05
M3M_{3}
88.56
77.42
90.32
84.81
88.06
M4M_{4}
87.06
79.03
91.51
86.08
88.80
Table 2:
Ablation study on LoCoMo (JJ, %) with raw or SFT-initialized Qwen3-4B-Instruct as start point (M0M_{0}).
5.3 Ablation Study
Ablation setting. To validate the effectiveness of Uni-Mem’s module
designs, we conduct an incremental ablation study. Specifically, M0M_{0} serves as the starting point, using either the original or SFT-initialized base manager. M1M_{1} jointly optimizes memory construction and retrieval using standard GRPO. M2M_{2} adds Event-Wise Credit Assignment to M1M_{1}. M3M_{3} replaces standard GRPO in M2M_{2} with Stage-Decoupled GRPO. Finally, M4M_{4} incorporates update-robust evidence attribution into M3M_{3}, yielding the full Uni-Mem framework. Table 2 presents the ablation results using either the original or SFT-initialized Qwen3-4B-Instruct model. The single-stage rollout and reward-assignment procedures for
M1M_{1} and M2M_{2} are detailed in Appendix E.2.
Event-wise Credit Assignment.
Sparse outcome rewards offer limited guidance for credit assignment across hundreds of memory operations. Adding ECA (M1→M2M_{1}\rightarrow M_{2}) improves overall JJ by 1.29 and 3.03 percentage points without and with SFT, respectively. These gains support the effectiveness of ECA, which localizes feedback to relevant events and their operations. Update-robust attribution (M3→M4M_{3}\rightarrow M_{4}) continually improves the performance, supporting the additional benefit of credit tracing along memory update lineages.
Stage-Decoupled GRPO.
M0→M1M_{0}\rightarrow M_{1} brings very limited gain (only 0.37%), which indicates that group-relative advantages can conflate construction and retrieval effects.
Stage-Decoupled GRPO (M2→M3M_{2}\rightarrow M_{3}) improves overall JJ by 2.48 percentage points without SFT. In particular, temporal accuracy rises by 10.55 points in this setting, and multi-hop accuracy increases by 4.48 points after SFT.
These gains support the effectiveness of alternating optimization and controlling cross-stage variation in Stage-Decoupled GRPO.
5.4 Further Analysis
Memory
Search
Multi
Open
Single
Temp.
Overall
P0P_{0}
Q0Q_{0}
78.61
70.97
84.04
57.81
76.58
P0P_{0}
Q+Q_{+}
86.07
67.74
88.29
62.45
81.08
P+P_{+}
Q0Q_{0}
77.61
69.35
89.98
81.86
84.76
P+P_{+}
Q+Q_{+}
87.06
79.03
91.51
86.08
88.80
Table 3:
Evaluation of memory construction and retrieval capabilities on LoCoMo (JJ, %) by cross-combining memory banks and retrieval policies.
P0P_{0} and P+P_{+} denote memory banks constructed by the base Qwen3-4B manager and the trained Uni-Mem manager, respectively;
Q0Q_{0} and Q+Q_{+} denote their corresponding retrieval policies.
Evaluating Construction and Retrieval Capabilities.
We analyze the memory construction and retrieval capabilities of Uni-Mem.
To assess the improvements in each capability, we cross-combine memory banks and retrieval policies from the base and trained managers, as shown in Table 3.
With the base retrieval policy fixed, replacing P0P_{0} with P+P_{+} improves the overall JJ score by 8.18 percentage points, demonstrating the increased downstream utility of the constructed memories.
Conversely, replacing Q0Q_{0} with Q+Q_{+} improves the overall score by 4.50 and 4.04 points on P0P_{0} and P+P_{+}, respectively, indicating more effective evidence retrieval from the same memory bank.
Combining both trained capabilities achieves the highest overall score of 88.80%.
These results show that Uni-Mem improves both memory construction and retrieval, with each capability contributing gains even when paired with its untrained counterpart.
Figure 4:
Effects of manager training, scale, and alternating schedules on LoCoMo.
(a) Training and manager scale. The arrow shows the gain from SFT followed by RL.
(b) Construction–retrieval update ratio.
(c) Alternation granularity at a fixed 1:11:1 ratio.
CC and RR denote consecutive construction and retrieval updates per cycle.
Effect of Alternation Frequency and Update Balance.
To determine how construction and retrieval should be
combined for joint learning within a shared policy,
we compare training schedules under Stage-Decoupled GRPO
(Figure 4(b,c)).
Each schedule C:RC:R consists of CC consecutive construction
updates followed by RR retrieval updates. As shown in Figure 4(c), extending the update blocks from 1:11:1 to 32:3232:32 reduces
the overall JJ score from 88.80%88.80\% to 83.75%83.75\%,
indicating that less frequent alternation hinders
shared-policy learning. As shown in Figure 4(b), the 1:81:8 and 8:18:1 schedules reduce performance by
4.32 and 2.85 percentage points, respectively, indicating
that allocating too many updates to either stage also
harms unified learning. These results indicate that both balanced allocation and frequent alternation are important for coordinated construction and retrieval learning under a shared policy.
Effect of Training and Manager Scale.
We compare Uni-Mem with the base Qwen3-4B-Instruct and four larger off-the-shelf Qwen3.5 models, all serving as memory managers within the same agent harness.
As shown in Figure 4(a), SFT followed by RL improves the 4B manager’s JJ score from 76.58% to 88.80%.
Uni-Mem also outperforms all off-the-shelf managers, including the strongest one, Qwen3.5-27B (87.05%), and the largest one, Qwen3.5-397B-A17B (85.86%).
These results demonstrate that a small manager trained jointly for construction and retrieval can outperform substantially larger off-the-shelf alternatives.
Figure 5:
Case studies comparing Uni-Mem and the base Qwen3-4B manager in (a) memory construction and (b) memory retrieval.
5.5 Case Study
As shown in Figure 5, we compare
Uni-Mem with the base Qwen3-4B memory manager
to illustrate the capabilities acquired through our training.
In Figure 5(a), both memory banks initially encode the mistaken belief that Jolene is married, with their underlying entries differing in wording and content.
Following Jolene’s clarification, Uni-Mem locates the conflicting entry through SEARCH and corrects it through UPDATE, while the base manager uses CREATE and leaves contradictory claims unresolved.
This illustrates Uni-Mem’s ability to
identify and resolve memory conflicts, maintaining
an internally consistent memory bank.
In Figure 5(b), Uni-Mem uses the ceramic bowl reference discovered during its initial search to refine the next query and retrieve the answer-bearing memory. This highlights Uni-Mem’s ability to
iteratively refine its search based on intermediate
results to retrieve answer-relevant evidence.
Conclusion
We presented Uni-Mem, a unified memory manager
that jointly learns agentic memory construction and
retrieval through end-to-end online reinforcement learning
under a shared policy.
By combining Stage-Decoupled GRPO with Event-Wise Credit
Assignment, Uni-Mem obtains stage-specific learning
signals and assigns credit to relevant memory operations,
enabling coordinated online learning of construction and
retrieval over long horizons.
Experiments demonstrate that Uni-Mem outperforms
strong RL-based memory baselines on LoCoMo and generalizes
to LongMemEval.
Limitations
Uni-Mem unifies memory construction and retrieval through online RL, enabling coordinated policy learning with stage-specific optimization and event-wise credit assignment. However, training remains resource-intensive because rewards require completing downstream QA and obtaining judge-model evaluations. These evaluations typically rely on external APIs or stronger deployed models, introducing more costs and latency. Running judge evaluations concurrently or adopting asynchronous RL could alleviate the latency bottleneck. Future work will explore more robust process-level supervision to reduce reliance on outcome-based evaluations.
References
Chhikara et al. (2025)
P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Yadav
Mem0: building production-ready AI agents with scalable long-term memory.
External Links: 2504.19413,
Document,
Link
Cited by: §B.1,
Appendix D,
§E.1,
§1,
§2,
§5.1,
§5.1.
Fang et al. (2026)
J. Fang, X. Deng, H. Xu, Z. Jiang, Y. Tang, Z. Xu, S. Deng, Y. Yao, M. Wang, S. Qiao, H. Chen, and N. Zhang
LightMem: lightweight and efficient memory-augmented generation.
In International Conference on Learning Representations,
Vol. 2026, pp. 98706–98729.
External Links: Link
Cited by: §2.
Gutiérrez et al. (2025)
B. J. Gutiérrez, Y. Shu, W. Qi, S. Zhou, and Y. Su
From RAG to memory: non-parametric continual learning for large language models.
In Proceedings of the 42nd International Conference on Machine Learning,
Proceedings of Machine Learning Research, Vol. 267, pp. 21497–21515.
External Links: Link
Cited by: §2.
Hsu et al. (2026)
H. Hsu, N. L. Kuang, B. Liu, Z. Yao, and Y. He
Organize then retrieve: hierarchical memory navigation for efficient agents.
External Links: 2606.11680,
Document,
Link
Cited by: §2.
Jin et al. (2025)
B. Jin, H. Zeng, Z. Yue, J. Yoon, S. O. Arik, D. Wang, H. Zamani, and J. Han
Search-R1: training LLMs to reason and leverage search engines with reinforcement learning.
In Second Conference on Language Modeling,
External Links: Link
Cited by: Appendix A.
Kang et al. (2025)
J. Kang, M. Ji, Z. Zhao, and T. Bai
Memory OS of AI agent.
In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,
Suzhou, China, pp. 25961–25970.
External Links: ISBN 979-8-89176-332-6,
Document,
Link
Cited by: Appendix D,
§2,
§5.1.
Lewis et al. (2020)
P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela
Retrieval-augmented generation for knowledge-intensive NLP tasks.
In Advances in Neural Information Processing Systems,
Vol. 33, pp. 9459–9474.
External Links: Link
Cited by: §2.
Ma et al. (2026)
W. Ma, X. Feng, L. Huang, X. Feng, Z. Ma, J. Xu, J. Gao, J. Hao, R. He, and B. Qin
Fine-Mem: fine-grained feedback alignment for long-horizon memory management.
In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),
San Diego, California, United States, pp. 19666–19684.
External Links: ISBN 979-8-89176-390-6,
Document,
Link
Cited by: §2.
Maharana et al. (2024)
A. Maharana, D. Lee, S. Tulyakov, M. Bansal, F. Barbieri, and Y. Fang
Evaluating very long-term conversational memory of LLM agents.
In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),
Bangkok, Thailand, pp. 13851–13870.
External Links: Document,
Link
Cited by: §C.1,
§5.1.
OpenAI (2024)
OpenAI
GPT-4o system card.
External Links: 2410.21276,
Document,
Link
Cited by: §5.1.
OpenAI (2025)
OpenAI
gpt-oss-120b & gpt-oss-20b model card.
External Links: 2508.10925,
Document,
Link
Cited by: Appendix D,
§5.1.
Packer et al. (2023)
C. Packer, S. Wooders, K. Lin, V. Fang, S. G. Patil, I. Stoica, and J. E. Gonzalez
MemGPT: towards LLMs as operating systems.
External Links: 2310.08560,
Document,
Link
Cited by: §1,
§2.
Park et al. (2023)
J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein
Generative agents: interactive simulacra of human behavior.
In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology,
UIST ’23, New York, NY, USA, pp. 1–22.
External Links: Document,
Link
Cited by: §2.
Rasmussen et al. (2025)
P. Rasmussen, P. Paliychuk, T. Beauvais, J. Ryan, and D. Chalef
Zep: a temporal knowledge graph architecture for agent memory.
External Links: 2501.13956,
Document,
Link
Cited by: §2.
Shao et al. (2024)
Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo
DeepSeekMath: pushing the limits of mathematical reasoning in open language models.
External Links: 2402.03300,
Document,
Link
Cited by: §1.
Shen et al. (2026)
Z. Shen, Z. Wu, F. Lai, S. Lian, and Y. Rao
MemBuilder: reinforcing LLMs for long-term memory construction via attributed dense rewards.
In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),
San Diego, California, United States, pp. 27868–27887.
External Links: ISBN 979-8-89176-390-6,
Document,
Link
Cited by: §2.
Tan et al. (2025)
Z. Tan, J. Yan, I. Hsu, R. Han, Z. Wang, L. Le, Y. Song, Y. Chen, H. Palangi, G. Lee, A. R. Iyer, T. Chen, H. Liu, C. Lee, and T. Pfister
In prospect and retrospect: reflective memory management for long-term personalized dialogue agents.
In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),
Vienna, Austria, pp. 8416–8439.
External Links: ISBN 979-8-89176-251-0,
Document,
Link
Cited by: §2.
Trivedi et al. (2023)
H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal
Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions.
In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),
Toronto, Canada, pp. 10014–10037.
External Links: Document,
Link
Cited by: Appendix A.
Wang et al. (2023)
W. Wang, L. Dong, H. Cheng, X. Liu, X. Yan, J. Gao, and F. Wei
Augmenting language models with long-term memory.
In Advances in Neural Information Processing Systems,
Vol. 36, pp. 74530–74543.
External Links: Document,
Link
Cited by: §1.
Wang et al. (2025)
Y. Wang, R. Takanobu, Z. Liang, Y. Mao, Y. Hu, J. McAuley, and X. Wu
Mem-α\alpha: learning memory construction via reinforcement learning.
External Links: 2509.25911,
Document,
Link
Cited by: Appendix A,
§1,
§2.
Wu et al. (2025)
D. Wu, H. Wang, W. Yu, Y. Zhang, K. Chang, and D. Yu
LongMemEval: benchmarking chat assistants on long-term interactive memory.
In International Conference on Learning Representations,
Vol. 2025, pp. 86809–86836.
External Links: Link
Cited by: §5.1,
§5.2.
Xiong et al. (2026)
Z. Xiong, Y. Lin, W. Xie, P. He, Z. Liu, J. Tang, H. Lakkaraju, and Z. Xiang
How memory management impacts LLM agents: an empirical study of experience-following behavior.
In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),
San Diego, California, United States, pp. 623–645.
External Links: ISBN 979-8-89176-390-6,
Document,
Link
Cited by: §1.
Xu et al. (2025)
W. Xu, Z. Liang, K. Mei, H. Gao, J. Tan, and Y. Zhang
A-Mem: agentic memory for LLM agents.
In Advances in Neural Information Processing Systems,
Vol. 38, pp. 17577–17604.
External Links: Document,
Link
Cited by: Appendix D,
§1,
§2,
§5.1.
Yan et al. (2025)
B. Y. Yan, C. Li, H. Qian, S. Lu, and Z. Liu
General agentic memory via deep research.
External Links: 2511.18423,
Document,
Link
Cited by: §2.
Yan et al. (2026a)
S. Yan, A. Bahloul, E. Nie, S. Schwarzmann, R. Trivisonno, V. Tresp, and Y. Ma
Memory-R2: fair credit assignment for long-horizon memory-augmented LLM agents.
External Links: 2605.21768,
Document,
Link
Cited by: Appendix A,
Appendix D,
§E.1,
§E.1,
Table 8,
§1,
§2,
§5.1,
§5.1,
Table 1,
Table 1.
Yan et al. (2026b)
S. Yan, X. Yang, Z. Huang, E. Nie, Z. Ding, Z. Li, X. Ma, J. Bi, K. Kersting, J. Z. Pan, H. Schuetze, V. Tresp, and Y. Ma
Memory-R1: enhancing large language model agents to manage and utilize memories via reinforcement learning.
In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),
San Diego, California, United States, pp. 12805–12825.
External Links: ISBN 979-8-89176-390-6,
Document,
Link
Cited by: Appendix A,
Appendix D,
§2,
§5.1.
Yang et al. (2025)
A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu
Qwen3 technical report.
External Links: 2505.09388,
Document,
Link
Cited by: §5.1.
Yao et al. (2023)
S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao
ReAct: synergizing reasoning and acting in language models.
In International Conference on Learning Representations,
External Links: Link
Cited by: Appendix A.
Yu et al. (2026a)
H. Yu, T. Chen, J. Feng, J. Chen, W. Dai, Q. Yu, Y. Zhang, W. Ma, J. Liu, M. Wang, and H. Zhou
MemAgent: reshaping long-context LLM with multi-conv RL-based memory agent.
In International Conference on Learning Representations,
Vol. 2026, pp. 39458–39486.
External Links: Link
Cited by: Appendix A,
Appendix D,
§2,
§5.1.
Yu et al. (2026b)
Y. Yu, L. Yao, Y. Xie, Q. Tan, J. Feng, Y. Li, and L. Wu
Agentic memory: learning unified long-term and short-term memory management for large language model agents.
In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),
San Diego, California, United States, pp. 21457–21483.
External Links: ISBN 979-8-89176-390-6,
Document,
Link
Cited by: §1,
§2.
Yuan et al. (2026)
X. Yuan, S. Di, J. Tang, L. Zheng, and J. Yin
Towards self-cognitive exploration: metacognitive knowledge graph retrieval augmented generation.
In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1,
KDD ’26, New York, NY, USA, pp. 1844–1855.
External Links: Document,
Link
Cited by: §2.
Yue et al. (2026)
Y. Yue, B. Peng, X. Fan, J. Guo, Q. Li, and Y. Zhang
Mem-T: densifying rewards for long-horizon memory agents.
External Links: 2601.23014,
Document,
Link
Cited by: Appendix A,
Appendix D,
§1,
§2,
§5.1.
Zhong et al. (2024)
W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang
MemoryBank: enhancing large language models with long-term memory.
Proceedings of the AAAI Conference on Artificial Intelligence 38 (17), pp. 19724–19731.
External Links: Document,
Link
Cited by: §2.
Zhou et al. (2026)
Z. Zhou, A. Qu, Z. Wu, S. Kim, A. Prakash, D. Rus, B. K. H. Low, and P. P. Liang
MEM1: learning to synergize memory and reasoning for efficient long-horizon agents.
In International Conference on Learning Representations,
Vol. 2026, pp. 58413–58438.
External Links: Link
Cited by: Appendix A,
Appendix D,
§2,
§5.1.
Appendix Overview
This appendix provides supplementary methodological details and experimental analyses to complement the main text and facilitate reproducibility, organized as follows:
Appendix A positions Uni-Mem relative to existing memory agents in terms of interaction patterns and training paradigms, highlighting its differences from Mem-T.
Appendix B details the memory representation, provenance tracking, tool interfaces, and execution protocol.
Appendix C describes supervised cold-start training, event-wise credit computation, RL hyperparameters, and computational resources.
Appendix D summarizes the baseline methods.
Appendix E documents the Memory-R2 re-evaluation, judge-prompt sensitivity, and ablation configurations.
Appendix F presents training curves and detailed experimental results.
Finally, Appendix G provides the prompt templates for memory management, answer generation, and LLM-as-a-Judge evaluation.
Appendix A Positioning Uni-Mem with Related Methods
We position Uni-Mem with RL-trained memory agents from two
perspectives: how the agent interacts with tools within a single
construction or retrieval event, and how the two stages are trained.
Table 4 gives an overview.
Interaction
Training
Method
Con.
Ret.
Con.
Ret.
MEM1
S
–
RL
–
MemAgent
S
–
RL
–
Memory-R1
S
Fixed
RL
Fixed
Mem-α\alpha
S
Fixed
RL
Fixed
Memory-R2
S
Fixed
RL
Fixed
Mem-T
P
M
Offline
RL
Uni-Mem
M
M
RL
RL
Table 4: Interaction pattern and training of construction (Con.)
and retrieval (Ret.). S: single-turn; P: fixed multi-step pipeline;
M: multi-turn with tool observations; Fixed: similarity retrieval
without learning; –: no explicit stage. Offline: imitation of
hindsight-filtered actions.
Memory manager as an agent.
Although several prior works describe their memory modules as agents,
the memory manager in most of them acts as a one-step policy: given an
input and a context prepared by a fixed procedure, it produces its
operations in a single generation, and their results are never fed
back to it. We refer to such events as single-turn. In Uni-Mem,
the manager is instead an agent that interacts with the memory bank
over multiple turns, as in ReAct (Yao et al., 2023). At each step, it
chooses a tool call, observes the returned result, and decides what to
do next, including when to stop.
Among prior methods, MEM1 (Zhou et al., 2026) interacts with an
external environment over multiple turns, but its memory is an
internal state rewritten in one generation at every turn, with no tool
access to the memory itself. MemAgent (Yu et al., 2026a) overwrites
a fixed-length memory once per chunk and answers from the final memory.
Memory-R1 (Yan et al., 2026b), Memory-R2 (Yan et al., 2026a),
and Mem-α\alpha (Wang et al., 2025) generate the memory operations for
each input in one pass, and answer questions with fixed similarity
retrieval (embedding or BM25).
Mem-T (Yue et al., 2026) performs multi-turn search at query time,
but its construction is a fixed two-step pipeline: a formation step
proposes memory candidates, and an evolution step decides one
operation for each candidate. Neither step issues its own searches or
sees the execution results of earlier operations.
Why an agentic manager helps.
The key difference is who decides what the manager sees. In
single-turn methods, the context for each decision is prepared by a
fixed similarity search against the incoming fact or the raw query;
if it misses a relevant record, the policy cannot recover, and RL
cannot improve this step. An agentic manager gathers its own context,
making this step part of the learned policy. During construction, it
can search for records that a new chunk duplicates or contradicts
before writing. During retrieval, it can reformulate the query and
follow intermediate results for multi-hop questions, as found useful
in multi-step QA (Trivedi et al., 2023; Jin et al., 2025). In line
with this, the trained retrieval policy improves the overall JJ score
by 4.50 and 4.04 points on the same memory banks
(Table 3).
A.1 Differences from Mem-T
Mem-T is the closest work to ours. Beyond the construction pipeline
discussed above, the main difference lies in how construction is
trained. Mem-T trains retrieval online with MoT-GRPO but learns
construction offline, imitating actions that are scored in hindsight
from retrieval rollouts. These scores come from an earlier policy and
memory state and can become outdated as the shared parameters change.
Moreover, they depend on whether the sampled retrieval surfaces the
stored memory, so a correctly stored fact can receive little credit
when retrieval fails. Uni-Mem trains both stages with
on-policy RL and scores construction with evidence-indexed memory
scoring (Eq. (5)), which does not depend on the
learned retriever. Consistent with this, longer alternation blocks,
which let one stage lag behind the other, lower the overall JJ score
from 88.80% to 83.75% (Figure 4(c)).
Appendix B Implementation Details
B.1 Memory Representation and Provenance Tracking
We use the storage and retrieval layer of Mem0 Chhikara et al. (2025)
(with Qdrant as the vector store) as the memory bank, rather than a
naive record list, since a capable manager should be able to operate
such a well-engineered store.
Crucially, we bypass Mem0’s native interface, in which a fixed LLM
pipeline extracts facts and chooses among ADD,
UPDATE, DELETE, and NOOP, and the question
is issued as a single search.
Instead, the learned policy decides what to store, when and how to
revise records, and when and how to search; Mem0 only executes these
decisions.
CREATE inserts agent-written content verbatim without Mem0’s
fact extraction, UPDATE replaces a record’s text and
embedding under the same ID, DELETE removes it, and
SEARCH returns the most similar records with their IDs and text.
The single source-chunk index cic_{i} in the main text is a
simplified notation for record provenance. In implementation,
each record maintains a cumulative list of chunk IDs:
CREATE initializes this list with the current chunk ID,
and UPDATE appends the current chunk ID while retaining
previous associations.
Index⁡(Ek,ℬ)\operatorname{Index}(E_{k},\mathcal{B}) selects the current
versions of all surviving records whose provenance lists
intersect EkE_{k}, including records updated rather than
originally created in an evidence chunk.
This metadata is hidden from the manager and used only for
evidence-indexed scoring (Eq. (5))
and credit assignment during training.
B.2 Tool Specifications and Execution Protocol
Tools – specify the manager’s
interface when using an external answer model.
Each card lists the inputs, outputs, and execution semantics.
Arguments are required unless marked optional.
⬇
memory_search(query, limit)
[Input]
query: string; semantic search query.
limit: integer, optional; requested result limit.
[Output]
Matching memory records with IDs and text.
The result set may be empty.
[Execution]
Search the current memory bank without modifying
it. The environment determines the effective
result limit from the event configuration.
⬇
create_memory(content)
[Input]
content: string; one self-contained memory.
[Output]
Created record: id, memory, and event="ADD".
[Execution]
Insert the supplied text as a new record.
Preserve names, dates, and quantities; store
each new fact in a separate record.
⬇
update_memory(memory_id, content)
[Input]
memory_id: string; target record ID.
content: string; replacement memory text.
[Output]
Update confirmation, or an error on failure.
[Execution]
Replace the record’s text and embedding while
preserving its ID. Use for an explicit correction
or supersession of the same stored fact.
Store new facts through CREATE.
⬇
delete_memory(memory_id)
[Input]
memory_id: string; target record ID.
[Output]
Deletion confirmation, or an error on failure.
[Execution]
Remove the specified record from the memory bank.
Use when the record is wrong and no replacement
is needed.
⬇
finish_event(summary)
[Input]
summary: string, optional; brief completion note.
[Output]
The next event’s observation, or termination
after the final event. A rejected call returns
an error observation.
[Execution]
Close the current event. For retrieval events,
hand the accumulated evidence to the external
answer model before advancing.
[Call example]
<tool_call>
{"name": "finish_event", "arguments": {}}
</tool_call>
Construction events.
Each event starts with a chunk of up to two new dialogue messages
and up to ten preceding turns from the same session as read-only
context.
The manager may interleave searches with memory writes.
A write requires a successful search response observed in an
earlier assistant turn of the same event; an empty result is valid.
A write bundled with the first search is therefore rejected.
FINISH closes the event and advances to the next input,
including when no memory changes are needed.
Retrieval events.
Each event starts with a question and a reference date, when available.
Only SEARCH and FINISH are permitted.
The manager may refine its queries over multiple rounds;
retrieved records are accumulated and deduplicated by ID.
A voluntary FINISH requires a successful search response
and hands the accumulated evidence to the frozen answer model.
The environment then clears the retrieval cache and advances
to the next question, or terminates after the final question.
Execution rules.
Each call is a JSON object containing name and
arguments, enclosed in <tool_call> tags.
Calls within a response execute sequentially, and their results
or errors become observations for the next decision.
For updates and deletions, displayed memory aliases are resolved
to backend record IDs.
FINISH must be issued alone: when bundled with other
calls, it is ignored while the remaining calls undergo
their usual checks.
In our RL configuration, each event permits at most eight
searches and 24 manager responses; exhausting the response
budget forces event closure.
The search result limit defaults to 10 for construction and
20 for retrieval. Smaller requested limits are raised to
these values, and all limits are capped at 200.
Appendix C Training Details
C.1 Supervised Cold-Start Training
Source data construction.
Following the data-generation methodology of LoCoMo Maharana et al. (2024),
we construct four additional multi-session conversation tasks as source data
for SFT. These tasks span short, medium, long, and extra-long histories,
ranging from 162 to 1,054 turns. As summarized in
Table 5, the corpus contains 117 sessions,
2,290 turns, and 773 question–answer pairs covering single-hop, multi-hop,
temporal, and open-domain questions.
QA category
Length
Task
Sessions
Turns
QA pairs
Single-hop
Multi-hop
Temporal
Open-domain
Short
conv_05
10
162
114
66
21
16
11
Medium
conv_07
20
371
128
68
23
25
12
Long
conv_12
35
703
116
57
13
34
12
Extra-long
conv_19
52
1,054
415
198
74
113
30
Total
4 tasks
117
2,290
773
389
131
188
65
Table 5: Statistics of the four source tasks for SFT demonstration generation.
A turn is one nonempty message; session dates and speaker metadata are excluded.
QA counts include all source questions, with no adversarial questions.
SFT demonstration generation.
To initialize the agent’s tool-use and memory-management capabilities,
we use GPT-5.4 as a teacher to generate interactive demonstrations on these
four source tasks. The teacher processes dialogue chunks to construct
memory and performs tool-based retrieval for the associated questions,
conditioning each decision on preceding tool observations. We record the
tool calls and returned observations as supervised dialogue sequences,
with teacher responses serving as training targets.
Optimization and checkpoint selection.
Figure 6: SFT training loss, measured as the mean negative
log-likelihood over supervised assistant tokens (nats/token).
The horizontal axis counts optimizer updates.
The light curve shows the logged loss; the dark curve shows
a trailing 100-step moving average.
We fine-tune Qwen3-4B-Instruct with a learning rate of 10−510^{-5},
a warmup ratio of 0.1, and a global batch size of 8
(one example per GPU on eight GPUs).
Training uses bfloat16 precision and a maximum sequence length of
16,384 tokens, with loss applied only to assistant tokens.
The SFT budget is set to three epochs, with checkpoint saving
and validation-loss evaluation every 431 optimizer updates.
Figure 6 shows the recorded training loss over
4,257 optimizer updates.
The mean loss decreases from 1.579 over the
first 100 updates to 0.213 over the last 100 updates.
For checkpoint selection, we evaluate downstream QA on the LoCoMo
validation partition from the 2:1:72{:}1{:}7 train/validation/test split
used in the main experiments. We first apply a 95% tool-format gate,
measuring the fraction of assistant turns with at least one parseable,
recognized tool call; this measures format compliance rather than
execution success.
Across three stochastic runs, pass@1 is the mean judge accuracy,
while pass@3 is the fraction of questions answered correctly in at
least one run. Their difference, gap, measures the additional
coverage available through sampling. F1 provides a complementary
token-overlap diagnostic. To assess policy diversity, we also measure
mean next-token entropy on held-out SFT prompts, using the partial
entropy sum over returned top-token probabilities.
Finally, finish-bundled counts turns that combine
finish_event with another action, which the environment rejects.
Table 6 reports the checkpoints
evaluated for downstream QA, from step 431 to step 3448
at intervals of 431 optimizer updates.
We select step 862 for subsequent RL, balancing task performance,
tool-use behavior, and policy diversity. Compared with step 431,
pass@1 improves from 81.93% to 83.35%, and finish-bundled turns
decrease from 182 to 54, while the entropy estimate remains at
0.4058 nats/token and the gap at 11.06 percentage points.
Although step 1724 achieves the highest pass@1 and F1,
step 862 retains higher entropy and a larger sampling gap.
Checkpoint
Tool format (%)
Pass@1 (%)
Pass@3 (%)
Gap (pp)
F1 (%)
Entropy
Finish-bundled
Base (pre-SFT)
99.32
79.57
90.45
10.89
24.80
0.1706
1,610
Step 431
99.60
81.93
94.94
13.01
26.52
0.4482
182
Step 862 (selected)
99.34
83.35
94.42
11.06
26.96
0.4058
54
Step 1293
99.70
83.59
94.09
10.50
26.98
0.3717
67
Step 1724
99.74
85.19
94.94
9.74
27.66
0.3776
21
Step 2155
99.81
85.06
94.55
9.48
27.24
0.3632
21
Step 2586
99.77
84.72
94.94
10.22
27.32
0.3159
53
Step 3017
99.84
84.37
94.55
10.17
27.46
0.3239
46
Step 3448
99.69
83.57
95.32
11.75
27.14
0.3424
29
Table 6: SFT checkpoint selection for RL initialization.
Bold marks the selected checkpoint, not per-column maxima.
All checkpoints pass the 95% tool-format gate.
Pass@1, pass@3, and F1 use judged, non-adversarial QA records;
gap is pass@3 minus pass@1 in percentage points (pp), computed
before rounding. Entropy is a partial-sum estimate in nats/token
on held-out SFT prompts. Tool format and finish-bundled are
measured from the first evaluation run; finish-bundled is a raw count.
C.2 Event-Wise Credit Computation
We detail the update-robust evidence attribution in
Section 4.1. During construction rollouts,
we log each record’s UPDATE and DELETE
operations together with their triggering events.
For query qkq_{k}, we identify records created or updated
in the original evidence chunks EkE_{k} and follow their
subsequent modifications. Their surviving current versions
form Index⁡(Ek,ℬ(n))\operatorname{Index}(E_{k},\mathcal{B}^{(n)}).
The rollout-specific expanded credit set,
ℰ~kcon,(n)\widetilde{\mathcal{E}}_{k}^{\mathrm{con},(n)}, contains the
original evidence events and subsequent update or deletion
events on these records, and is used in
Eq. (6).
This expansion affects only credit recipients: memory
selection always uses the original EkE_{k}, without including
unrelated records written by the added events.
For example, if event 5 creates an evidence record,
event 12 updates it, and event 20 deletes it, all three
events share the score computed from the remaining indexed
memories. Deletion alone does not imply
𝒮k​(ℬ(n))=0\mathcal{S}_{k}(\mathcal{B}^{(n)})=0.
The modification logs remain available after deletion,
and lineage-based credit assignment requires no additional
answer-model calls.
C.3 RL Training Hyperparameters and Computational Resources
We optimize the SFT-initialized Qwen3-4B-Instruct manager with
Stage-Decoupled GRPO, alternating construction and retrieval
iterations at a 1:11{:}1 ratio.
During RL training, Qwen3.8-27B serves as both the answer model and
the judge, using the Mem0 judge prompt, while Qwen3-Embedding-4B
provides memory embeddings. These service models remain frozen.
Table 7 summarizes the main settings.
We use two Ascend 910B nodes: one runs RL on eight NPUs with
verl 0.8, and the other hosts the answer, judge, embedding,
and memory-storage services.
Answer and judge models during training.
RL training requires a large number of answer-model and judge calls:
every construction rollout is scored on all training queries through
evidence-indexed scoring, and every retrieval rollout is answered and
judged. Due to computational and API cost constraints, we use a
locally deployed Qwen3.8-27B as both the frozen answer model gϕg_{\phi}
and the judge 𝒥ψ\mathcal{J}_{\psi} during training. For evaluation, we
use GPT-OSS-120B as the answer model and GPT-4o-mini as the judge. Since the manager is trained with one
answer model and evaluated with another, the reported gains also
suggest that the learned memory behaviors transfer across answer
models.
Hyperparameter
Value
Actor learning rate
2×10−62\times 10^{-6}
Task batch size
2
Rollouts per task (NN)
8
PPO mini-batch size
1
Sampling temperature / top-pp
0.7 / 1.0
Training epochs
200
Advantage normalization
Group mean/std
KL / entropy coefficients
0 / 0
Model precision
bfloat16
Validation / checkpoint interval
10 / 10 steps
Table 7: Main RL training hyperparameters. Task batch size is
specified before sampling the eight rollouts per task.
Appendix D Details of Baselines.
We describe the baseline methods below. Training-free methods manage memory without task-specific parameter updates, whereas training-based methods learn memory-related policies from task feedback.
RAG.
RAG indexes conversation history and retrieves relevant text segments through embedding similarity. The retrieved segments are provided to an answer model as supporting context. It provides a basic conversation-retrieval baseline without explicit fact extraction or memory editing.
A-Mem.
A-Mem (Xu et al., 2025) organizes interactions into structured notes containing the original content, timestamps, keywords, tags, and contextual descriptions. It links semantically related notes and revises existing notes when new information arrives. At query time, relevant notes are retrieved through embedding similarity to support answer generation.
Mem0.
Mem0 (Chhikara et al., 2025) maintains an external memory bank through an extraction-and-update pipeline. An LLM extracts salient facts from incoming interactions, retrieves related memories, and selects among ADD, UPDATE, DELETE, and NOOP to maintain the bank. Relevant memories are subsequently retrieved to answer questions.
MemoryOS.
MemoryOS (Kang et al., 2025) organizes information into short-term, mid-term, and long-term personal memory. Dialogue records are grouped into topic-based segments, with dialogue-chain FIFO updates and heat-based consolidation transferring information across levels. Retrieval combines relevant dialogue segments and pages with recent context and persistent user and agent information for response generation.
MEM1.
MEM1 (Zhou et al., 2026) integrates memory consolidation into an agent’s reasoning process. At each interaction step, it combines the previous internal state with new observations to generate an updated state and the next action. Earlier interaction content is removed from the active context, and RL trains the agent to preserve task-relevant information within this bounded context.
MemAgent.
MemAgent (Yu et al., 2026a) processes long inputs sequentially while maintaining a bounded textual memory. Each new chunk triggers a rewrite of the memory, and the final answer is generated from the question and the resulting memory. Its multi-conversation RL algorithm uses final-answer rewards to optimize the sequence of memory updates and answer generation.
Memory-R1.
Memory-R1 (Yan et al., 2026b) contains a memory manager and a separate answer agent. The manager learns to select ADD, UPDATE, DELETE, or NOOP for extracted information. Fixed similarity-based retrieval supplies candidate memories to the answer agent, which learns to filter and reason over them through memory distillation. Both components are optimized using downstream answer correctness.
Memory-R2.
Memory-R2 (Yan et al., 2026a) trains a shared-parameter fact extractor and memory manager over chunked, multi-session conversations. Its LoGo-GRPO algorithm combines global trajectory supervision with local session rerollouts from shared intermediate memory states to improve credit assignment. A curriculum progressively increases the session horizon, while similarity-based retrieval remains fixed. The standard variant uses a separately trained Qwen2.5-7B-Instruct answer agent. We additionally evaluate a variant that pairs the Memory-R2 memory policy with GPT-OSS-120B (OpenAI, 2025) as the answer model. This variant matches the answer model used by Uni-Mem, enabling a comparison of the memory systems with the same downstream answer model.
Mem-T.
Mem-T (Yue et al., 2026) maintains working, factual, experiential, and raw memories. Memory construction follows a formation-and-evolution pipeline, while retrieval uses multiple search steps to gather evidence. Its MoT-GRPO algorithm trains retrieval through branching rollout trees and propagates answer feedback to intermediate operations. Construction is trained through offline imitation of actions selected by hindsight scores derived from retrieval outcomes and evidence alignment.
Appendix E Experimental Setup and Evaluation Protocol
E.1 Memory-R2 Re-evaluation and Judge-Prompt Sensitivity
As shown in Table 8,
our re-evaluated Memory-R2 results are lower than those
reported in the original work (Yan et al., 2026a).
These results are obtained by directly evaluating the
officially released checkpoints, without additional
training.
The discrepancy stems from a difference in the
LLM-as-a-Judge prompt. Our evaluation adopts a prompt
aligned with Mem0 (Chhikara et al., 2025), which differs
from the prompt used in the original Memory-R2 evaluation
and consequently changes how generated answers are judged.
To maintain consistent judging conditions, we apply this
prompt to both Uni-Mem and the re-evaluated baselines.
The complete template is provided in
Appendix G.2.
Setting
Multi
Open
Single
Temp.
Overall
Memory-R2
Reported
80.93
67.53
86.80
69.90
80.99
Re-evaluated
69.15
61.29
78.61
57.38
71.26
Memory-R2 (OSS)
Reported
85.07
74.14
90.58
83.33
87.10
Re-evaluated
74.13
70.97
81.15
74.68
77.87
Table 8: Reported and re-evaluated Memory-R2 results
on LoCoMo (JJ, %). Reported results are
from Yan et al. (2026a). Re-evaluated results
are obtained from the officially released checkpoints
using our Mem0-aligned judge prompt.
Comparability of reported baseline results.
Except for Memory-R2, Memory-R2 (OSS), and Mem-T, which we re-evaluate
from officially released checkpoints, the baseline results in
Table 1 are taken from
Yan et al. (2026a). They were obtained under the original
evaluation protocol of that work, which differs from ours in the judge
prompt and, for some methods, the answer model.
As Table 8 shows, the original judge
prompt yields higher scores than our Mem0-aligned prompt, by 9.73 and
9.23 overall points for Memory-R2 and Memory-R2 (OSS), respectively.
These reported results should therefore be regarded as reference
points rather than controlled comparisons. Our conclusions rely only
on the re-evaluated baselines, which share the same test split, judge
prompt, and judge model with Uni-Mem.
E.2 Single-Stage GRPO and Ablation Configurations
M1M_{1} and M2M_{2} sample continuous trajectories in which memory construction
is followed by retrieval. Each rollout constructs its own memory bank
ℬ(n)\mathcal{B}^{(n)} and retrieves evidence from that bank.
Both event types contribute to the same policy update.
M1M_{1}: Standard GRPO.
QA outcome rewards are aggregated into a trajectory-level return.
Standard GRPO normalizes these returns across sibling rollouts
and broadcasts the resulting advantage to every manager decision
step, covering both construction and retrieval.
M2M_{2}: Single-stage GRPO with ECA.
Evidence-to-event localization
(Eq. (4)) first identifies
the construction events ℰkcon\mathcal{E}_{k}^{\mathrm{con}} associated
with each query.
For an incorrect answer, we re-answer the query from evidence-indexed
memories and obtain
sk(n)=𝒮k​(ℬ(n))s_{k}^{(n)}=\mathcal{S}_{k}(\mathcal{B}^{(n)})
using Eq. (5).
Let ck(n)c_{k}^{(n)} denote the credit assigned to each localized
construction event and rk(n)r_{k}^{(n)} the retrieval-event credit:
(ck(n),rk(n))={(1,1),Rk(n)=1,(0,⊥),Rk(n)=0,sk(n)=0,(1,0),Rk(n)=0,sk(n)=1.(c_{k}^{(n)},r_{k}^{(n)})=\begin{cases}(1,1),&R_{k}^{(n)}=1,\\
(0,\bot),&R_{k}^{(n)}=0,\ s_{k}^{(n)}=0,\\
(1,0),&R_{k}^{(n)}=0,\ s_{k}^{(n)}=1.\end{cases}
(11)
Thus, a failed evidence-indexed re-answer attributes the failure
to construction and excludes the retrieval event.
A successful re-answer credits construction and assigns zero
reward to the failed retrieval.
Here, ⊥\bot excludes an event from both group statistics and
policy optimization; it is distinct from a zero reward.
Each construction event sums the credits of its related queries;
events without related queries are excluded.
The retained returns are normalized across corresponding events
in the rollout group using Eq. (8).
Each event’s advantage is then broadcast to all manager decision
steps within that event.
Appendix F Supplementary Experiments
F.1 Gains from the Harness versus the RL Algorithm
Setting
Overall JJ
Δ\Delta vs. M0M_{0}
Re-evaluated baselines
Memory-R2 (OSS)
77.87
–
Mem-T
76.49
–
Our harness, off-the-shelf manager
Qwen3.5-27B (strongest)
87.05
–
Qwen3.5-397B-A17B (largest)
85.86
–
Our harness, Qwen3-4B-Instruct w/o SFT
Start point (M0M_{0})
76.58
–
+ standard GRPO (M1M_{1})
76.95
+0.37
+ our RL (M4M_{4})
80.99
+4.41
Our harness, Qwen3-4B-Instruct w/ SFT
Start point (M0M_{0})
83.56
–
+ standard GRPO (M1M_{1})
84.02
+0.46
+ our RL (M4M_{4})
88.80
+5.24
Table 9: Overall JJ (%) on LoCoMo under the same harness.
Δ\Delta is measured against the start point M0M_{0} of each block.
Our RL combines ECA, Stage-Decoupled GRPO, and update-robust
attribution; see Table 2 for per-component results
and Table 10 for all off-the-shelf managers.
Table 9 separates the contributions of the
agentic harness, the initialization, and the RL algorithm. First, the
harness alone does not explain the advantage over prior methods: the
untrained manager performs on par with the re-evaluated baselines.
Second, the improvement stems from our RL algorithm rather than from
RL per se: under both initializations, standard GRPO in the same
harness yields only marginal gains, whereas our RL adds 4.41 and 5.24
points, respectively. Without any SFT, our RL alone already surpasses
all re-evaluated baselines. Third, SFT and our RL are complementary:
SFT provides a stronger starting point, and our RL still adds 5.24
points on top of it despite the smaller remaining headroom. Finally,
no off-the-shelf manager in the same harness, including the strongest
(Qwen3.5-27B) and the largest (Qwen3.5-397B-A17B, a MoE model with
17B active parameters), reaches the trained 4B manager.
F.2 Training Curves
Figure 7: Training-time LLM-as-a-Judge scores (JJ, %) of
Uni-Mem, with Qwen3.8-27B used for both answer generation
and judging. The light curve shows the logged scores;
the dark curve shows a trailing 20-step moving average.
Figure 7 reports the training-time
LLM-as-a-Judge scores (JJ) of Uni-Mem over 200 logged
training steps. In this run, Qwen3.8-27B serves as both
the frozen answer model and the fixed judge model, while
the Qwen3-4B-Instruct memory manager is optimized with
Stage-Decoupled GRPO. The mean JJ increases from
75.76%75.76\% over the first 20 steps to 81.41%81.41\% over the
final 20 steps, an improvement of 5.655.65 percentage points.
Despite fluctuations in individual scores, the moving
average exhibits an overall upward trend, with smaller
changes in the later part of training.
F.3 Detailed Results on Training and Manager Scale
Table 10 provides the category-level
results corresponding to
Figure 4(a).
We evaluate Qwen3-4B and four larger Qwen3.5 models as
off-the-shelf memory managers within the same agentic
framework.
Uni-Mem uses Qwen3-4B initialized through supervised
fine-tuning and subsequently optimized with Stage-Decoupled
GRPO.
All settings follow the evaluation protocol described
in the main text.
Manager
Multi
Open
Single
Temp.
Overall
Off-the-shelf managers
Qwen3-4B
78.61
70.97
84.04
57.81
76.58
Qwen3.5-27B
85.07
74.19
89.13
86.92
87.05
Qwen3.5-35B-A3B
82.09
74.19
89.30
85.23
86.23
Qwen3.5-122B-A10B
84.08
67.74
88.46
85.65
85.86
Qwen3.5-397B-A17B
84.08
75.81
86.93
87.34
85.86
Trained manager
Uni-Mem (Qwen3-4B)
87.06
79.03
91.51
86.08
88.80
Table 10: Detailed results on training and manager scale
on LoCoMo (JJ, %). Multi, Open, Single, and Temp.
denote multi-hop, open-domain, single-hop, and temporal
questions, respectively. Best scores in each column
are bold.
F.4 Detailed Analysis of Alternating Training Schedules
Table 11 provides the
category-level results corresponding to
Figure 4(b,c).
Each schedule C:RC{:}R performs CC consecutive construction
updates followed by RR consecutive retrieval updates
under Stage-Decoupled GRPO.
The first block varies the allocation of updates between
the two stages.
The second block varies the number of consecutive updates
per stage while keeping construction and retrieval updates
equally allocated.
Schedule
Multi
Open
Single
Temp.
Overall
Construction–retrieval update ratio
1:81{:}8
81.09
66.13
89.47
79.75
84.48
1:41{:}4
86.07
64.52
90.83
81.01
86.32
1:21{:}2
87.56
70.97
91.85
83.97
88.15
1:11{:}1
87.06
79.03
91.51
86.08
88.80
2:12{:}1
89.55
74.19
89.81
82.28
87.24
4:14{:}1
83.58
69.35
91.17
80.59
86.23
8:18{:}1
84.58
72.58
90.15
80.17
85.95
Update block length with equal allocation
1:11{:}1
87.06
79.03
91.51
86.08
88.80
2:22{:}2
88.06
75.81
90.66
84.81
88.06
4:44{:}4
88.56
82.26
90.15
84.39
88.15
8:88{:}8
83.08
66.13
89.13
83.97
85.58
16:1616{:}16
81.59
61.29
88.12
82.28
84.11
32:3232{:}32
80.10
62.90
88.62
80.17
83.75
Table 11: Detailed results for alternating training
schedules on LoCoMo (JJ, %).
C:RC{:}R specifies consecutive construction and retrieval
updates per cycle. Best scores in each column within
each block are bold.
F.5 Retrieval Budget and Token Efficiency
To examine the accuracy-cost trade-off of iterative retrieval,
we vary the maximum retrieval steps kk and report JJ and token
consumption. Uni-Mem serves as the memory manager, with
Qwen3.8-27B used for both answer generation and judging.
As shown in Figure 8, increasing kk
from 1 to 3 raises JJ from 78.06%78.06\% to 83.67%83.67\%, highlighting
the benefit of multi-step retrieval guided by intermediate
evidence. Increasing kk from 3 to 8 yields a
further 2.002.00-point gain, reaching 85.67%85.67\%, but raises token
consumption from 10,889 to 58,642 (5.39×5.39\times), indicating
diminishing returns from larger retrieval budgets.
Figure 8: Accuracy (JJ) versus token consumption under different
maximum retrieval steps kk. Highlighted points and annotations
compare k=3k=3 and k=8k=8; lines connect the measured settings.
Appendix G Prompt Templates
G.1 Memory Manager Prompt
The same system prompt governs memory construction and query-time retrieval
(Prompt ). The manager maintains long-term memory
and retrieves evidence for an external answer model through a shared tool
interface.
⬇
# Role
You are an autonomous memory manager controlling a long-term memory store for
an external assistant. You see one external event at a time. The event is either
new conversation content to ingest or a user query to answer. You cannot know
how many future events remain, and you must never try to end the whole episode.
# Operating Principles
Maintain an accurate, useful memory store.
- Store durable user-specific facts, preferences, relationships, plans,
commitments, recurring projects, explicit instructions, and dated events.
- Update a memory only when the current messages explicitly correct or
supersede the same stored event, preference, or relationship, making its
claim wrong or stale – never just to add more detail. A different event, a
different date, a compatible preference or relationship, or an unrelated
topic is a new memory; create it.
- Delete memories only when they are wrong and no replacement is needed.
- Use the shortest useful action sequence that satisfies the current event.
# Memory Writing Standard
The memories you store are later semantically searched and handed to a separate
answerer that uses them to answer future questions. Write each memory so that
the right one is easy to retrieve by a natural-language question and complete
enough to answer from on its own.
- One focused unit per memory: a single event, preference, decision, or
relationship. Capture that one unit with its own essential specifics
(names, date).
- One occasion or topic per memory. Different events, different dates, or
unrelated topics each get their own memory; never merge them into one. A memory
that spans several dates or several unrelated events is too broad: it is hard to
retrieve, and its single timestamp is wrong for most of what it contains.
- Concise but complete: a focused memory is usually 1-2 sentences (about 15-80
words; up to 3 sentences / 100 words only when it is dense with proper nouns,
dates, or quantities). The range is a guide, not a hard cap, and completeness
always wins over word count – never drop a proper noun, date, or number to stay
short. When a topic carries many details, split it into several focused
memories rather than packing one wide memory; when unsure, prefer two memories
over one merged.
- Self-contained: understandable without other memories. Start with the
speaker’s actual name and replace pronouns with names.
- Preserve specifics verbatim: proper nouns and titles (people, places, books,
films, brands, dishes), exact quantities, and qualifiers. Never generalize a
concrete detail ("Ferrari 488 GTB", not "a sports car") – the exact wording is
what a future question is scored against.
- Anchor time to the Observation Date as absolute dates; never store
"last week"/"recently", and never blur an explicit number or duration.
- Resolve a relative day-of-week reference to its absolute date before storing.
If on 11 May, 2022 the conversation mentions something that happened "on
Monday", infer the actual date (9 May, 2022) and write the memory as "On 9
May, 2022, <what happened>" – never store the bare weekday.
# Memory Write Gate
create_memory, update_memory, and delete_memory require previously observed
retrieval evidence for the related person, entity, date, or topic.
- Before any write tool call, you must have already observed relevant results
from memory_search in an earlier assistant response during the
current event.
- A retrieval result from the same assistant response is not yet observed. Do
not call memory_search and then write in that same response.
- If you have not observed relevant retrieval evidence, do not write. Either
retrieve first or finish the event if there is no durable memory work to do.
- Search queries should use the relevant names, entities, dates, and topic
keywords needed to check for duplicates, stale memories, or contradictions.
- Create a new memory for each new fact, event, preference, feeling,
reflection, question asked, or decision.
- Call update_memory only when the current messages explicitly correct or
supersede the same retrieved event, preference, or relationship, making its
stored claim wrong or stale and requiring in-place correction.
- Capture every fact the conversation states; do not skip one because it seems
minor.
# Query Handling
For query events, answer from long-term memory evidence when the question
depends on remembered user-specific information.
- Retrieve and observe enough relevant memories before answering a memory-based
query.
- If the available memories do not support an answer, do not invent one.
- You do not answer the question yourself. Retrieve useful evidence, then
finish the event so a separate external answerer can respond from it.
# Tool Contract
You may use memory_search, create_memory, update_memory,
delete_memory, and finish_event when they are
available in the tool schema.
finish_event must be the only tool call in its assistant response. Do not bundle
finish_event with memory_search, create_memory, update_memory, or
delete_memory.
# Output Format
Wrap every tool call in:
<tool_call>
{"name": "<tool>", "arguments": {<json>}}
</tool_call>
G.2 LLM-as-a-Judge (JJ) Prompt
Prompt  gives the mem0_base evaluation
template, supplied as a user message. Given a question, a reference answer,
and a generated answer, the judge returns CORRECT or WRONG.
We report JJ as the percentage of evaluated answers labeled
CORRECT. Braced fields denote runtime inputs.
⬇
Your task is to label an answer to a question as "CORRECT" or "WRONG". You will be given the following data: (1) a question (posed by one user to another user), (2) a ’gold’ (ground truth) answer, (3) a generated answer which you will score as CORRECT/WRONG.
The point of the question is to ask about something one user should know about the other user based on their prior conversations. The gold answer will usually be a concise and short answer that includes the referenced topic, for example:
Question: Do you remember what I got the last time I went to Hawaii?
Gold answer: A shell necklace
The generated answer might be much longer, but you should be generous with your grading - as long as it touches on the same topic as the gold answer, it should be counted as CORRECT.
For time related questions, the gold answer will be a specific date, month, year, etc. The generated answer might be much longer or use relative time references (like ’last Tuesday’ or ’next month’), but you should be generous with your grading - as long as it refers to the same date or time period as the gold answer, it should be counted as CORRECT. Even if the format differs (e.g., ’May 7th’ vs ’7 May’), consider it CORRECT if it’s the same date.
Now it’s time for the real question:
Question: {question}
Gold answer: {answer}
Generated answer: {response}
First, provide a short (one sentence) explanation of your reasoning, then finish with CORRECT or WRONG. Do NOT include both CORRECT and WRONG in your response, or it will break the evaluation script.
Just return the label CORRECT or WRONG in a json format with the key as "label".
G.3 Answer Prompt
Prompt  shows the default prompt used by the frozen
external answer model in run_eval. The system message specifies
how to use memory evidence, and the user message supplies the question and
retrieved memories. Braced fields denote runtime inputs; the memory block
lists records chronologically with date prefixes. Message-role headings
are shown for clarity and are not part of the prompt text.
⬇
[System message]
You are answering a question using retrieved memories written by an agent that observed past conversations. Follow these rules internally:
Read every memory, verify the entity, combine related facts, prefer specific evidence, handle dates relative to {reference_date}, and answer directly.
Never invent specific names, titles, places, or dates that do not appear in the memories. If no memory is remotely related, use the configured fallback.
Output only:
ANSWER: <final answer>
[User message]
Question:
{question}
Retrieved memories:
{memory_block}
ANSWER:
Experimental support, please
view the build logs
for errors. Generated by
L
A
T
E
xml
.
Instructions for reporting errors
We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile
support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the
methods listed below:
Click the "Report Issue" (
) button, located in the page header.
Tip: You can select the relevant text first, to include it in your report.
Our team has already identified the following issues. We appreciate your time reviewing and reporting rendering errors we
may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability
should not be a barrier to accessing research. Thank you for your continued support in championing open access for
all.
Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion, and welcome developer contributions.
We gratefully acknowledge support from our major funders,
member institutions, ,
and all contributors.
About ·
Help ·
Contact ·
Subscribe ·
Copyright ·
Privacy ·
Accessibility ·
Operational Status (opens in new tab)