Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory

Abstract

Multimodal models and agents have emerged to enable AI systems to learn from and interact with the real world more like humans by integrating information across multiple modalities. However, enabling multimodal agents to continually acquire useful information from complex, heterogeneous multimodal data remains a significant challenge as they move toward long-horizon tasks and lifelong settings. Addressing this challenge requires effective memory mechanisms that can maintain continuity across sessions, adapt to changing users and environments, and refine memory-management policies through accumulated experience. We introduce a two-axis analytical framework for multimodal agent memory structured around two key questions: how multimodal information is formed and represented (Axis I, Formation and Representation), and how its management adapts with accumulated experience (Axis II, Memory Management). Axis I, Formation and Representation, characterizes what information is retained and how it is encoded (e.g., modality-specific embeddings, textual and structured records, latent and parametric representations) to support subsequent reasoning and action. Axis II, Memory Management, distinguishes three tiers: T1 Static Memory reuses fixed memory; T2 Dynamic Memory updates memory through a stable policy; and T3 Self-Evolving Memory updates memory through an evolving policy. Based on this framework, we synthesize recurring design patterns and derive principles for balancing source fidelity with abstraction, maintaining temporal validity and provenance, grounding memory use in recoverable evidence, and evolving memory policies over time. Furthermore, we discuss the challenges and potential future directions for the memory mechanism of multimodal agents.

1 Introduction

Large foundation models (OpenAI, 2026; Anthropic, 2026; DeepSeek-AI et al., 2026; Google DeepMind, 2026), such as LLMs and MLLMs, have become the reasoning and perception backbones of AI agents that interpret goals, plan, use tools, and act through interaction with users and environments. An agent places the model within a continuing loop of observation, reasoning, and action (Xi et al., 2025; Wei et al., 2026; Plaat et al., 2025; Chen, Celia, 2026), increasingly spanning multimodal digital and physical environments. As this interaction unfolds over time, later decisions often depend on information and experience accumulated beyond the immediately available context. Memory provides the persistent state that bridges these interactions, carrying relevant information from prior observations and experience into subsequent reasoning and action (Hu et al., 2025b; Zhang et al., 2025e; Huang et al., 2026b).

Memory is therefore a core component of AI agents (Xu et al., 2025; Hu et al., 2025b). At the most basic level, it enables continuity by preserving task state and relevant experience beyond the current interaction, allowing agents to operate coherently over long horizons (Packer et al., 2023) and across repeated interactions. Beyond continuity, persistent memory allows agents to accumulate information over time, supporting personalization through user-specific experience (Hong et al., 2026) and maintaining consistent behavior as users, tasks, and environments evolve (Zhong et al., 2024). By retaining task outcomes, feedback (Shinn et al., 2023), failure traces (Tu et al., 2026), and successful strategies (Fan et al., 2026b), memory enables agents to refine future decisions and accumulate reusable knowledge (Li et al., 2024) and skills (Wang et al., 2024a).

The growing demand for agents that can interpret visual information (Nguyen et al., 2026b; Yu et al., 2025c), process temporally structured video and speech (Lin et al., 2026d; Someki et al., 2026), reason about and navigate 3D spaces (Gu et al., 2024), and interact with graphical and physical environments (Agashe et al., 2025; Liu et al., 2025b) has made multimodal capabilities increasingly essential. To operate in these settings, an agent must integrate heterogeneous, high-dimensional observations, capture their temporal and spatial structure, and produce appropriate actions or multimodal outputs (Wang et al., 2024b). Unlike textual-only inputs, these multimodal observations cannot always be compactly or losslessly reduced to a single representation for later use. This shift does not merely increase the volume of experience an agent may encounter; it changes the nature of what must be remembered and organized. Consequently, multimodal memory must determine both what aspects of experience should persist and how those retained memories should remain useful as new observations accumulate and the underlying world evolves.

To investigate these two requirements, we characterize multimodal agent memory through a lifecycle with two coupled stages: formation determines what experience is retained and in what representation, while management governs how the resulting memory persists, changes, is accessed, and guides later reasoning and action. These stages are tightly coupled: decisions made during formation determine what evidence can later be recovered, revised, or verified, while subsequent use and new observations may require memories to be updated, reconciled, consolidated, or forgotten. Multimodality makes both stages substantially more challenging.

The first challenge arises during formation and representation: A multimodal agent must decide what experience to retain, how to divide continuous or heterogeneous observations into memory units, and in what form those memories should persist. Unlike text-centered interactions, which can be retained in a compact textual form that is directly consumable and editable (Packer et al., 2023; Zhong et al., 2024; Chhikara et al., 2025) by LLMs, multimodal experience may contain perceptual, temporal, spatial, and acoustic information that is expensive to preserve exhaustively and difficult to compress without loss. A memory unit may correspond to an image (Nguyen et al., 2026b), image region (Wang et al., 2026j), video frame or clip (Song et al., 2024), audiovisual event (Lin et al., 2026d), object track (Fan et al., 2024), interface state (Wang and Chen, 2025), action–observation transition (Zhang et al., 2026d), trajectory (Xu et al., 2026), or complete episode (Zhang et al., 2025c). Each unit may be stored as source media (Yu et al., 2025c), encoded as modality-specific features (He et al., 2024) or geometric state (Gu et al., 2024), abstracted into textual (Feng et al., 2026) or structured records (Long et al., 2026), compressed into a latent state (Qian et al., 2024), or internalized in model parameters (Suri et al., 2026). Each choice preserves some aspects of experience while discarding or obscuring others, creating trade-offs among fidelity, efficiency, editability, source recoverability, and compatibility with downstream reasoning and action.

The second challenge arises during memory management: the memory state formed from prior experience must remain valid as new observations arrive, tasks progress, and environments change. The central question of multimodal memory management is not only how the memory system responds as its environment changes and experience accumulates, but also how to reconcile heterogeneous and partial observations into a consistent, evolving state. New evidence may extend an existing record (Long et al., 2026), correct an earlier belief (Ma et al., 2026), invalidate stale state (Lei et al., 2025), connect previously separate events (Yan et al., 2026), or provide feedback about the effectiveness of prior memory decisions (Bärmann et al., 2026). Multimodal memory management is particularly challenging because evidence is often partial and viewpoint-dependent, while the same entity or event may be represented across source media, modality-specific features, textual summaries, and structured world states (Fan et al., 2024). Effective management must reconcile new evidence across these linked representations (Feng et al., 2026), preserve connections to the supporting observations (Lin et al., 2026d), and prevent stale or conflicting state from guiding later actions (Yeo et al., 2026b). Memory systems differ in where persistent adaptation occurs. Their responses may range from reusing formed memory without persistent change (Mu et al., 2026), to updating memory content or organization under a stable policy (Mondal et al., 2026), to using feedback to improve and evolve the policy governing future memory writing (Zou et al., 2026a), maintenance (Bärmann et al., 2026), and retrieval (Fan et al., 2026b).

Motivated by these lifecycle challenges, we present a unified analytical framework for multimodal agent memory spanning images and documents, video and audiovisual streams, audio and speech, GUI interaction, and embodied or 3D environments. Our framework organizes the field along two analytically distinct but operationally coupled axes, illustrated in Figure 1. Axis I, Formation and Representation, characterizes what experience is retained, how it is organized into memory units, and the representation ultimately supplied to downstream reasoning or action. We distinguish source- and modality-specific representations, textual and structured representations, and latent and parametric representations, while separately examining hybrid architectures that combine multiple paths and often link compact abstractions to recoverable source evidence. Axis II, Memory Management, characterizes where persistent change occurs as experience accumulates: T1 Static Memory systems reuse formed memory without persistent modification; T2 Dynamic Memory systems update the memory state through a stable management policy; and T3 Self-Evolving Memory systems use feedback to update the policy governing future memory writing, maintenance, or retrieval. Together, these axes capture both what multimodal evidence survives into future computation and how the memory system evolves through continued experience.

Beyond providing a unified taxonomy, this study offers three broader contributions. First, we provide comprehensive coverage of multimodal agent memory across modalities, tasks, architectures, and evaluation settings, including a corpus of 297 memory architectures and 82 benchmark records in the current version of our taxonomy. Second, by jointly analyzing memory representation and management, we identify cross-cutting patterns that are difficult to observe from either dimension alone, including the prevalence of dynamic memory updates, the widespread use of hybrid representations, and the relative scarcity of deployment-time memory-policy evolution. Third, we synthesize the limitations of current systems and derive future directions around recoverable and provenance-aware memory, temporal validity and cross-representation consistency, self-improving memory management, bounded lifelong memory, governed forgetting, and lifecycle-aware evaluation.

This study focuses on memory mechanisms in multimodal agents. We define memory as experience-derived state that persists beyond the step in which it is formed and later affects reasoning, generation, planning, or action. At least one non-text modality must contribute to its formation or use; transient activations, one-pass reasoning traces, and incidental prompt context are excluded unless explicitly persisted and reused. Existing studies of agent memory classify memory by source (Zhang et al., 2025e), representation (Hu et al., 2025b; Zhang et al., 2025e; Du et al., 2025), function (Hu et al., 2025b; Wu et al., 2025c; Du et al., 2025), temporal scope (Wu et al., 2025c; Du et al., 2025), and operation (Zhang et al., 2025e; Du et al., 2025), while others trace its evolution from storage to reusable experience (Hu et al., 2025b; Huang et al., 2026b; Luo et al., 2026a). Multimodal extensions cover LLM or MLLM memory types (Jia et al., 2025), VLM taxonomies (Xia et al., 2026a), and multimodal-agent architectures and storage (Xie et al., 2025; Mokaria et al., 2026), benchmarks (Xie et al., 2025), and applications (Xie et al., 2025; Mokaria et al., 2026). In contrast, we shift the unit of analysis from memory components to the operational lifecycle: how multimodal experience becomes persistent memory, how representation constrains subsequent retrieval, revision, and verification, and where adaptation occurs as experience accumulates.

The remainder of this paper is organized as follows. Section 2 introduces the background, formal scope, and multimodal memory lifecycle. Section 3 studies memory formation and representation, from source-preserving and modality-native memory to explicit textual and structured representations, latent and parametric memory, and hybrid source-linked designs. Section 4 examines memory management across T1 Static Memory, T2 Dynamic Memory, and T3 Self-Evolving Memory. Section 5 analyzes how representation choices constrain memory management and synthesizes corpus-level patterns across the two axes. Section 6 examines the systems mechanisms that realize these logical memory designs under resource and coordination constraints. Section 7 examines evaluation benchmarks and applications, while Section 8 discusses open challenges and future directions. Section 9 concludes the paper.

2 Background

Multimodal agents extend foundation vision-language models from one-shot prediction to an interaction loop in which the system observes an environment, reasons about a goal, selects an action, and interprets the resulting state (Xie et al., 2025; Mokaria et al., 2026). Their observations may include text, images (Deng et al., 2026), video (Rege et al., 2026), audio (Jiang et al., 2026b), graphical interfaces (Choi et al., 2026), robot poses (Yan et al., 2025b), or other sensor streams (Lin et al., 2026c). Because these inputs accumulate across steps and often describe a changing world, the current context alone is rarely sufficient for long-horizon reasoning. Memory provides continuity by retaining selected information from earlier interaction and making it available to later decisions. Figures 2 and 3 summarize representative multimodal agent memory methods across modalities, publication years, memory-management tiers, and their growth over time.

This definition includes external evidence stores (Sun et al., 2026c), image and video banks (Fan et al., 2024), maps (Li et al., 2026i), scene graphs (Gu et al., 2024), bounded visual buffers (Wen et al., 2026), recurrent states explicitly designed as memory (Wang et al., 2025b), experience libraries (Wang et al., 2026g), executable skills (Wang et al., 2024a), and parameters generated or adapted for subsequent reuse (Suri et al., 2026). It excludes ordinary feed-forward activations, one-pass chains of thought, and incidental prompt tokens when retained state and later reuse are not part of the paper’s memory contribution.

This distinction separates memory from context. Context is the information currently exposed to a model, whereas memory is the mechanism that determines what information can survive, how it is organized, and when it can re-enter the context or policy. A retrieved image can therefore be both a stored memory item and part of the current context, while an image seen once and immediately discarded is only an observation. The distinction is especially important in multimodal systems, where storing every perceptual input is computationally impractical and often unnecessary.

In this study, we use source material to denote the external stream, document, interaction, or environment from which an agent obtains information. For each architecture, an observation is a bounded input item made available to its memory-formation process at a particular interaction step. We use evidence in a functional sense to describe an observation or identifiable memory item that is used or made available to support, ground, verify, or revise a claim, decision, action, or memory state.

Text-based-agent research established many of the designs now reused by multimodal systems: MemGPT (Packer et al., 2023) organizes working, recall, and archival memory; Generative Agents (Park et al., 2023) transform observation streams into higher-level reflections; MemoryBank (Zhong et al., 2024) combines dialogue history, user modeling, and forgetting; Reflexion (Shinn et al., 2023) stores feedback as reusable linguistic lessons; and Voyager (Wang et al., 2024a) preserves executable programs that can be retrieved and composed across tasks. Together, these works show that memory may serve as a context manager, a structured knowledge store, a reflective record, or a procedural skill library.

Multimodal memory inherits these functions but introduces a more difficult representation problem. Images contain spatial detail that may not be captured by a caption (Yu et al., 2025c); video and audio unfold continuously and must be segmented over time (Zhang et al., 2025d); interface observations bind appearance to actionable screen coordinates (Choi et al., 2026); and embodied experience must be associated across viewpoints, object identities, and changing locations (Fan et al., 2025; Wen and Bekris, 2021; Wen et al., 2023; Xie et al., 2026a). As a result, the decision to retain a frame (Yu et al., 2025a), caption an event (Wang et al., 2025a), fuse a 3D map (Zhu et al., 2026a), cache visual tokens (Song et al., 2026), or internalize experience in parameters (Liu et al., 2025a) changes what the agent can later recover.

Two broad strategies have emerged. Some systems preserve modality-specific evidence or states so that later computation can return to visual (Shlapentokh-Rothman et al., 2026), auditory (Someki et al., 2026), or geometric information (Liu et al., 2025b). Others translate experience into language (Feng et al., 2026), structured symbols (Long et al., 2026), shared latent spaces (Yu et al., 2026b), or parameters (Suri et al., 2026) that are easier to search and reuse. Native preservation keeps a wider range of perceptual detail but increases storage and retrieval cost. Translation produces compact and editable memory units but can remove information that was not anticipated when the memory was constructed. This trade-off motivates our memory formation and representation axis.

2.1 Classical Memory in AI & Cognitive Science

Memory has long been viewed as a requirement for intelligent behavior. In cognitive science, models often distinguish between short-lived perceptual storage, limited-capacity working memory (Baddeley and Hitch, 1974), and more durable long-term memory (Atkinson and Shiffrin, 1968). This view suggests that memory is not only a place to keep information, but also a mechanism for selecting, organizing, and using information during reasoning and action.

Cognitive theories also distinguish memory by function. Episodic memory stores experiences tied to particular events, while semantic memory stores general knowledge that can be reused across contexts (Tulving, 1972). Procedural memory supports skills and action routines (Squire, 1992). These distinctions are useful for agent systems because memory may preserve specific past experience (Li et al., 2026g), provide reusable abstractions (Li et al., 2026c), or support future behavior (Qin et al., 2026).

Classical AI studied memory from a computational perspective. Symbolic systems represented knowledge as facts, rules, frames, scripts, or production rules, while cognitive architectures such as SOAR (Newell, 1990) and ACT-R (Anderson et al., 2004) placed memory inside a control loop for problem solving and decision making. Case-based reasoning made experience reuse explicit by solving new problems through the retrieval and adaptation of past cases (Aamodt and Plaza, 1994). These ideas anticipate many concerns in modern agent memory, including indexing (Zhu et al., 2026c), retrieval (Qiu et al., 2026), consolidation (Chen et al., 2026c), forgetting (Mi et al., 2026), and reuse (He et al., 2026).

Neural models changed the implementation of memory. Recurrent networks store information in hidden states (Hochreiter and Schmidhuber, 1997), while attention-based models can use the context as temporary memory (Vaswani et al., 2017). Memory-augmented neural networks further introduce external modules that can be read and written during computation (Graves et al., 2014; Weston et al., 2015). For modern agents, the central issue is therefore not only how information is represented internally, but also what experience persists beyond the current computation and how that retained information is later accessed or changed. This perspective connects classical memory research to the two functional stages considered in this study: formation and representation, which determine what experience is retained and in what form it reaches downstream computation, and memory management, which determines how retained memory is accessed, maintained, and adapted over time.

2.2 Foundations of Modern Agentic Memory

Retrieval-augmented generation (RAG) provides an important conceptual foundation for modern agentic memory by separating model parameters from an external, queryable information source (Lewis et al., 2020). At inference time, retrieved evidence is introduced into the model context, enabling access to information beyond parametric knowledge and the current input. In its standard form, however, RAG primarily addresses how stored information is accessed. It does not by itself specify how an agent should form memory from its own experience, maintain that memory over time, or revise it through later interactions.

Modern agent memory broadens this retrieval-centered view by treating prior interaction and experience as persistent state that can be retained, transformed, and reused across tasks or sessions. Systems such as MemGPT (Packer et al., 2023), Generative Agents (Park et al., 2023), and MemoryBank (Zhong et al., 2024) helped establish this shift from external knowledge retrieval toward persistent experience management. Taken together, these systems move memory beyond factual retrieval toward persistence, transformation, and experience reuse across interactions. They provide conceptual foundations for multimodal agent memory, where the same problems arise over richer forms of experience and a broader range of representations.

When considering how long-horizon agents retain and use information accumulated over extended interactions, it is important to distinguish three closely related concepts: long-context processing, context management, and memory. Long-context processing refers to a model’s capacity to capture and use information distributed across extended input sequences at manageable computational cost. Context management determines which information is included in the active context at each inference step and how it is presented to the model. Memory, as defined in this study, denotes experience-derived state that persists beyond the current inference step and subsequently influences reasoning, generation, planning, or action. Depending on its representation and integration, memory can support long-context processing by carrying historical information across input segments and support context management by selecting relevant memory content as context for the model. Long-context processing and memory augmentation serve complementary aspects for building long-horizon and lifelong agents: memory extends access to information beyond the active context, while long-context models integrate retained information from memory into current computation (Wu et al., 2022; He et al., 2025b). Beyond these functions, persistent memory enables agents to accumulate experience across interactions, tasks, and sessions, providing a substrate for inference-time adaptation and longer-term agent evolution.

2.3 The Multimodal Memory Lifecycle

Multimodal agent memory can be understood through two coupled functional stages: (1) formation and representation and (2) memory management. Formation determines what experience is retained, how it is organized, and the representation through which it becomes available to downstream computation. Memory management determines how the resulting state persists, changes, is accessed, and is used during deployment. These stages form a lifecycle rather than independent modules: management is constrained by what formation preserves, while the outcomes of memory use may trigger further formation, revision, consolidation, or forgetting. Figure 4 illustrates the two coupled stages of multimodal agent memory and the three memory-management tiers through a household-robot example.

Formation and Representation.

The first stage, formation and representation, determines what experience is saved as memory, how it is organized into memory units, and how those units are represented. The definition of a memory unit varies across modalities and levels of granularity. For static images, a unit can be an entire image, region, object, or scene (Nguyen et al., 2026b; Wang et al., 2026j). For video and audio, it may correspond to a keyframe, clip, event, object track, or temporally aligned audiovisual segment (Song et al., 2024; Lin et al., 2026d). In embodied settings, memory units may further capture multimodal observations, estimated world and object states, action-observation transitions, trajectories, or complete interaction (Liu et al., 2025b; Zhang et al., 2026d; Weng et al., 2024). Once these units are formed, the agent may preserve raw or compressed sensory evidence (Fan et al., 2024), encode modality-native features or tokens (He et al., 2024), translate perceptual observations into textual or structured symbolic records (Feng et al., 2026), organize spatial and temporal relations as event graphs, scene graphs, or spatial maps (Gu et al., 2024), encode accumulated experience in recurrent latent states (Qian et al., 2024), or internalize it through updates to model parameters (Suri et al., 2026).

Memory Management.

The second stage governs how formed memories are retained, updated, accessed, and used during deployment. Axis II distinguishes three tiers according to whether the memory state and the policy that manages it change during use. T1: Static Memory stores and reuses fixed memory without modifying its content, organization, access indices, or metadata. T2: Dynamic Memory updates the memory state through a stable management policy, including appending new observations (Liu et al., 2025b), revising existing state (Feng et al., 2026), merging redundant entries (Song et al., 2024), consolidating repeated evidence (Li et al., 2026b), invalidating outdated information (Liu et al., 2026b), and forgetting under capacity constraints (Xiao et al., 2026b). T3: Self-Evolving Memory updates the memory state while also revising the policy that governs memory writing, maintenance, or access in response to feedback. Access includes the retrieval (Zhang et al., 2026e), routing (Sun et al., 2026b), or activation (Wang et al., 2026h) mechanisms that make relevant memory available to current computation. Utilization determines how the accessed information is incorporated into reasoning, generation, planning, action, or learning. These processes form a coupled management loop but remain distinct failure points: an agent may fail to retrieve relevant evidence, retrieve it but apply it incorrectly, or produce a correct output from model priors without using memory (Tang et al., 2026c; Rathi, 2026). Management is also constrained by formation and representation: no access mechanism can recover information that was discarded during formation or subsequently corrupted, invalidated, or forgotten.

Axis I, Formation and Representation, classifies the memory representation through which retained experience becomes available to downstream computation. Axis II, Memory Management, classifies systems by the scope of persistent deployment-time change they support in memory state and management policy. The resulting taxonomy is summarized in Figure 5.

3 Axis I: Formation and Representation

In Axis I, we examine how multimodal experience forms persistent memory and the representations that will be retrieved and used in future reasoning or action. As summarized in Figure 6, we classify by the representation ultimately supplied to the reasoner or action policy, rather than merely by the format used as retrieval index. Formation mechanisms determine the representational form in which raw experiences are encoded and retained as memory. This process creates distinct memory units and determines what information remains available for subsequent use. The same image, video, audio stream, or embodied episode may be retained as source evidence (Fan et al., 2024), transformed into modality-specific visual tokens (Song et al., 2024), acoustic features (Wu et al., 2026b), temporally organized video features (He et al., 2024), or geometric state (Gu et al., 2024), abstracted into linguistic (Yin et al., 2026), symbolic (Jiang et al., 2026a), relational (Long et al., 2026), or executable records (Wang et al., 2024a), or maintained as persistent latent state (Qian et al., 2024) or knowledge encoded in model parameters (Suri et al., 2026). Accordingly, we organize this axis into three families.

Many systems combine multiple representations (Yan et al., 2025b; Li et al., 2026g; Huang et al., 2026c). For example, a system might use a semantic graph to locate compressed visual features and then reopen the source image for verification (Fan et al., 2024; Long et al., 2026). We therefore discuss hybrid and evidence-linked architectures separately as system-level compositions rather than as a fourth representation type.

3.1 Source- and Modality-Specific Representations

At the least abstract end of the representation axis, multimodal experiences are saved in their originating modality. We distinguish two forms. Retained source evidence preserves images (Zhang et al., 2026a), video or audio segments (Liu et al., 2026j), and sensor observations (Dorbala and Manocha, 2026) as media that the agent can reopen during inference. Modality-native encodings instead retain learned features (Diko et al., 2025), tokens (Zhao et al., 2025a), or geometric state (Honerkamp et al., 2024) that preserve modality-specific structure without requiring the original media to be processed again. Both avoid reducing experience to language or symbolic records, but they serve different priorities: source evidence remains directly verifiable and interpretable, whereas encoded state is more compact and computationally efficient but is often lossy and difficult to interpret.

3.1.1 Retained Perceptual Source Evidence

Depending on the input, these items may be complete images (Nguyen et al., 2026b), regions or crops (Choi et al., 2026), document screenshots (Yu et al., 2025c), sampled video frames (Zhang et al., 2025d), audiovisual segments (Lin et al., 2026d), RGB-D observations (Yang et al., 2025b), or observations paired with timestamps and agent poses (Yan et al., 2025b).

For static images and graphical user interfaces (GUIs), retaining visual evidence preserves layout, typography, color, fine-grained appearance, and spatial organization that textualization may discard (Yu et al., 2025c; Wang et al., 2026j). PixelRAG (Wang et al., 2026j), for example, renders webpages as screenshot tiles, indexes them using visual embeddings, and supplies the highest-ranked tiles directly to a VLM as pixel inputs. In this case, the embeddings serve only as a retrieval index, whereas the retrieved screenshots constitute the source-level visual evidence made available during inference. Source-evidence memory can also operate at different granularities: related systems retain complete screenshots (Liu et al., 2025d), action-relevant GUI regions (Choi et al., 2026), cropped interface elements (Zeng et al., 2026) , or complete visual interaction histories with frame- and region-level access (Han et al., 2026), and selectively retrieve the visual observations most relevant to the current decision.

Video and audio pose an additional access challenge because relevant evidence must first be localized within a continuous temporal stream. Existing systems address this challenge through different organizational structures (Lian et al., 2026). ToolMerge (Shlapentokh-Rothman et al., 2026) retains sampled video frames and uses an LLM planner to compose complementary visual retrieval tools, including global image-text matching, region-level entity matching, and OCR. HippoMM (Lin et al., 2026d) adopts an event-centered organization, where audiovisual experience is segmented into event records with semantic summaries, temporal boundaries, and pointers to source spans. M3-Agent (Long et al., 2026) instead organizes memory around persistent entities, linking text, image, and audio evidence to people, objects, and events over time. These designs correspond to frame-, event-, and entity-level organization, respectively, showing that retaining source media is only one part of the problem; the evidence must also be indexed at a granularity suited to later retrieval.

Spatial indexing plays a similar role in embodied agents (Li et al., 2026e). GraphEQA (Saxena et al., 2025) combines a hierarchical 3D scene graph with a task-relevant memory of images captured during exploration, allowing retrieved images to restore local visual details that may be abstracted away in the global graph. DovSG (Yan et al., 2025b) associates RGB-D observations and camera poses with a dynamic object-level map so that historical observations can support relocalization and local map refinement when the environment changes. These systems show that source evidence and structured spatial memory serve complementary roles: the structured representation supports compact, global reasoning, while linked observations preserve detailed perceptual evidence for inspection, grounding, and update.

The primary advantage of preserving source-modality evidence is recoverability: information retained in the original observation can be retrieved at different levels of granularity for later uses that may not have been anticipated during memory formation. This preserves fine-grained visual details, temporal structure (Zuo et al., 2025), acoustic cues (Someki et al., 2026), and observational context (Anwar et al., 2025), while also allowing later reasoning and actions to be grounded in and checked against specific observations.

3.1.2 Modality-Specific Encodings and Geometric State

Retaining source observations maximizes recoverability, but storing, retrieving, and repeatedly encoding every image, video frame, audio segment, or sensor observation can be prohibitively expensive for long-running agents. A more compact strategy is to persist encoded features or geometric state. We assign a memory architecture to this category when the stored value is itself consumed by the reasoner or action policy and preserves task-relevant structure inherited from the originating modality, such as spatial organization, temporal order, acoustic dynamics, or metric geometry.

Video provides the clearest setting for modality-specific encoded memory because long streams require compact representations that preserve both spatial and temporal information. Representative systems differ mainly in the granularity and consolidation strategy of the retained features. MovieChat (Song et al., 2024) accumulates frame-level visual tokens in a fixed-capacity short-term memory and repeatedly merges similar representations from temporally adjacent frames before transferring the consolidated features to long-term memory. MA-LMM (He et al., 2024) instead stores patch-level features in a visual memory bank, where they directly serve as keys and values for cross-attention; when capacity is exceeded, temporally adjacent features are merged at corresponding spatial locations. Flash-VStream (Zhang et al., 2025b) uses a coarse-to-fine design in which low-resolution feature centroids provide broad temporal coverage and high-resolution feature maps preserve detail for representative key frames. At query time, these features are combined according to their temporal positions and supplied to the language model through the visual projector. These systems illustrate three forms of encoded video memory: temporally consolidated frame tokens, spatially aligned patch features, and coarse-to-fine representations of a continuing stream. Despite their different structures, all persist visual features as the memory records itself rather than merely as retrieval indices for source frames. Modality-specific encoding can therefore limit memory growth while retaining spatial and temporal structure needed for later inference, although the resulting memory remains tied to a compatible model interface.

Image memory follows a different pattern. Representative agent systems more commonly retain decodable source images or pointers to them and use visual embeddings only for indexing and retrieval (Nguyen et al., 2026a; Deng et al., 2026). M3-Agent (Long et al., 2026), for example, stores image content alongside embeddings used for similarity-based retrieval, while M2A (Feng et al., 2026) associates stored images with textual captions and cross-modal embeddings that support both text- and image-based search. Other systems transform visual observations into textual (Gao et al., 2025) or structured records (Wen et al., 2024) and discard the source images. UniversalRAG (Yeo et al., 2026a) extends this indexing pattern across heterogeneous corpora by routing queries to modality- and granularity-specific retrievers. MIRIX (Wang and Chen, 2025) follows this approach for screenshot-based memory. These designs show that image encoders frequently support memory formation and retrieval, but their feature vectors less commonly serve as persistent memory values consumed directly by the downstream model.

Persistent encoded audio memory remains underexplored among the analyzed systems. Unlike speaker embeddings used only as retrieval keys (Yao et al., 2025b), such memory would preserve acoustic information as the stored value and expose it directly to the downstream model. OMAC (Wu et al., 2026b), introduced in O-MARC, provides a relevant encoding mechanism. It retains high-importance audio tokens as anchors and merges nearby discarded tokens into them through weighted aggregation. The resulting tokens remain temporally ordered and are consumed directly by a compatible omnimodal backbone without reconstructing the original waveform. OMAC therefore illustrates how acoustic features could serve as memory unit values rather than retrieval indices. However, it compresses the current audiovisual context rather than preserving information across interactions, so it is better viewed as a transferable representation mechanism than as an established example of persistent agent memory. This distinction also highlights how little persistent audio-feature memory has been explored relative to its visual counterpart.

Embodied agents extend modality-specific memory from neural feature sequences to world-centered geometric state (Huang et al., 2025; Zhu et al., 2026a). DynaMem (Liu et al., 2025b) converts posed RGB-D observations into an evolving sparse voxel map in a global coordinate frame. Its voxels combine metric location, observation history, timestamps, and vision-language features, while new depth observations update or remove inconsistent state. This structure preserves metric relations and occupancy changes that would be difficult to recover from an unordered feature collection. A different design is to retain perceptual history for action generation rather than to construct an explicit world model. MemoryVLA (Shi et al., 2026a) compresses visual-encoder outputs into perceptual tokens that query a temporally indexed long-term bank. Retrieved history is fused with the current perceptual state and used to condition the action policy, while capacity is controlled by merging similar temporally adjacent entries. The resulting bank therefore stores a bounded history of policy-relevant visual features without reconstructing source observations. The key distinction lies in the role of memory: DynaMem organizes memory around persistent world geometry, whereas MemoryVLA organizes it around perceptual history required by the control policy.

Across these systems, the representation is decided by the structure required by the downstream interface. Video memories preserve spatial and temporal organization (He et al., 2024), emerging acoustic encodings retain temporally localized sound information (Wu et al., 2026b), geometric maps expose metric layout and occupancy (Liu et al., 2025b), and policy-facing memories retain action-relevant perceptual history (Shi et al., 2026a). Persisting these representations reduces storage, context, and repeated encoding costs while preserving information that may be difficult to express through language or symbolic records. The resulting compression is generally non-invertible: information removed through sampling (Yu et al., 2025b), merging (Song et al., 2026), clustering (Zhang et al., 2025b), spatial reduction (Guo et al., 2026a), or voxelization (Lin et al., 2026c) cannot be recovered from the encoded state alone. Learned feature memories are also model-specific and normally require a compatible projector, Q-Former, attention module, or policy interface. Geometric state is more explicitly structured, but discretization and feature aggregation can still remove perceptual detail. Modality-specific encodings and geometric state therefore trade recoverability and portability for compact, task-aligned access to perceptual and spatial structure.

3.2 Cross-Represented Explicit Memory

Multimodal experience can also be transformed into natural language or structured representations such as knowledge graphs. For example, an image may be stored as a caption (Feng et al., 2026), while an embodied observation may be represented as a scene graph containing nodes and relations between them (Gu et al., 2024). Because the relevant content is expressed through readable text or named graph components, it can be directly retrieved and used for reasoning or action. Compared with modality-specific encodings and latent or parametric memory, these representations can usually be inspected, searched, edited, or invalidated at the level of individual entries. However, their construction is also lossy: detailed visual information omitted during captioning, entity extraction, or relation construction usually cannot be recovered from the resulting representation alone.

Within Cross-Represented Explicit Memory, Perception-to-Text Records and Summaries translate multimodal experience into natural-language entries that are retrieved and supplied to the agent as text. Although these entries may be indexed by timestamps, identifiers, or embedding similarity, the content placed in the backbone LLM’s context remains textual. Spatial, Temporal, and Cross-Modal Structured Memory organizes information as graph-linked entities and relations or schema-defined fields that can be queried and updated individually. For example, a vector database of captions is textual memory, whereas a scene graph is structured memory when its nodes, relations, or topology support retrieval, reasoning, or action. Because a single architecture may combine multiple representation forms, we discuss each form in its respective section rather than assigning the architecture as a whole to a single category.

3.2.1 Perception-to-Text Records and Summaries

Textual memory expresses past experience as descriptions (Jain et al., 2025), event records (Zhang et al., 2026c), summaries (Rasheed and Kabtoul, 2026), facts (Liu et al., 2026g), profiles (Wang and Chen, 2025), or natural-language procedures (Fan et al., 2026b). Its memory unit is a linguistic entry that can be retrieved through semantic similarity, lexical matching, temporal filtering, or structured querying and then inserted directly into an LLM context. This representation aligns naturally with language-model reasoning and permits human inspection and correction without requiring a decoder. Textual memory is also well suited to evolving state because updates, corrections, and overrides can be directly incorporated into compact, ordered records, whereas modifying or reconciling multimodal evidence and its embeddings is less straightforward . It also separates storage from the original perception model: a textual memory produced by one visual or audio encoder can generally be read by another language model.

Long video provides a clear setting for this strategy because retaining every frame is expensive, while many downstream questions concern events, actions, and state changes that can be summarized linguistically. VideoAgent (Fan et al., 2024) constructs a compact semantic memory of temporal event descriptions and allows an LLM agent to query these records while deciding when additional visual evidence is needed. Video-EM (Wang et al., 2025a) makes the event abstraction more explicit. It localizes potentially relevant moments, groups them into temporally coherent events, and writes grounded episodic records containing temporal indices and when, where, what, and who cues. A chain-of-thought retrieval process then selects a minimal set of event memories for answering the current question. ReMA (Chen et al., 2026a) processes a lifelong video stream as fixed-duration clips, writes timestamp-aligned descriptions into a vector-indexed memory bank, and merges temporally overlapping observations into updated summaries. Its controller later searches these textual records and adds the results of query-directed inspections back into the same belief state. Across these systems, linguistic abstraction changes the basic memory unit from a frame to a semantically delimited event or observation, allowing temporally distant evidence to be addressed without repeatedly processing the complete video.

Textual memory is also prominent in GUI agents, where screenshots and interactions can be converted into persistent facts (Gao et al., 2025), events (Liu et al., 2025d), and workflows (Mi et al., 2026). MIRIX (Wang and Chen, 2025) filters visually redundant screenshots and routes new screen and conversational evidence to specialized memory managers. These managers construct natural-language summaries, event details, semantic records, workflow steps, document excerpts, and user-profile facts across six memory components. During retrieval, the system searches these components using embedding-, BM25-, or string-based methods, and the selected entries are ultimately supplied to the chat agent as labeled text. The resulting representation abstracts transient interface appearance into concepts that remain useful across later interactions, such as user activities, previously accessed resources, or recurring workflows.

Persistent image memory is less commonly represented solely in text. A single image is comparatively inexpensive to retain, while future queries may depend on typography, color, layout, object appearance, or fine spatial relations that cannot always be anticipated when a caption is written. Purely textual image memory is therefore most useful when images occur as part of a longer personal or interaction history, such as a screenshot stream or camera roll (Wang and Chen, 2025; Nguyen et al., 2026a), and the relevant memory objects are people, activities, events, or preferences rather than arbitrary visual details. In such settings, captions and event summaries provide efficient semantic access, while source images can be retained separately when later visual verification is required (Jiang et al., 2025b). This pattern explains why text-only image memory is less dominant than textual event memory for long video and interaction histories.

Pure textual memory is less common for embodied experience as well. Text can summarize observed events and completed tasks, but embodied action often depends on precise and current information about object locations, metric distances, containment, connectivity, and other spatial relations that cannot be reliably reconstructed from isolated free-form text records. Embodied systems therefore typically associate linguistic descriptions with spatial maps or entity-centered state rather than relying on unstructured text alone (Riva et al., 2026; Lei et al., 2025). Textual records remain valuable for episodic histories, task outcomes, and high-level instructions, while localization and planning generally rely on the structured representations discussed in the next section.

Across modalities, textualization provides a compact and interpretable interface between perception and reasoning. It supports semantic retrieval, direct insertion into model context, human editing, and consolidation of observations across modalities. Its primary limitation is the semantic bottleneck introduced when memories are written. Captions and summaries retain information judged salient by the formation model, but may omit visual details, flatten temporal or spatial relationships, discard acoustic cues, or introduce unsupported interpretations. Because retrieval cannot recover information that was never encoded in the textual record, the quality of textual memory depends not only on retrieval accuracy but also on the coverage, grounding, and factual reliability of the textualization process.

3.2.2 Spatial, Temporal, and Cross-Modal Structured Memory

Graphs and structured records represent experience through explicitly organized and addressable entities (Zhang et al., 2025c), attributes (Sun et al., 2026e), relations (Jiang et al., 2025a), state variables (Wang et al., 2025c), temporal links (Chang and Ren, 2026), or spatial structures (Honerkamp et al., 2024). Their meaning derives not only from the content of individual nodes or records but also from the relationships and organization among them. Accordingly, access may involve field-level lookup, graph traversal, temporal filtering, relational propagation, or spatial queries, rather than treating each entry as an independent text passage. Such representations are particularly valuable when an agent must determine not only what occurred, but also how entities are related, which state is currently valid, or where an action should be executed.

Embodied agents provide particularly clear examples of structured memory, as navigation and manipulation often require persistent representations of objects, their spatial and functional relationships, and, in dynamic environments, changes in scene state over time. ConceptGraphs (Gu et al., 2024) lifts open-vocabulary detections from successive RGB-D observations into a 3D coordinate frame and incrementally fuses them into object nodes and relational edges. The resulting scene graph supports language-grounded, object-centric, and spatial access, allowing an agent to retrieve entities and their locations for downstream perception and planning. DovSG (Yan et al., 2025b) extends this design to changing environments. It maintains persistent 3D object nodes with geometric and semantic features, connects them through spatial relations, and locally refines regions affected by robot interactions or environmental changes. The task planner reads target coordinates, object features, and graph relations to decompose instructions and support navigation, grasping, and placement. In this case, the graph is not merely an index over observations; it is the current world state on which planning operates. GraphEQA (Saxena et al., 2025) demonstrates how structured spatial memory can guide active information acquisition. During exploration, it incrementally constructs a hierarchical 3D metric-semantic scene graph containing object, room, and frontier information. The graph exposes what has already been observed, how the explored space is organized, and where additional evidence may be obtained. This state is supplied to a VLM planner that decides whether the available memory is sufficient to answer the question or whether the agent should continue exploring. Structured spatial memory therefore supports both retrospective access and prospective action: it preserves previously acquired observations while guiding the agent toward locations where additional evidence can be gathered.

Video agents impose similar structure on temporal experience. PyraVid (Yan et al., 2026) organizes streaming video into a three-level graph comprising fine-grained fact nodes, clip-level summaries, and a global narrative. Hierarchical aggregation edges connect facts to clips and clips to the global memory, while additional logical links connect related evidence across time. Retrieval begins from query-relevant seed nodes, checks whether the evidence is sufficient, and traverses the graph to expand or prune the active evidence set. WorldMM (Yeo et al., 2026b) constructs multiple temporal knowledge graphs from video segments at different resolutions and an evolving semantic graph from consolidated entity-action-entity triplets. Episodic access uses graph-based Personalized PageRank followed by cross-scale reranking, while semantic access performs edge-focused traversal over the consolidated graph. New triplets can update or remove outdated and conflicting relations. These systems show how explicit temporal and relational structure can preserve event dependencies that would be difficult to recover from an unordered collection of captions.

For persistent image memory, structured representations are particularly useful when an agent must track entities and their attributes across multiple observations. TAME (Hong et al., 2026), for example, maintains a Dynamic State Memory for transient or changing attributes and a Static Personalized Memory for stable attributes, both indexed by personalized concept identifiers. Attributes that remain stable can migrate from dynamic to static memory, while outdated attributes are removed from the bounded dynamic store. During retrieval, the system identifies the personalized concept in the current image and retrieves its associated attributes from both memories. Separating dynamic from stable attributes helps the agent distinguish transient observations, such as a recent appearance or condition, from persistent user-specific properties.

Structured visual memory can also encode the presence status of remembered concepts. PolarMem (Chen et al., 2026h) constructs a polarized graph that links visual memories to concept nodes through HAS and NOT_HAS edges, thereby distinguishing verified presence from verified absence. At query time, its logic-aware retrieval procedure enforces logical consistency before ranking candidates by semantic similarity. Memories that conflict with the query constraints are therefore suppressed even when they are semantically similar. PolarMem thus demonstrates that graph structure can represent not only associations among memory items but also positive and negative evidence. This capability is particularly valuable when an agent must explicitly rule out concepts that are visually plausible but absent from the observed evidence.

Persistent structured memory is also relatively uncommon in GUI agents. Although a GUI presents structured elements at any given moment, its layout and element hierarchy may change across application versions, window sizes, navigation states, and dynamically generated content. Memories tied to absolute screen coordinates or specific interface graphs may therefore generalize poorly across tasks and sessions. Most GUI agents more commonly retain textual interaction histories (Agashe et al., 2025) or parameterized workflow templates (Xiao et al., 2026a) that abstract away from a particular interface configuration. When structured GUI memory is used, it therefore tends to represent task state (Wang et al., 2024c), interface entities (Yu et al., 2026a), or reusable procedures (Lee et al., 2024) rather than a persistent geometric map of the interface.

Graphs and structured records support compositional queries, localized updates, temporal reasoning, and relation-aware planning that are difficult to perform over independent, free-form text. These capabilities, however, introduce substantial construction and maintenance costs. Entities must be detected and resolved across observations; attributes and relations must be extracted accurately; schemas must accommodate previously unseen concepts; and updates must maintain consistency as the environment evolves. Errors in entity creation, relation extraction, or cross-observation data association may propagate through retrieval, graph traversal, and downstream planning. Structured representations therefore improve addressability and control but do not inherently guarantee faithful memory content. Their reliability depends on both accurately transforming perceptual evidence into structured semantic content and correctly reconciling new evidence with the existing graph by updating existing entities and relations or introducing new ones.

3.3 Internalized Latent and Parametric Memory

While external memory preserves information as independently stored and retrievable artifacts, such as raw observations, textual records, structured entries, or modality-specific features, latent and parametric memory integrate experience more directly into the model’s computation. Latent memory maintains experience as continuous states that are carried and updated during inference, whereas parametric memory internalizes experience in model weights or dynamically generated adapters that shape subsequent reasoning and action.

We distinguish two forms of memory that integrate experience directly into model computation. Persistent multimodal latent states represent experience as continuous vectors, memory tokens, hidden states, or key–value units that are separate from backbone parameters. These states can be updated during inference and integrated into the model through attention, residual connections, or learned projection and fusion modules. Recent systems apply this approach to retain visual information and intermediate reasoning states (Yu et al., 2026c), coordinate memory across visual agents (Yu et al., 2026b), and compress GUI interaction histories into model-compatible continuous context (Zhang et al., 2026b). On the other hand, parametric internalization encodes experience in model weights or experience-specific adapters. Internalized memory influences subsequent inference implicitly, without requiring retrieval or inclusion of the original multimodal evidence in the input context. For example, Frames2LoRA (Suri et al., 2026) generates a LoRA adapter from a video, enabling the adapted model to use information from that video without processing its visual tokens again. Instead, MemVerse (Liu et al., 2025a) periodically distills selected knowledge from an explicit long-term memory store into a parametric memory model.

Both forms provide compact access to past experience and integrate memory closely with the model’s reasoning and control processes. They can reduce repeated encoding, retrieval costs, and context growth while retaining information that may be difficult to capture in language or symbolic structures. This close integration, however, limits direct control over the stored information. Latent and parametric memories are generally more difficult to inspect, edit, trace to specific observations, or transfer across models with incompatible architectures. Latent compression may discard information that cannot be recovered, while repeated parameter updates may cause interference or forgetting.

3.3.1 Persistent Multimodal Latent State

Common implementations use hidden states (Qian et al., 2024), memory tokens (Wang et al., 2025b), or key–value caches (Xiao et al., 2026b) and integrate them with the current input through attention, residual connections, or learned projection and fusion layers.

KV caches can also serve as persistent latent memory when they are deliberately retained and reused beyond the computation that produced them. ReKV, for example, incrementally constructs and retains video-derived KV states and selectively restores query-relevant blocks for later question answering (Di et al., 2025). MuKV further compresses multi-grained KV states to support long-horizon streaming video question answering under bounded memory (Xiao et al., 2026b).

Shared and recurrent latent states have been widely explored in image-based agents to preserve visual information and intermediate reasoning across inference steps. VisMem (Yu et al., 2026c) uses a short-term memory former to generate latent tokens containing fine-grained evidence from the current visual input, and a long-term memory former provides higher-level semantic knowledge relevant to the current reasoning context. The model learns when to invoke each memory type and inserts the resulting latent tokens into the inference sequence. TRAM (Liu et al., 2026h) preserves information from earlier reasoning steps so that it can inform later stages of a multi-step reasoning process. After each reasoning step, the model summarizes its internal activations and uses them to update two recurrent latent states with different decay rates. The fast state emphasizes recently completed reasoning, while the slow state retains information over a longer portion of the trajectory. Their combined representation is then added to selected decoder layers at the beginning of the next reasoning step. These methods address a common limitation of extended multimodal reasoning: visual evidence and intermediate conclusions may lose influence as the generation grows longer (Wu et al., 2026a).

For long video streams, latent memory can reduce the cost of repeatedly processing accumulated visual context (Chen et al., 2026c). Light-Omni (Nie et al., 2026b) maintains a bounded, non-parametric global state by progressively summarizing and merging episodic scripts. This hierarchical state preserves recent events in greater detail while representing older events through higher-level summaries. Given the global state and the current multimodal input, the model generates latent features that determine whether to speak or search and produce an embedding for retrieving relevant entries from long-term memory. Light-Omni therefore adopts a hybrid design: explicit episodic scripts and their consolidated global state preserve historical information, while the generated latent features support immediate action control and context-aware retrieval. This design is well suited to long-video tasks, which require both a compact representation of global context and access to specific events, temporal details, or original multimodal evidence.

Beyond modality-specific designs for images and video, latent memory can also support information sharing among multiple vision-language agents. L2-VMAS (Yu et al., 2026b) maintains two shared memory banks: a perception memory that stores latent representations of visual inputs and a thinking memory that stores hidden states from previous agents’ reasoning. Both are organized as key-value memory units that can be retrieved and integrated into subsequent agent computation.

GUI agents use latent memory to connect interaction history with the current interface state (Wu et al., 2025b). Mem-W (Zhang et al., 2026b) uses a shared trajectory-to-latent compressor to encode two sources of GUI memory: completed trajectories retrieved from an external memory bank and the expired prefix of the current interaction. The resulting memory tokens are combined with the task instruction, current screenshot, and recent observation-action context in a unified continuous embedding sequence. Retrieved trajectories provide procedural evidence from both successful and failed experiences, while the compressed in-session history preserves unfinished task progress. Both forms of experience can therefore directly inform the agent’s next action without being converted into textual summaries. This representation is well suited to GUI tasks, where the relevant state is distributed across visual observations, previous actions, and intermediate task progress.

This approach extends naturally from GUI interaction to embodied control, where agents must relate current observations to previous actions and task progress (Li et al., 2026j). LaMem-VLA (Qu et al., 2026) organizes an agent’s interaction history into two complementary memory vaults: a short-term vault that retains recent visual information and a long-term vault that preserves task progress and action continuity. Given the current observation and instruction, the model retrieves relevant information from both vaults and compresses it into fixed-length latent memory tokens. These tokens are combined with the current visual, instruction, and action-query tokens in a unified embedding sequence, allowing historical experience to directly guide action generation . This design is well suited to Vision-Language-Action (VLAs) models because it integrates perception, memory, and control within a shared continuous representation space.

Persistent latent memory remains relatively underexplored in audio and speech agents. These systems more commonly preserve semantic content as transcripts (Jiang et al., 2026b) and represent speaker identity, prosody, and other acoustic properties as separately accessible features (Someki et al., 2026). Such representations support direct retrieval and speaker attribution while retaining temporal and acoustic evidence that recurrent compression may discard. Consequently, persistent latent states designed to preserve general auditory experience across long interactions or multiple episodes remain less developed than their counterparts in visual reasoning, GUI interaction, and embodied control.

Across modalities, latent memory operates over different persistence horizons. Some latent states are maintained within a single reasoning trajectory (Yu et al., 2026c; Liu et al., 2026h), whereas others preserve information across successive agents or interaction steps (Yu et al., 2026b; Zhang et al., 2026b). Latent representations may also be constructed on demand from retrieved long-term experience (Zhang et al., 2026b; Qu et al., 2026). Despite these differences, these mechanisms share a common property: past experience is represented in a continuous form that directly contributes to subsequent model computation.

3.3.2 Experience-Conditioned Parametric Memory

A system may update shared model parameters through gradient-based learning (Lin et al., 2026e) or encode experience in instance- or task-specific adapters attached to a frozen backbone (Wang et al., 2026d; Wang et al., 2026k). Once encoded in shared weights or a selected adapter, the information influences model computation without retrieving the original observations or adding them to the input context.

Image-based parametric memory has primarily been explored through multimodal knowledge editing, where the objective is to retain newly introduced or corrected visual facts within a model. MemEIC (Seong et al., 2025) adopts a hybrid design that combines explicit evidence with parametric internalization. It maintains separate LoRA adapters for visual and textual knowledge updates while keeping the pretrained backbone frozen. A visual edit updates only the visual adapter, whereas a textual edit updates only the textual adapter. During inference, the system activates the adapter associated with the query modality; when a query requires both modalities, it activates both adapters and uses a learned connector to combine their representations. This design illustrates both an advantage and a challenge of parametric memory. Incorporating an edit into learned parameters allows the updated knowledge to participate directly in subsequent computation, but repeatedly writing heterogeneous information into shared parameters can cause interference and forgetting. MemEIC addresses this problem by separating parameter updates by modality, while retaining explicit edit records externally for reliable retrieval and verification.

Video introduces a different motivation for parametric memory: supporting repeated queries without retaining and reprocessing a long sequence of visual tokens. Frames2LoRA (Suri et al., 2026) first encodes a video using a frozen vision-language model. A Perceiver-based hypernetwork then converts the resulting layer-wise representations into a video-specific LoRA adapter in a single forward pass. This adapter is attached to the frozen model and acts as a reusable parametric representation of the video. For each subsequent question, the model receives only the textual query, while information from the video influences its response through the adapter parameters; neither the original video nor its visual tokens are included in the input context. Unlike conventional per-video fine-tuning, the method does not iteratively optimize a new adapter for each video. Instead, its trained hypernetwork directly generates the adapter from the video representations. Frames2LoRA therefore demonstrates how parametric memory can convert a particular video into a compact and reusable parameter component, reducing the cost of repeated inference while sacrificing direct access to the original visual evidence.

In GUI agents, parametric memory more commonly captures reusable operational knowledge than the appearance of a particular screen (Jin et al., 2026b; Lin et al., 2026e). SE-GA (Jin et al., 2026b) illustrates this form of experience internalization through a hybrid memory architecture. During task execution, its Test-Time Memory Extension component maintains an explicit repository containing current task state, general GUI rules, and successful trajectories from previous tasks. Its Memory-Augmented Self-Evolution pipeline subsequently uses the accumulated interaction data to improve the agent’s policy through grounding supervision and reinforcement learning, with parameter-efficient LoRA updates used during self-evolution. The explicit repository makes recent and task-specific experience immediately retrievable, whereas the learned policy parameters absorb recurring interaction patterns so that they can influence future actions without replaying every source trajectory. SE-GA therefore exemplifies a slower form of parametric consolidation in which accumulated GUI experience is converted into a more general action policy. Fully parametric, episode-specific GUI memory remains uncommon because successful interaction often depends on exact screenshots, interface states, and action histories that are easier to inspect, revise, and reuse when retained explicitly.

Embodied control provides a clear example of parametric memory stored in separate, reusable adapters. Rather than merging all experience into the model’s shared weights, VLA-Pro (Si et al., 2026) maintains task-specific LoRA adapters that can be retrieved and combined according to the current situation. VLA-Pro first trains a shared LoRA adapter across previously seen manipulation tasks and then learns a separate task-specific adapter for each task. Each memory entry pairs its adapter with a sequence of structured procedural states describing when the stored behavior is relevant. These states specify the action type, object geometry, end-effector orientation, and target interaction point. Given the current visual observation, language instruction, and interaction history, the system extracts the agent’s current procedural state and retrieves the most relevant task memories. It then combines their LoRA adapters using similarity-based weights, temporarily attaches the fused adapter to the vision-language-action model, and uses the adapted policy to generate the next action chunk. VLA-Pro demonstrates that parametric memory need not be incorporated permanently into a single set of shared weights. Parameterized experiences can instead remain modular, indexed by explicit descriptors, and selectively composed according to the current situation. This organization achieves a milestone improvement in cross-task transfer while limiting interference from unrelated manipulation behaviors.

Together, these systems show that parametric memory can operate at different granularities and persistence horizons. MemEIC (Seong et al., 2025) accumulates modality-specific knowledge edits, Frames2LoRA (Suri et al., 2026) generates a separate adapter for each video, SE-GA (Jin et al., 2026b) consolidates interaction experience into a shared GUI policy, and VLA-Pro (Si et al., 2026) stores task-specific action adapters that can be retrieved and composed. Their common property is that past information influences later computation through learned parameters rather than solely through replayed observations or records. Parametric memory can therefore reduce repeated context processing and support direct behavioral adaptation, but it also makes the underlying evidence less accessible and introduces challenges in updating, attribution, interference control, and deletion. Consequently, many multimodal agents combine parametric internalization with explicit memory that preserves source evidence and provides more controllable access.

3.4 Hybrid Representations

The preceding subsections describe three primary forms of multimodal memory: source- and modality-specific evidence, textual and structured records, and latent or parametric representations. Each form offers different advantages. Source evidence preserves perceptual detail, textual and structured records support efficient retrieval and explicit reasoning, and latent or parametric representations can contribute directly to model computation. In practice, however, no single form is sufficient for every memory operation. Hybrid memory is not an additional elementary encoding type; rather, it describes how different representations are organized and used together within a memory system. An important subtype is source-linked memory, in which an abstract record retains a reference to the original observation or to selected evidence preserved from it. The agent can then search and reason over compact abstractions while recovering the corresponding image, frame, audio segment, or interface region when finer perceptual evidence is required.

For collections of images and visually rich documents, hybrid representations allow an agent to organize large amounts of retrieved evidence without reducing all visual content to text (Wang et al., 2026a). VimRAG (Wang et al., 2026e) represents iterative multimodal retrieval and reasoning as a directed acyclic graph. Each node records its dependencies on earlier nodes, the subquery that produced it, a concise textual summary, and a multimodal memory bank containing visual tokens from retrieved images or video frames. The graph exposes the temporal and logical structure of the reasoning process, while the visual tokens preserve information that may be lost in textual summarization. To operate under a bounded context budget, VimRAG dynamically adjusts the number of visual tokens retained for each memory item according to its semantic relevance, recency, and position in the reasoning graph. The method therefore demonstrates a combined hybrid representation: structured and textual components organize the agent’s reasoning state, whereas continuous visual representations preserve perceptual information for later verification and answer generation.

Video streams create a stronger need to connect compact semantic records with recoverable visual evidence because relevant events may occur far apart, and details omitted during summarization may become important only after a query is received. FOLIO (Fan et al., 2026a) combines a short-term buffer of recent frames, a long-term semantic memory, and a visual-evidence cache. As the video arrives, the system constructs structured records describing entities, attributes, locations, relations, actions, and state changes. These records contain timestamps and references to selected keyframes retained in the evidence cache. At query time, FOLIO retrieves relevant semantic records and can restore their associated keyframes when additional visual grounding is needed. The structured memory supports efficient long-horizon access, while the linked frames preserve visual details that the records may omit or describe incorrectly. FOLIO thus exemplifies source-linked hybrid memory: semantic abstraction reduces the cost of storing and searching a long video, but it does not permanently replace the observations from which the memory was derived.

In GUI interaction, source linkage is useful because a textual description may capture the meaning of an earlier interface event while omitting the precise visual target or interface state needed for a later action. MementoGUI (Zeng et al., 2026) maintains both within-episode working memory and cross-episode episodic memory. For a selected interaction step, a working-memory entry contains a textual event summary, the bounding box of the relevant interface region, a crop of that region, and a visual embedding used only to organize the memory. When older entries are compressed, their textual content is summarized, but identifiers for selected visual regions are retained and later resolved back into the corresponding crops. Episodic-memory entries similarly combine trajectory summaries and action metadata with representative interface crops and retrieval embeddings. During action generation, the agent receives the retrieved textual summaries and region-of-interest images through the frozen GUI model’s standard multimodal input interface. MementoGUI therefore represents interface experience through source-linked textual and visual components: text records task progress and the significance of earlier events, while linked crops recover localized details such as icons, labels, selected values, and interface states. Its retrieval embeddings serve only as indices and do not themselves constitute the memory records to be retrieved and used.

In interactive visual environments, hybrid memory can combine a continuously revised semantic state with a complete record of visual interaction, since details that appear unimportant at one step may become useful later. VISTA (Han et al., 2026) maintains two forms of memory in parallel: revisable textual notes that summarize reusable environment knowledge and the agent’s current state, and a lossless visual memory that stores every returned frame in its original form. During reasoning, the agent can rely on the compact textual state for routine planning while selectively revisiting arbitrary past frames or spatial regions when finer-grained perceptual evidence is needed. Unlike source-linked designs in which individual abstract records explicitly reference particular observations, VISTA keeps its textual state and visual history separately addressable and combines them on demand.

Embodied agents require hybrid memory to preserve perceptual evidence while organizing it according to the spatial structure of the environment (Li et al., 2026e). ObsGraph (Lee et al., 2026) constructs an observation-centered hierarchical scene graph with room, view, and object layers. Room-level representations provide coarse semantic anchors, view-level memories preserve observations in which multiple objects appear together, and object-level memories retain fine-grained visual information about individual objects. Relations among these layers connect object evidence to the views and rooms in which it was observed. Given a task, the agent retrieves memory progressively from rooms to views and objects under a bounded budget. The level at which relevant evidence is missing also determines whether the agent should search another room, refine an existing viewpoint, or explore an unobserved frontier. ObsGraph therefore combines semantic hierarchy, spatial structure, and retained visual observations. The structured graph supports efficient retrieval and planning, while its linked visual evidence preserves appearance and co-visibility information that semantic labels alone cannot express. This combination is particularly important in embodied settings because memory must support not only retrospective question answering but also decisions about where to move and what additional evidence to acquire. Beyond navigation, HiMe (Ji et al., 2026) addresses long-horizon manipulation with a hierarchical hybrid-memory architecture that combines recent visual working memory, image–text contextual records, and a structured procedural plan to retain evolving object states and task progress across action sequences.

Across modalities, hybrid representations exploit the strengths of different memory forms. Textual and structured records provide compact semantic access, explicit relations, and controllable updates, whereas retained observations and modality-specific representations preserve perceptual detail and the provenance of abstracted information. Their combination enables a tiered access strategy: agents can rely on compact abstractions for routine retrieval and reasoning, while selectively recovering higher-fidelity evidence when a task requires verification, reinterpretation, or details lost during abstraction. This capability is likely to become increasingly important as multimodal agents operate over longer histories and in dynamic environments, where retaining all observations in the active context is impractical but irreversible compression is unreliable. Hybrid memory may therefore emerge as a prominent design pattern, particularly for applications in which both efficiency and evidential grounding are important. Its effectiveness, however, depends on more than storing multiple representations. Systems must preserve valid links as memories are revised, merged, or removed; maintain consistency between abstract records and their supporting observations; and learn when compact memory is sufficient and when source evidence should be restored. The central challenge is thus to coordinate representations of different fidelity, cost, and accessibility so that an agent can move between efficient abstraction and grounded perceptual recall as its information needs change.

4 Axis II: Memory Management

After forming memory records from raw messages, experience, and related metadata, the memory system must control how these representations are maintained, accessed, and changed as experience accumulates. Figure 7 summarizes this progression from T1 Static Memory to T2 Dynamic Memory and T3 Self-Evolving Memory. Specifically, memory management operations include writing, updating, consolidation, reorganization, retention, and forgetting, while retrieval selects relevant memory for ongoing reasoning and action. These operation mechanisms may be governed by heuristic rules (Yang et al., 2025b), fixed prompts (Ali et al., 2025), frozen learned modules (Li et al., 2026a), optimization procedures (Wang et al., 2026m), or policies that themselves adapt from deployment feedback (Zou et al., 2026a).

Multimodal memory introduces another representational challenge compared to textual inputs. Multimodal inputs are often high-dimensional and expensive to retain in their original form. Therefore, systems must balance storage and retrieval efficiency against the ability to recover source evidence, leading to the diverse representation choices discussed in Axis I. Hybrid systems may preserve multiple representations to balance information fidelity and computational efficiency. Consequently, multimodal memory management involves not only updating memory content but also maintaining consistency across these linked representations. It must determine which representations should change, how updates should propagate among related representations, and whether abstracted memories remain valid and grounded in their source evidence as experience accumulates. Conceptually, we can model these representations as nodes in a representation graph connected by transformation, alignment, fusion, and grounding relations. Memory operations then act on both the representation states and the relations connecting them. We distinguish three tiers according to what can change during inference.

The distinction between memory-state updates and policy evolution depends on the function of the updated component. Memory state includes stored content, organization, access indices, and descriptive metadata, while the management policy governs how memory operations are selected and performed. Changes to retrieval keys, skill descriptions, or usage statistics can alter subsequent retrieval results without changing this policy. Such changes should remain within T2 Dynamic Memory when the selection and update mechanisms remain stable. T3 Self-Evolving Memory goes beyond state modification by using feedback to persistently revise reusable management rules, instructions, procedures, or learned policy components.

Accordingly, the three tiers distinguish whether the memory state and the policy that manages it change during inference. Axis II, Memory Management, organizes systems by the scope of persistent deployment-time change they support. Across these tiers, multimodal information may remain within one representation, be projected into another representation, be fused across multiple modalities, or be jointly maintained across linked representations. The tiers therefore progress from reusing fixed memory, to updating memory through a stable policy, and finally to updating memory through an evolving policy, while the transition patterns characterize how multimodal information moves through the representation system. Updating memory state and evolving management policies can help agents adapt to changing environments and mitigate training–deployment mismatch, but these changes may also introduce instability and make memory updates harder to control. The following subsections examine memory access and reuse in T1 Static Memory, five memory-state operations in T2 Dynamic Memory, and the evolution of writing, maintenance, and retrieval policies in T3 Self-Evolving Memory.

4.1 T1: Static Memory

T1 Static Memory is well suited to large, static corpora (Yu et al., 2025c), stable knowledge bases (Yu et al., 2025c; Hu et al., 2023), and invariant environment or task information that must be retrieved and reused across subsequent reasoning steps or tasks, such as long-video analysis (Fan et al., 2024) or navigation with a preconstructed environment map (Huang et al., 2025). The absence of an update policy preserves the memory state across tasks and prevents unverified outputs from modifying it. Representative systems across modalities follow this pattern by constructing memory before downstream task execution and leaving it unchanged during subsequent use (Zhang et al., 2025a; Zhang et al., 2025d; Li et al., 2026d).

Because T1 Static Memory undergoes a single formation cycle, its stored representations and their relationships remain unchanged after formation. Subsequent access and retrieval operations neither modify nor reorganize the persistent memory. Accordingly, the principal design variables in T1 Static Memory systems are the indexing and retrieval mechanisms, which may be optimized during system development or memory formation but remain fixed during inference. Fixed multimodal memories may be organized as flat sequential records (Wu et al., 2025a), vector-indexed collections (Wang et al., 2026g), hierarchical or multi-tier (Wang et al., 2026c), relational graphs (Su et al., 2026), inference-time latent states (Wu et al., 2025b), or parametric modules (Si et al., 2026). Retrieval-augmented generation provides a common access framework for vector-indexed memory, using dense cross-modal similarity, late interaction, or reranking to retrieve relevant images, document pages, video segments, audio spans, or cross-represented textual records (Yu et al., 2025c; Wang et al., 2026j; Yeo et al., 2026a; Fan et al., 2024). Metadata-aware and spatiotemporal retrieval further exploits timestamps, locations, speaker identities, and temporal boundaries to localize relevant experiences, sometimes through hierarchical coarse-to-fine search (Jiang et al., 2025b; Someki et al., 2026). Relational memories support explicit traversal over connections among objects, states, events, or interface transitions (Shen et al., 2025; Wen et al., 2024; Yu et al., 2026a). Planner- or tool-assisted retrieval decomposes a query and routes its subqueries to modality-specific indexes, temporal search tools, or structured databases (Fan et al., 2024; Someki et al., 2026). Inference-time latent-state and parametric memory systems are less common in this setting because their stored information is less explicit and interpretable (Suri et al., 2026).

In practice, static memory supports three main types of downstream applications: static knowledge bases, reusable evidence or environment stores, and reusable procedural knowledge. First, it can retain external knowledge or documentation for domain-specific applications, such as medicine, thereby improving grounding and reducing hallucinations (Xia et al., 2025). Second, it can store expensive-to-process but largely static content, such as long videos (Fan et al., 2024) or GUI environments (Wen et al., 2024), through one-time ingestion and reuse this information across subsequent tasks (Fan et al., 2024; Wen et al., 2024). Finally, static memory can preserve procedural knowledge for repeated action planning and execution (Si et al., 2026).

T1 Static Memory is most effective when the underlying information changes slowly and the stored representation will be reused across tasks. Expensive operations such as document encoding, video analysis, environment exploration, and adapter training are performed during memory formation. Because the stored content remains unchanged, each task accesses the same evidence, environment model, or procedural library. This stability supports consistent behavior.

However, this stability limits the system’s ability to respond to change. T1 Static Memory preserves both the useful information and any omissions or errors introduced during its formation. During normal task execution, it cannot incorporate corrections to stored evidence, changes in an interface or physical environment, or procedural knowledge gained from later successes and failures. The stored representation may therefore become incomplete or outdated as the source, environment, or task distribution changes.

4.2 T2: Dynamic Memory

We organize T2 Dynamic Memory methods around five fundamental memory operations.

In multimodal memory systems, the central design challenge is to perform these operations across heterogeneous representations while preserving relevant information and cross-modal relationships. Therefore, to systematically examine T2 Dynamic Memory methods, we characterize each fundamental memory operation along two multimodal-specific dimensions. The representation-handling pattern indicates whether an operation acts locally within a single representation, coordinates distinct representations while keeping them separable, or transforms information across representations through projection or fusion. The computational mechanism specifies how the resulting memory change is produced: through hard-coded heuristics, a prompted LLM, a trained functional update module, or iterative optimization under a fixed objective.

4.2.1 Memory Update

Memory update incorporates new observations, interactions, or feedback into persistent memory. Depending on the target representation, an update may append new evidence, revise an existing record, modify a structured state, integrate information into a latent state, or edit specific model parameters. The central multimodal challenge is to incorporate new information while preserving existing content in abstract representations, cross-modal alignment, and relationships among linked representations.

Source-preserving memories incorporate new information by inserting observations, replacing outdated evidence, or modifying associated metadata. Source-preserving memories commonly incorporate new information by appending raw or minimally processed multimodal observations to an evidence log or feature buffer, thereby retaining prior perceptual evidence without rewriting it (Diko et al., 2025; Feng et al., 2026). The appended evidence may include metadata, source links, or derived retrieval indices that support efficient access while preserving the original observations (Feng et al., 2026).

Modality-specific feature representations are generally updated through within-modality matching followed by feature aggregation or insertion. Each incoming feature is compared with stored representations from the same modality. If a match is identified, the corresponding memory entry is augmented, averaged, or replaced; otherwise, a new entry is created. This mechanism preserves modality-specific evidence while using shared entity identifiers to coordinate related information across modalities (Long et al., 2026).

Geometric memory representations are updated by first registering new observations within a common spatial reference frame and then associating the projected measurements or detected instances with existing spatial elements. Matched elements may be updated through feature fusion or attribute modification, whereas unmatched observations may create new map cells, objects, nodes, or relations. Invalid or obsolete elements may likewise be removed (Liu et al., 2025b; Jiang et al., 2025a; Mitash et al., 2020). These predefined association and fusion procedures provide explicit control over spatial updates and help preserve geometric consistency, but their reliability depends on fixed matching criteria and their robustness to noisy, incomplete, or conflicting observations.

Structured memory representations organize information as entities, relations, events, or procedures. New information is incorporated by identifying the relevant structural unit and then modifying its attributes, inserting or removing nodes and edges, adjusting relation weights, or updating transition statistics. Target units may be selected through similarity-based gating, while their indices and relations are maintained through moving-average aggregation, count updates, and graph-editing operators (Jiang et al., 2026a).

Textual semantic records derived from multimodal experience retain perceptual or interaction information as natural-language facts, attributes, or event descriptions. Unlike structured memories, whose updates modify explicit entities, relations, or topology, these records are maintained through content-level semantic edits. Updating them requires interpreting incoming evidence relative to existing content and selecting an operation such as addition, modification, confirmation, deletion, or no change. Prompted LLMs or VLMs are used to perform this comparison because the system must determine whether new evidence confirms, extends, contradicts, or supersedes an existing record from the multimodal experience (Feng et al., 2026; Hong et al., 2026; Ma et al., 2026; Ji et al., 2026).

Latent memory incorporates new information through trained parameterized modules that either transform an existing latent state or construct new latent memory entries. Recurrent state-transition modules integrate the preceding latent state with the current multimodal representation to produce a successor state (Qian et al., 2024). Query-based write modules instead distill incoming features, instructions, and retrieved context into compact tokens that are added to a latent memory bank (Diko et al., 2025). Thus, the former modifies an existing latent representation, whereas the latter expands the memory with newly constructed latent entries. Because both forms distribute information across latent vectors, their updates are generally difficult to inspect, attribute to specific observations, or correct selectively.

Parametric memory incorporates new information by routing it to a designated subset of model parameters and optimizing those parameters under a fixed editing objective (Liu et al., 2025a). Modality-specific adapters can restrict visual and textual edits to separate parameter subsets, limiting interference across modalities and unrelated model behavior (Seong et al., 2025). Unlike a trained functional module, which computes the next memory state through a forward pass, a parametric editor produces the updated memory through inference-time optimization. The central challenge is to internalize new information while preserving previously stored knowledge and maintaining the model’s behavior on unrelated inputs.

Hybrid memory systems coordinate updates across multiple representational forms, such as source evidence, semantic records, latent tokens, and model parameters. Common mechanisms include linking editable semantic records to immutable source evidence (Feng et al., 2026), maintaining compact latent representations alongside detailed perceptual features (Diko et al., 2025), and coupling parametric edits with corresponding external records (Seong et al., 2025). These coordinated pathways preserve the complementary properties and provenance of different representations. However, they must keep linked memory components consistent as new information is incorporated, without collapsing representational distinctions or breaking associations.

4.2.2 Memory Consolidation

Memory consolidation transforms related stored information into a less redundant, more compact representation. Unlike update, which incorporates an incoming experience, consolidation operates across accumulated memories. Its central multimodal challenge is to reduce storage and retrieval costs without removing perceptual details, temporal distinctions, modality-specific evidence, or cross-representation links that may remain important for later reasoning. Existing mechanisms primarily perform feature-space compression, semantic abstraction, structured merging, or coordinated consolidation across representations (Lian et al., 2026).

Feature-space compression reduces redundancy among stored perceptual or latent encodings. Temporally adjacent or semantically similar visual features can be selected through fixed similarity criteria and then averaged (He et al., 2024), clustered (Zhang et al., 2025b), or merged (Song et al., 2026) into a smaller set of representative tokens. More adaptive mechanisms vary the degree of compression according to temporal change or semantic salience, preserving more features around distinctive events while compressing repetitive observations more aggressively (Song et al., 2024; Fan et al., 2026a; Wang et al., 2025a). FluxMem (Xie et al., 2026b), for example, achieves competitive performance by retaining recent observations at full resolution, removing temporally redundant tokens from mid-term memory, and further consolidating spatially repetitive regions in long-term memory. These mechanisms bound memory growth, but overly aggressive compression may remove brief actions, fine visual details, or temporal transitions that cannot be recovered from the consolidated features.

Semantic abstraction can likewise consolidate a growing video history in language space. ReMA (Chen et al., 2026a) first appends timestamp-aligned descriptions for successive video clips. Whenever a new passive or query-directed observation overlaps existing temporal nodes, a prompted summarizer merges the old and new content and replaces those nodes with a unified record; non-overlapping observations are added as new entries. The same procedure operates during initial stream ingestion and recursive query answering, so the belief state continues to change while the memory-management policy remains stable. ReMA therefore exemplifies consolidation in T2 Dynamic Memory rather than policy evolution in T3 Self-Evolving Memory: task-specific inspection can refine what is stored, but task outcomes do not update the rule governing later memory operations.

Structured merging consolidates memories whose internal organization must be preserved. Graph alignment and matching procedures identify equivalent entities, actions, or procedural steps before merging their nodes, edges, alternative paths, and associated statistics (Yeo et al., 2026b). Recurring action sequences may also be extracted from multiple episodes and converted into reusable procedural structures (Jiang et al., 2026a). Unlike feature averaging, structured merging must determine which elements are genuinely equivalent and which represent meaningful alternatives; incorrect alignment can collapse distinct entities, actions, or execution paths.

Coordinated cross-representation consolidation creates a compact representation while retaining links to complementary forms of evidence. A visual sequence may be compressed into representative features and paired with a textual clue (Xiong et al., 2025), an audiovisual episode may be summarized while preserving timestamps and representative frames (Yan et al., 2026), or an entity-level fact may remain linked to the observations from which it was derived (Ma et al., 2026). This coordination allows the compact memory to support efficient retrieval while the linked source or lower-level representation preserves details omitted during abstraction. The central requirement is therefore not to preserve every representation at equal resolution, but to maintain sufficient links for the consolidated memory to be interpreted, verified, or expanded when needed.

4.2.3 Memory Revision

Memory revision corrects an existing memory when new evidence indicates that its content is inaccurate, inconsistent, incomplete, or outdated. The revision operation is narrower than update: adding an independent observation constitutes an update, whereas modifying or replacing an existing record in response to conflicting or superseding evidence constitutes revision. It also differs from forgetting, which removes information because of capacity, recency, or importance rather than because new experiences show the information to be incorrect. In multimodal systems, revision primarily involves spatial and object-state correction, semantic revision, and conflict reconciliation across linked representations.

Spatial and object-state correction revises geometric or structured environment memory when new observations indicate that an object has moved, disappeared, changed state, or been incorrectly associated with an existing entity. The mechanism registers each new observation within a common spatial reference frame and identifies the affected object or map region through geometric and semantic matching. It then replaces outdated attributes, removes invalid entities, updates object geometry, or recomputes spatial relations within the affected subgraph (Jiang et al., 2025a; Yan et al., 2025b). Restricting revision to the affected region avoids reconstructing the entire environment representation, but reliable correction requires maintaining object identity despite occlusion, viewpoint changes, and noisy observations.

Semantic revision operates on explicit facts, attributes, and event records derived from multimodal experience. Incoming evidence is compared with existing records to determine whether it confirms, extends, contradicts, or supersedes their content. Prompted LLMs or VLMs commonly perform this comparison and select among fixed operations such as modification, replacement, deletion, confirmation, or no change (Feng et al., 2026; Hong et al., 2026; Ma et al., 2026; Yeo et al., 2026b). Effective revision must correct outdated information without replacing compatible records merely because the new observation is incomplete or expressed differently.

Cross-representation conflict reconciliation is required when the same experience is retained as source evidence and as one or more derived semantic or structured records. Evidence identifiers or entity associations can be used to retrieve the observations supporting a disputed record before the record is revised. The correction is then applied to the derived representation while its links to the supporting visual, auditory, or interaction evidence are retained (Feng et al., 2026; Ma et al., 2026). This mechanism preserves provenance and supports later verification, but it must determine whether an apparent conflict reflects a genuine change, an unreliable observation, or information that is valid only under a different temporal or situational context.

4.2.4 Memory Reorganization

Memory reorganization changes the structural role, placement, or access path of information that remains in memory. It does not primarily add observations, correct content, or combine multiple memories. Instead, it determines where retained information resides and how it is connected or retrieved. Multimodal memory systems primarily perform reorganization through tier migration, graph relinking, and cross-representation reindexing or association.

Tier migration transfers retained memory records between storages with different temporal or functional roles, such as short- and long-term memory or dynamic and static memory. A fixed trigger first identifies information that meets a capacity, persistence, or stability criterion, then moves or copies it to the appropriate destination. Compact visual representations can be transferred from a bounded short-term buffer into long-term memory (Song et al., 2024), while attributes judged to be persistent can migrate from dynamic conversational state into static personalized memory (Hong et al., 2026). Here, tiers denote temporal or functional memory roles rather than hardware storage levels. Moving an unchanged artifact between accelerator memory, host memory, and storage is a physical residency decision, discussed in Sections 6.1 and 6.2; it does not by itself reorganize the logical memory.

Graph relinking changes the relations through which retained entities or events are structurally connected. When an embodied scene changes, links involving affected objects can be invalidated and spatial relations within the local subgraph recomputed without rebuilding the complete scene representation (Yan et al., 2025b). Relinking can also establish or reinforce associations between separately retained modality-specific evidence, such as face and voice representations that are determined to refer to the same persistent entity (Long et al., 2026). These mechanisms support entity continuity and cross-modal traversal, but incorrect associations can connect unrelated evidence or propagate a local error through the graph.

Cross-representation reindexing changes the retrieval representation associated with retained content without necessarily modifying the underlying content or the retrieval policy. Incoming observations can be compared with the goal- and step-level index vectors associated with an existing procedural graph, after which the matched vectors are adjusted through a fixed moving-average rule (Jiang et al., 2026a). The procedure itself remains unchanged, but its retrieval keys can be updated so that varied descriptions of the same task can reach a shared procedural memory. More generally, reindexing and association maintenance keep dense retrieval keys, modality-specific identifiers, and structured memory items aligned as experience accumulates (Zhu et al., 2026c; Yan et al., 2026). Excessive index drift or incorrect identity association, however, may make earlier access paths unreliable.

4.2.5 Memory Forgetting

Memory forgetting controls which accumulated information should remain available to the agent and at what level of detail. The forgetting operation may remove an item from an active store, reduce the detail preserved in one representation, or lower a memory’s priority during retrieval. In multimodal systems, forgetting need not affect every representation of an experience equally: detailed perceptual evidence may leave active memory while a compact event record, textual summary, or linked long-term representation remains available. The major mechanisms in the current literatures are active evidence eviction, representation-detail decay, semantic accessibility decay, and coordinated selective forgetting across linked representations.

Active evidence eviction removes source-preserving evidence or their encodings from a bounded active memory. Selection may follow insertion order such as first-in, first-out (FIFO), event-level reservoir sampling, object identity, frequency and recency statistics, or recency-weighted retention probabilities (Lin et al., 2026c; Wen et al., 2026; Wang et al., 2025c; Xiong et al., 2025). Eviction bounds active-memory cost, and removal from the active buffer does not imply that all information derived from the observation has been erased from long-term memory. They can be reorganized or migrated to another memory store before being forgotten.

Representation-detail decay retains a memory in its source modality while progressively reducing the detail or size of its representation. Interaction histories rendered as memory images can be downsampled as they recede in time, therefore reducing their resolution and visual-token allocation while a more detailed textual record remains available (Li et al., 2026h). Optical-forgetting mechanisms similarly apply progressive resolution reduction to visually represented memory pages, preserving coarse semantic structure while discarding fine visual details (Chang and Ren, 2026).

Semantic accessibility decay preserves the content of an episodic or semantic record while lowering its retrieval priority. A time-dependent decay factor can reduce a record’s retrieval priority over time, while repeated recall or contextual relevance may restore its priority (Liu et al., 2025a). When retrieval priority decreases, the memory remains stored but becomes less likely to enter the agent’s working context. This mechanism provides more gradual control than eviction, although an overly aggressive decay function may render stored experiences effectively inaccessible.

Coordinated selective forgetting operates on experiences stored in multiple representations and determines which representation should be removed, compressed, or deprioritized while other representations are retained. The goal is to exploit the complementary strengths of different representations while reducing redundant storage. Detailed visual information may be selectively discarded or compressed while a more compact long-term representation is preserved (Xiong et al., 2025; Li et al., 2026h; Wang et al., 2025c).

4.3 T3: Self-Evolving Memory

T3 Self-Evolving Memory represents a memory-centered form of inference-time learning in which experience accumulated during inference drives improvements in memory management policies. Memory therefore supports inference-time learning at two levels: accumulated experience supports the agent’s inference-time learning, while the memory system itself can also evolve through feedback-driven updates to its management policies.

The evolving policy may be realized through learned parameters (Zou et al., 2026a), symbolic rules (Bärmann et al., 2026), utility-based decision criteria (Sun et al., 2026b), or explicit procedures such as reusable retrieval strategies (Fan et al., 2026b) and executable programs (El Hattami et al., 2026).

For multimodal memory, policy evolution must account for the different roles of heterogeneous representations. A write policy determines which source observations or features should be retained and how they should be transformed into persistent records. A maintenance policy determines which representations and levels of detail should remain available as memory accumulates. A retrieval policy determines which representation or store should be searched and how evidence across representations should be coordinated. We therefore organize T3 Self-Evolving Memory methods into writing-policy, maintenance-policy, and retrieval-policy evolution.

4.3.1 Write-Policy Evolution

Write-policy evolution uses feedback to improve how raw experiences are selected and transformed into memory. Unlike a fixed write mechanism, which repeatedly applies the same admission and representation-transformation procedures, an evolving write policy adapts what information is retained and how that information is represented in response to feedback obtained during deployment.

The selection policy determines which parts of raw experience should be retained as memory records. Feedback from new interactions or completed tasks is used to adapt the policy toward retaining the most relevant parts of raw experience that are likely to support subsequent reasoning (Zou et al., 2026a). However, this adaptation introduces a key trade-off: when the task or environment changes substantially during deployment, a policy optimized for prior experience may discard important information in the new setting.

The transformation policy changes how selected multimodal evidence is converted into the target memory representation. This evolution currently operates through three main mechanisms. Representational abstraction determines the semantic granularity and level of detail preserved when raw experience is converted into an explicit record. Normalization and grounding improve temporal alignment, entity resolution, and contextual grounding across modality-specific observations before they are committed to memory. Structural construction changes how grounded evidence is organized into entities, events, attributes, relations, or graph structures. In source-linked memory, it may also adapt how derived semantic or structured records are connected to their supporting visual observations, interaction traces, timestamps, and entity identifiers. Failure traces can produce validated policy changes that repair graph construction and source-to-record mappings while preserving provenance and consistency across linked representations (Tian et al., 2026). Across these mechanisms, the central tension is to improve task relevance and representational efficiency without losing source-grounded details, temporal consistency, or cross-modal relationships needed by future tasks.

4.3.2 Maintenance-Policy Evolution

Maintenance-policy evolution uses feedback to improve how existing memory representations are retained, revised, and removed over time. The maintenance policy determines which stored representations persist and how much information is preserved during operations such as consolidation, reorganization, and forgetting. Unlike a fixed maintenance policy, an evolving policy adapts these decisions based on user feedback or accumulated evidence of downstream utility. Two main mechanisms emerge for maintenance-policy evolution.

Feedback-driven retention uses user feedback to revise which memory representations should remain available over time. When a failed memory query reveals that relevant information was previously removed, the feedback updates the retention policy so that similar information encountered in the future is retained longer (Bärmann et al., 2026). This mechanism improves future retention decisions but cannot recover information already removed unless it remains available through another linked representation. Feedback-driven representation abstraction uses feedback to determine the information to be preserved when a memory representation is being compressed. Updated relevance criteria guide subsequent grouping and summarization toward preserving information that has proved useful to the user or downstream tasks.

4.3.3 Retrieval-Policy Evolution

Retrieval-policy evolution uses feedback to improve the reusable strategies that govern memory access. Specifically, the retrieval policy determines how queries are formulated and how candidate memories are prioritized for subsequent reasoning. Existing methods primarily focus on two policy targets: search formulation and control, and memory selection and prioritization.

Search formulation and control adapt how queries are constructed and how incomplete retrieval is repaired. Successful and failed retrieval trajectories can be distilled into reusable guidance describing evidence requirements, query patterns, and corrective searches. In long-video reasoning, this guidance informs subsequent query construction over the current memory graph, while answer generation remains grounded in evidence retrieved from the current video (Fan et al., 2026b).

After searching, memory selection and prioritization determine which candidate records receive a higher probability of selection by the retrieval. In embodied memory, (Tian et al., 2026) utilizes validated rules to prioritize object records supported by direct observations, using location, recency, and links to source evidence to guide retrieval in later tasks. (Sun et al., 2026b) updates memory utility estimates from downstream feedback in clinical settings.

The evolution of memory-management policies benefits from feedback across the memory lifecycle, as errors introduced at one stage can affect later stages. Information omitted during writing or lost during maintenance cannot be recovered through improvements to the retrieval policy. A failed memory retrieval may therefore reflect earlier information loss, ineffective retrieval, or incorrect use of retrieved evidence. Distinguishing the causes of these failures motivates evaluating writing, maintenance, and retrieval jointly and targeting policy updates to the mechanisms that contribute to failure.

5 Cross-Axis Analysis

Axis I and Axis II describe different aspects of multimodal agent memory, but they interact in system design. Memory representation constrains how reliably the operations required at each management tier can be performed. T1 Static Memory requires fixed memory to be located, accessed, and reused in downstream reasoning or action. T2 Dynamic Memory further requires the correct memory units to be updated through a stable policy, verified, and synchronized across linked representations without affecting unrelated content. T3 Self-Evolving Memory adds a stronger requirement: feedback must be attributed to a memory-management decision, and the relevant writing, maintenance, or retrieval policy must evolve safely and persistently.

As the scope of persistent change expands from memory state to management policy, each tier inherits the representational bottlenecks of the preceding tier and adds another level of uncertainty. As shown in Table 1, T1 depends on preserving and accessing task-relevant information. T2 additionally depends on correctly changing and coordinating the memory state. T3 further depends on evaluating these management decisions and attributing feedback to a reusable policy component.

We therefore examine the representation requirements for T1 Static Memory, T2 Dynamic Memory, and T3 Self-Evolving Memory, followed by corpus-level patterns across the two axes.

5.1 Representation Requirements for Static Memory

T1 Static Memory imposes the simplest requirement on memory representation. Once memory has been formed, the system only needs to find and use it for downstream reasoning or action. The main representational requirement for T1 Static Memory systems is that memory formation preserves the information needed for later tasks in a form that can be effectively accessed and reused. Effective access mechanisms and retrieval indices must identify the stored context most relevant to the current query and make it available to the agent (Yu et al., 2025c; Fan et al., 2024; Yeo et al., 2026a). T1 Static Memory requires no persistent memory change: the fixed memory only needs to be accessed and reused.

5.2 Representation Requirements for Dynamic Memory

T2 Dynamic Memory updates memory through a stable management policy. The memory state changes during use, while the policy that controls these changes remains stable. Finding and using memory is no longer sufficient. The system must change the correct memory unit, preserve unrelated information, and keep dependent representations consistent. We unfold our discussion into the properties below:

Editability

We use editability to describe whether a target memory unit can be directly modified, replaced, or removed, and at what granularity the change can be applied. We use operation scope to describe all memory units, relations, indices, and behaviors affected by the operation, including both intended and unintended effects. A controlled operation should have a minimal sufficient scope: it should change the target unit and its necessary dependencies while preserving unrelated memory.

The available editing scope depends on the representation, which we categorize them as source-preserving memory and cross-represented explicit memory. Source-preserving memory allows the system to append, replace, or remove individual observations. However, a semantic error may lie in an interpretation derived from the source evidence rather than in the retained observation itself. If an agent assigns the wrong identity to a person in a video, the stored frames still record the visual input available to the agent and need not be modified. The error lies in the annotation, entity association, or derived record. Correcting it therefore requires changing the derived representation rather than the source evidence. Cross-represented explicit memory exposes facts, events, entities, attributes, and relations as separate units. These units can be changed directly (Feng et al., 2026; Yeo et al., 2026b). However, an explicit memory unit may have several dependencies. Changing an entity identity may also require changes to its attributes, event records, summaries, relations, and retrieval indices. An operation that is too narrow leaves these dependent records outdated. An operation that is too broad may change unrelated information. Latent and parametric memory is harder to edit with a controlled scope. Information may be distributed across latent states, model weights, or adapters. A state transition or optimization step can change the representation, but the system may not know which numerical components encode the target information. The update may therefore affect unrelated knowledge or behavior.

Verifiability

Verifiability asks whether the result of a memory operation can be checked for correctness. It requires a trace that connects the previous memory state, the applied operation, the supporting evidence, and the resulting state. The representation used for verification may differ from the representation being changed. A revised textual event record may need to be checked against a video segment, GUI state, audio interval, or embodied interaction trace. Explicit and source-linked memory can support this comparison through evidence identifiers and provenance links. For latent or parametric memory, direct record-level verification is usually unavailable. The system must instead use behavioral tests to check whether the intended behavior changed while unrelated behavior remained stable (Seong et al., 2025).

Reversibility

Reversibility asks whether the memory state preceding that operation can be restored. Reversal is needed when later evidence, verification, or downstream outcomes reveal that a memory operation introduced an incorrect or harmful change, such as overwriting a valid record after an erroneous entity match. Preserving the original observation alone may not be sufficient for reversing a memory operation. Regenerating a summary or structured record from that observation may not reproduce the previous state when the formation process uses a stochastic model such as an LLM. Reliable reversal therefore requires a previous version, an operation log, or an inverse operation. Explicit memory can support item-level or version-level restoration. Latent and parametric memory often relies on model checkpoints, which may also remove valid updates made after the operation being reversed.

Update propagation

Update propagation describes how the effects of a memory operation are carried from one representation to other dependent representations. This issue arises in hybrid and source-linked memory, where the same information may be stored in several forms. An operation applied to one representation may therefore require corresponding changes to the others. Cross-representation consistency describes whether these dependent representations remain aligned after the operation.

Propagation must be both complete and selective. Correcting a person’s identity may require changes to textual records, entity links, summaries, and retrieval indices, while the original image remains unchanged as source evidence. A new observation of an object’s location may update its current-state record and retrieval index while preserving earlier observations as historical evidence. Under-propagation leaves outdated or conflicting records. Over-propagation may remove valid evidence or change unrelated information. Reliable propagation, therefore, requires links between representations that refer to the same experience. These links may use timestamps, source identifiers, entity identifiers, version identifiers, or provenance relations.

The representation requirements for T2 Dynamic Memory are therefore stronger than those for T1 Static Memory. Source-preserving memory retains evidence but often lacks direct targets for semantic correction. Cross-represented explicit memory supports precise edits but requires provenance, versioning, and dependency tracking. Latent and parametric memory can be updated, but its update scope is difficult to identify and verify. Hybrid and source-linked memory supports both explicit correction and source-based verification, but it must keep its representations synchronized.

5.3 Representation Requirements for Self-Evolving Memory

The representation requirements discussed for T2 Dynamic Memory remain relevant to T3 Self-Evolving Memory. T3 introduces an additional meta-level requirement: the system must represent its memory-management policies, the decisions produced by that policy, and the feedback used to evaluate and revise it. Self-evolution therefore depends on policy addressability, policy-level feedback attribution, and persistent, validated policy revision.

Policy Addressability

For a memory-management policy to evolve, the part responsible for a recurring behavior must provide an identifiable target for revision. It may be represented explicitly as a rule, instruction, workflow, program, or retrieval specification, or implicitly in a learned module or parameter set. Explicit policy representations allow more precise revisions, whereas distributed representations make the effects of individual changes harder to isolate.

An explicit memory representation and an explicit policy representation serve different purposes. The former exposes the content being managed, whereas the latter exposes the mechanism that determines how this content is formed, maintained, and accessed. Identifiable memory items can make a state change easier to inspect, but they do not by themselves identify which part of the management policy should be revised.

Policy Evaluation and Feedback Attribution

T3 must evaluate whether outcomes reveal a recurring weakness in the memory-management policy rather than an isolated error in the memory state. This requires connecting an outcome to the memory content involved, the management decisions that shaped its availability and use, and the policy component and version that produced those decisions. A policy-evaluation record may therefore include the relevant task context, affected memory items, operations performed, retrieved content, policy version, and subsequent feedback.

This requirement extends the operation-level traceability needed in T2. In T2, a trace supports verification or reversal of a particular memory-state change. In T3, traces across interactions support policy-level attribution: determining whether repeated outcomes justify changing the reusable mechanism that selects such operations. Feedback must therefore be sufficiently informative to distinguish failures of writing, maintenance, retrieval, reasoning, and action. A task-level reward alone may reveal that the agent failed without identifying which part of memory management should change.

Axis I constrains what evidence is available for this evaluation. Explicit memory units, provenance, operation histories, and source links permit more direct inspection of management decisions. Distributed latent or parametric representations expose fewer identifiable units and therefore place greater weight on behavioral evaluation across tasks. Thus, the memory representation affects how directly the memory-management design can be evaluated.

Persistent and Validated Policy Revision

A policy revision constitutes self-evolution only when it is retained as part of the reusable memory-management process and affects subsequent memory operations. The system must therefore represent the revised policy in a form that can be stored, reapplied, and associated with later decisions. Maintaining policy versions and their evaluation histories also supports comparison, auditing, and restoration when a revision proves harmful.

Validation must assess the revised policy rather than only the memory-state change that motivated it. Correcting one record demonstrates that a particular error has been repaired, but it does not establish that the revised policy will manage future memories more effectively. Policy-level validation therefore requires subsequent, replayed, or held-out cases that test both the targeted behavior and possible regressions in other memory functions.

T3 Self-Evolving Memory requires a meta-level representation that closes the loop from policy decisions to outcomes, from outcomes to evaluation, and from evaluation to persistent and validated policy change.

5.4 Corpus-Level Patterns Across the Two Axes

Figure 8 reports the distribution of architectures across the two taxonomy axes. Each architecture is assigned to its highest supported management tier and to one representation category. Hybrid architectures form a separate category in this plot; the remaining architectures are grouped by their representation family. The incidence plots in Figures 9 and 10 separately count each representation family present within an architecture, including those in hybrid systems.

Figures 10 and 9 show that multimodal agent memory architectures concentrate in T2 Dynamic Memory across modalities and in the overall corpus, while T3 Self-Evolving Memory remains relatively rare. Of the 297 architectures, 221 (74.4%) update memory state or its management policy during operation. T2 Dynamic Memory accounts for 204 architectures (68.7% of the corpus) that update memory through stable management policies, whereas T3 Self-Evolving Memory accounts for 17 architectures (5.7%) that update their management policies based on deployment-time feedback. Stable management policies allow agents to incorporate new experience through predictable and controllable update procedures, making them the dominant approach to memory updating.

Hybrid representations help address the competing requirements of multimodal memory. Hybrid systems constitute 139 of the 297 architectures (46.8%), making them the largest representation category. Their prevalence suggests that a single representation type is often insufficient to balance representational fidelity, preservation of modality-specific detail, retrieval efficiency, and storage cost for rich multimodal experiences. Hybrid architectures are therefore the most common representational choice, but they do not dominate the landscape: many systems address narrower memory problems using the single representation type best suited to their tasks.

Compact and explicit memory artifacts are the predominant target for deployment-time policy evolution. Of the 17 T3 Self-Evolving Memory architectures, 13 are cross-represented systems, while the other 4 are hybrid systems that include an explicit representation. By contrast, to the best of our knowledge, neither the source-preserving category nor the latent or parametric category contains a T3 Self-Evolving Memory system under our definition. Explicit rules, skills, workflows, retrieval descriptions, and programs occupy a useful middle ground for deployment-time evolution: they are more compact and directly editable than detailed source observations, while remaining more inspectable and controllable than information distributed across latent states or model parameters. These properties provide identifiable targets that can be revised from feedback, validated, and reused in subsequent tasks.

The analysis above characterizes multimodal agent memory at the logical level: how experience is represented, how memory state and management policies may change, and what requirements arise from their interaction. These logical requirements must ultimately be realized by concrete runtime mechanisms. Memory access requires stored representations to be located and materialized for computation, while memory updates require dependent artifacts to remain identifiable, synchronized, and valid as state changes. Section 6 therefore examines the systems mechanisms that realize these logical memory designs under finite compute, memory, storage, and communication resources.

6 Systems Infrastructure for Multimodal Agent Memory

6.1 Scope and System Abstractions

Sections 3, 4 and 5 characterize multimodal agent memory at the logical level. Axis I determines what experience is retained and the operative representation through which it becomes available to downstream reasoning or action. Axis II determines whether the system reuses fixed memory, updates memory state through a stable policy, or evolves the memory-management policy itself. Section 5 further shows that these choices impose requirements on editability, verifiability, reversibility, and update propagation. This section examines how those logical representations, operations, and requirements are realized through physical storage and runtime execution under finite compute, memory, storage, and communication resources. These distinctions organize the infrastructure discussion without introducing an additional axis of the memory taxonomy.

Logical memory denotes the experience-derived state defined in Section 2, whose operative representation determines its classification in Section 3. When the representation exposes an independently identifiable unit, such as an image, event record, graph node, or trajectory, we call that unit a logical memory object. This does not assume that every latent state or parameter update exposes equally addressable semantic units. A physical realization comprises the concrete records, tensors, parameter modules, replicas, and auxiliary indices that store this state or make it accessible. For example, PixelRAG indexes screenshot tiles using visual embeddings but supplies the retrieved screenshots to the VLM as source evidence (Wang et al., 2026j). An index or storage format therefore supports access to memory without necessarily defining its operative representation.

Materialized execution state is the runtime form through which selected memory participates in current computation. Materialization may assemble textual or visual context, construct model-compatible features or latent tokens, restore KV states, or activate an experience-specific parameter module. These are overlapping roles rather than disjoint storage classes: a retained tensor can be both an operative memory representation and an artifact used during execution. Mem-W, for example, constructs latent tokens from retrieved trajectories and expired in-session history for subsequent agent decisions (Zhang et al., 2026b). Conversely, an ordinary request-local activation or cache does not become agent memory merely because it occupies storage or accelerates decoding. The relevant criterion is its deliberate role in retaining experience for later reasoning or action. Such reuse can occur within a trajectory: TRAM carries auxiliary recurrent states across reasoning steps while resetting them for each input (Liu et al., 2026h). Accordingly, persistence here concerns the intended reuse horizon; it does not by itself imply cross-session storage or durability after a process or machine failure.

Logical and physical lifecycles must consequently be distinguished. The tier migration discussed in Section 4.2.4 changes a memory’s temporal or functional role, whereas moving an unchanged artifact from accelerator HBM to host DRAM changes its residency. Likewise, evicting a reconstructible cache or redundant replica need not forget the underlying experience. This separation is conditional on preserving the required information: deleting the last recoverable copy or applying irreversible lossy transformations can affect what the agent can subsequently recall. Physical cache replacement alone does not establish T2 memory-state adaptation, nor does workload-responsive scheduling alone establish T3 evolution of a memory-management policy.

Our coverage therefore includes both systems that retain multimodal experience and transferable mechanisms from model serving, storage, and retrieval. InferCept’s treatment of tool-interrupted KV state and SGLang’s prefix reuse provide useful runtime mechanisms, but neither mechanism alone specifies how an agent forms or revises experience-derived memory (Abhyankar et al., 2024; Zheng et al., 2024). We use such work to explain infrastructure capabilities and their limits, rather than treating every cache optimization as a multimodal memory architecture.

Figure 11 organizes this systems view. Sections 6.2 and 6.3 examine where artifacts reside and the cost of moving or reconstructing them. Section 6.4 separates evidence relevance, execution-state compatibility, and scheduling benefit. Section 6.5 considers transformations between retained experience and active computation, while Section 6.6 examines ownership, visibility, and compatibility across updates. The remaining integration challenges are discussed in Section 8.5.

6.2 Physical Storage and Residency

Long-running agents can accumulate substantially more historical state than an individual request consumes, motivating a separation between retained memory and its active working set (Omri et al., 2026). ReKV provides a multimodal example: it retains video KV caches beyond the current processing window in RAM or on disk, then loads the caches of a query-relevant subset of frames for question answering (Di et al., 2025). Under a bounded retrieval budget, retained history can therefore grow without a proportional increase in the state loaded for each answer. Residency management determines which artifacts occupy each storage tier, how long physical copies remain there, and when to promote, demote, replicate, or evict them.

Storage tiers offer different capacity and access-cost trade-offs. Accelerator HBM accommodates state used by active inference; host DRAM can retain inactive state without occupying GPU capacity; and local SSDs extend retention beyond host memory. ReKV illustrates the latter two options for historical video KV (Di et al., 2025). Placement also extends beyond a single machine. Mooncake pools cluster-wide DRAM and SSD resources for shared KV retention, while ITME uses disaggregated, SSD-backed CXL-hybrid memory to expand prefix-cache capacity (Qin et al., 2025; Jang et al., 2026). Network-attached pools and CXL-attached expansion provide different access paths, not interchangeable latency guarantees. Moreover, raw media, retrieval embeddings, model-consumed features, and KV states derived from the same experience need not be co-located: their sizes, reuse horizons, and reconstruction costs differ.

Tool calls make retention time an explicit decision. InferCept compares Preserve, Swap, and Discard: keeping intercepted KV on the GPU, moving it to host memory, or reconstructing it after resumption (Abhyankar et al., 2024). Continuum adds a time-to-live (TTL) to GPU retention, balancing the cost of occupying capacity against avoided reload or recomputation and subsequent queueing delay (Li et al., 2025a). TTL expiration ends protection under the scheduler’s rules; it does not make the remembered evidence semantically obsolete. MORI instead ranks complete agent programs by recent relative idleness, retaining busier programs in HBM and more-idle programs in DRAM (Xia et al., 2026b). Its sticky placement changes tiers when capacity pressure requires demotion or available HBM permits promotion, avoiding migration at every tool-call boundary. These mechanisms use different signals—interruption cost, retention time, and program activity—to balance occupancy against resumption overhead.

Distributed reuse introduces a second placement dimension: which nodes retain an artifact and how many copies are maintained. Mooncake tracks cache locations and supports replication so that frequently accessed KV blocks can be available on multiple workers (Qin et al., 2025). Local replicas reduce repeated remote access but increase aggregate storage; shared pools reduce the need to duplicate every artifact but expose remote-transfer costs. For multimodal memory, these decisions can be made separately for source media and expensive derived states. They must also respect the distinction between sharing an observation and reusing its computation: identical historical evidence does not imply that its KV state is compatible with every consuming context.

The central residency trade-off is therefore not simply hot versus cold storage. It combines expected reuse time, state size, reconstruction cost, concurrent demand, and access locality. Program admission and worker selection interact with these decisions, as discussed in Section 6.4. The cost of implementing a residency change depends on the transfer and restoration mechanisms considered next.

6.3 Data Movement, Compression, and Reconstruction

A retained artifact is not necessarily ready for execution. Storage access, interconnect transfer, decompression, and layout conversion can delay its use even when lookup succeeds. These costs concern the restoration path; they are distinct from deciding where an artifact should reside or whether it is compatible with the current request.

Pipelining reduces the transfer latency exposed on the critical path. CachedAttention uses layer-wise preloading and asynchronous saving to overlap KV access with computation (Gao et al., 2024). Across nodes, Mooncake supports asynchronous transfers over RDMA-capable infrastructure and streams newly produced KV toward decode workers during prefill (Qin et al., 2025). Buffering enables overlap but consumes additional memory, and concurrent transfers can compete with inference for bandwidth. Consequently, an asynchronous interface alone does not guarantee that restoration is hidden.

Compression and partial recomputation offer complementary ways to reduce transferred bytes. CacheGen combines quantization and entropy coding with chunked streaming, overlapping transmission with codec decoding (Liu et al., 2024a). Cake concurrently recomputes from the beginning of a reusable prefix and loads cached KV from its end, adapting the division of work to available computation and I/O resources (Jin et al., 2025b). The trade-off is therefore among transferred bytes, codec work, temporary storage, and recomputation. Lossy compression additionally affects approximation quality. In particular, a compact transport bitstream need not occupy proportionally less HBM after it is decoded into inference tensors.

Multimodal and agent workloads provide additional overlap windows. MPIC loads cached image state while computing missing image caches; this overlap is distinct from its selective recomputation for contextual reuse (Zhao et al., 2025b). TokenCake exploits tool-call intervals for opportunistic asynchronous offload and predictive upload, and initiates immediate upload when a tool returns earlier than predicted (Bian et al., 2027). Workflow-informed prefetching also uses anticipated agent execution, as discussed for KVFlow in Section 6.4. These mechanisms reduce exposed restoration cost when sufficient independent work is available, but short intervals, prediction errors, and contention can leave transfers on the critical path. Completing a transfer establishes availability, not reuse validity.

6.4 Indexing, Reuse, and Scheduling

The request-time control path separates three questions: which memory is relevant, whether a retained execution artifact is compatible, and whether reusing it is worthwhile. A logical-memory index identifies accessible evidence or experience; an execution-state index identifies potentially reusable computation associated with concrete inputs and execution dependencies. A hit in the first does not imply a hit in the second, and neither hit alone determines the best execution plan.

Logical-memory access reflects the organization of retained experience. M3-Agent combines semantic similarity with timestamp-based sequential retrieval and entity-based associative retrieval over text, image, and audio memory (Long et al., 2026). DynaMem couples language-based localization with a spatial-semantic map and can retrieve the latest image associated with a selected voxel (Liu et al., 2025b). These access paths exploit temporal order, entity links, spatial structure, and modality-bearing evidence. Their results identify what the agent should inspect; they do not certify the reusability of tensors previously computed from those objects.

Execution-state indexing instead exploits computational dependencies. SGLang’s RadixAttention organizes token prefixes and their KV states in a radix tree, with image hashes representing multimodal inputs within that indexing scheme (Zheng et al., 2024). Exact-prefix reuse requires matching the relevant input prefix and computational configuration, including positions and model dependencies; an image hash by itself is insufficient. Prompt Cache enables modular reuse within an explicitly defined positional schema, whereas CacheBlend selectively recomputes cached states to restore cross-chunk contextualization (Gim et al., 2024; Yao et al., 2025a). MPIC similarly addresses image-context reuse through position handling and selective recomputation (Zhao et al., 2025b). These methods should not be treated as equivalent guarantees: schema-constrained reuse and quality-evaluated approximate reconstruction differ from exact reuse of an unchanged prefix.

Scheduling determines how compatible reuse opportunities interact with resource demand. SGLang orders requests to exploit common cached prefixes, while Preble jointly considers prefix reuse and load balancing when dispatching requests across workers (Zheng et al., 2024; Srivatsa et al., 2025). Reusing a prefix on an overloaded worker can be less attractive than recomputing it elsewhere. For retrieval-mediated requests, evidence relevance governs what enters computation, whereas cache locality concerns where that computation can run efficiently. Neither should silently substitute for the other.

Agent execution provides signals beyond individual requests. ThunderAgent treats a complete agent program as a scheduling unit, using execution phase and context footprint to pause or restore programs under memory pressure (Kang et al., 2026). Its global waiting queue permits paused programs to resume on a worker with available capacity, rather than requiring their original placement. KVFlow exploits an exposed Agent Step Graph to estimate steps-to-execution, prioritize fixed-prefix KV nodes, and prefetch upcoming state from CPU to GPU (Pan et al., 2025). Its workflow-level reuse signal depends on an exposed graph, which may include conditional branches. Together with MORI’s tier-level admission control (Xia et al., 2026b), these designs connect execution readiness to cache retention and capacity allocation.

Here, admission concerns requests, programs, or physical cache entries, rather than the semantic decision to write an observation into logical memory. A useful reuse decision must account for validation, transfer, reconstruction, queueing, and displaced work, not just the amount of cached state. Section 6.5 examines the transformations selected by this control path, and Section 6.6 examines how subsequent updates affect their eligibility for reuse.

6.5 Materialization and Write-back

Materialization and write-back implement the two directions of the logical memory lifecycle described in Sections 3 and 4. On the read path, materialization converts a selected logical memory representation into model-consumable context, features, latent states, or parameter modules for current computation. On the write path, observations, execution history, or completed trajectories are transformed into persistent state that can subsequently participate in memory formation or management. The systems concern here is therefore not what should be remembered or updated, but when these transformations execute, what artifacts they produce, and what preparation and consumption costs they impose.

Textual and source-level inputs have different preparation costs. PixelRAG renders and indexes screenshot tiles during datastore construction, then supplies retrieved pixels to a VLM. Downsampling before reading reduces visual-token demand without rebuilding the datastore or retrieval index (Wang et al., 2026j). FOLIO supplies compact semantic records and can recover linked keyframes when its relevance criterion indicates insufficient grounding (Fan et al., 2026a). Reopening images may recover details missing from an abstraction, but incurs visual encoding and additional model input. Storage compression, visual-token reduction, and downstream attention cost are therefore distinct efficiency measures.

Latent-token and KV-based paths move work across different boundaries. Mem-W uses a visual–textual encoder and a Q-Former-based compressor to produce a fixed number of soft memory tokens per selected trajectory segment (Zhang et al., 2026b). The limit bounds expansion per segment, not the total cost of encoding or selecting multiple segments. ReKV instead constructs video KV incrementally before a question arrives and binds retrieved blocks into question-answering attention (Di et al., 2025). Its retrieved tokens receive consecutive positions rather than their original video positions. This is part of a specific attention and positional scheme, not evidence that arbitrary cached tensors can be reused after context changes. Latent compression pays preparation work to reduce the model-facing input, whereas KV precomputation moves part of that work ahead of the query and retains the resulting tensors.

Parametric memory follows a further path that is not captured by context insertion alone. Frames2LoRA generates a video-specific LoRA adapter from video-conditioned representations through a trained hypernetwork; subsequent questions use the adapted model without replaying the original visual tokens (Suri et al., 2026). From a runtime perspective, the relevant artifacts are the generated parameter module and the backbone to which it is attached. The corresponding costs include adapter production, retention, loading, and activation, rather than only context length. This complements the parametric representation discussion in Section 3.3.2 without treating ordinary pretrained model weights as experience-derived agent memory.

Write-back must likewise distinguish reuse horizons. Mem-W carries expired interaction history forward as compressed latent context, while MementoGUI both compacts older working-memory entries and uses a separate Episodic Writer to construct source-linked records for later episodes (Zhang et al., 2026b; Zeng et al., 2026). In-session compaction preserves task progress; episodic write-back creates a cross-episode retrieval artifact. Both can retain experience for later use, but only episodic write-back crosses that external episodic-storage boundary. Neither description alone specifies crash durability. Producing summaries, organizing visual crops, and constructing retrieval embeddings also consume resources before later reads can benefit.

Reasoning history can become either recurrent state or procedural experience. TRAM pools completed-step activations into fast and slow recurrent states that influence later reasoning within the same input (Liu et al., 2026h). RRM instead distills completed retrieval trajectories and feedback into procedural records, excluding historical answers and video-specific facts; retrieved procedures guide subsequent evidence acquisition rather than being inserted as factual answer context (Fan et al., 2026b). These paths differ in both what persists and the model interface through which it is reused.

Write-back may produce linked artifacts asynchronously. FOLIO builds semantic records associated with retained keyframes while a short-term visual buffer supplies recent evidence; M2A maintains raw message records alongside editable semantic observations linked by evidence identifiers (Fan et al., 2026a; Feng et al., 2026). Retaining these links enables later source recovery but entails constructing and maintaining both abstractions and supporting artifacts. The cost of memory therefore spans construction, retrieval, and consumption, rather than stored size alone (Omri et al., 2026). Publication and update coordination across these artifacts are considered next.

6.6 Consistency and Coordination

Consistency and coordination provide the systems support for the T2 representation requirements identified in Section 5.2. When memory is updated, later reads may encounter different record versions, physical replicas, or derived execution states. Editability requires the system to identify the affected logical object and its physical realizations; verifiability requires provenance and version information connecting the update to supporting evidence; reversibility requires an earlier version, operation log, checkpoint, or reconstructible source; and update propagation requires dependent artifacts to be refreshed or invalidated. Coordination determines when these changes become visible and which versions or derivatives remain eligible for use. Deciding whether new evidence should revise a belief remains a memory-management question, whereas publishing the selected revision consistently across access paths is a systems question.

Update ownership and evidence identity provide a starting point. M2A gives its MemoryManager exclusive read–write access to the memory bank and links semantic observations to supporting messages (Feng et al., 2026). This identifies an application-level owner and evidence trail, not a distributed transaction protocol. When a dependency changes, the associated derivation must be refreshed or its compatibility re-established before it is used for a request requiring the revised state. Historical queries may still use an earlier version when its temporal scope and access permissions permit. This selective treatment avoids invalidating unchanged source evidence merely because an interpretation has been corrected.

Publication rules determine what readers see while writes proceed. FOLIO excludes segment records still being written and instead answers from completed memory and recent visual context (Fan et al., 2026a). Manu coordinates vector-store subscribers through timestamped logs and time-tick progress markers, allowing queries to wait for a configured freshness level (Guo et al., 2022). These mechanisms operate at different boundaries: application records in the former and indexed data with row-level ACID support in the latter. Their guarantees do not automatically compose into atomic publication across separately maintained media, semantic records, retrieval indices, and model-side states.

Shared execution state also requires lifetime and aliasing control. Mooncake tracks prefix-dependent cache blocks and their replicas and protects blocks used by ongoing requests from eviction (Qin et al., 2025). Within an inference engine, vLLM uses reference-counted block sharing and copy-on-write when a sequence extends a shared KV block (Kwon et al., 2023). These mechanisms prevent premature reclamation and interference between branches, but do not determine whether the underlying memory has become stale. Similarly, Manu’s deletion bitmaps suppress deleted vectors in search before segment-index rebuilding (Guo et al., 2022). Read exclusion, storage reclamation, and invalidation of derived representations are separate operations.

Authorization imposes another visibility boundary. Collaborative Memory separates private and shared fragments and uses provenance and a current access graph to filter retrieval as permissions change (Rezazadeh et al., 2025). This controls exposure through its retrieval interface; it does not demonstrate retroactive removal of information already materialized in another agent’s context or KV state. Authorization is consequently distinct from copy-on-write: one governs who may consume an artifact, while the other protects shared physical state against a branch’s mutations.

Compatibility can change even when source observations do not. FastFill aligns embeddings across model updates and prioritizes partial backfilling using alignment uncertainty (Jaeckle et al., 2023). This addresses retrieval features, not arbitrary model-consumed latent memory. DroidSpeak selectively recomputes layer groups when reusing KV across profiled fine-tuned model variants (Liu et al., 2026k); its scope is model-pair-specific, and substantial distribution drift can degrade quality. Compatibility therefore depends on the producer, consumer, and validated transformation, not source identity alone.

The analyzed mechanisms coordinate particular records, retrieval views, or execution blocks. Their combination exposes trade-offs among freshness, read latency, sharing, bookkeeping, and reconstruction. A retained historical observation can remain valid after a current-state record changes, while an unchanged physical cache can become incompatible with a new consumer. Establishing explicit guarantees across these boundaries is the integration problem developed in Section 8.5. Sections 6.2 and 6.6 do not introduce new memory types; they characterize the runtime conditions under which the representation and management choices in Sections 3, 4 and 5 can be served efficiently, consistently, and within finite resources.

7 Evaluation and Applications

Section 5 analyzes how representation constrains memory management, while Section 6 examines how the resulting designs are realized at runtime. This section characterizes how multimodal memory systems are evaluated and what functions memory serves in their applications. Section 7.1 defines the capabilities that memory benchmarks attempt to measure, Section 7.2 characterizes the 82 benchmark records by evaluation setting, Section 7.3 describes the functions memory serves in deployed systems, and Section 7.4 reports what those benchmarks do and do not exercise. What the field should do about the gaps recorded here is argued in Section 8.

7.1 Evaluation Dimensions for Multimodal Agent Memory

End-task accuracy alone cannot establish whether an agent has used memory: a correct output may follow from the current context or parametric knowledge, while an error may arise during memory formation, access, maintenance, or utilization. Recent benchmarks therefore increasingly evaluate not only task outcomes, but also the evidence and state transitions supporting them (Ren et al., 2026; Liu et al., 2026c). Accordingly, Table 2 characterizes each benchmark through shared memory capabilities, modality-specific requirements, and evaluation settings. The six shared dimensions capture three stages of memory use.

These dimensions are modality-independent: they characterize whether memory preserves sufficient evidence, remains valid under change, and contributes to downstream decisions.

Shared dimensions alone, however, cannot express the internal structure that must survive within each modality. Image-memory benchmarks therefore distinguish fine-grained visual-detail retention (FG), entity tracking across observations (ET), and visual state-change tracking (VC). Video benchmarks add temporal localization (TL), temporal-relation reasoning over event order and duration (TR), and multi-event evidence integration (ME) (Zhang et al., 2025b; Wang et al., 2026i). Audio and speech benchmarks require speaker tracking (SP), acoustic-cue retention (AC), and repair persistence (RP). SP tests whether a memory preserves who spoke across turns or sessions; AC tests whether it retains non-lexical evidence such as prosody, vocal style, and background sound; and RP tests whether spoken feedback or later evidence produces a persistent correction rather than a transient response (Xie et al., 2026c; Jiang et al., 2026b; Chattopadhyay et al., 2026). GUI evaluation emphasizes persistent interface state (GS), associations between actions and resulting states (AO), and reuse of experience across sessions (XR) (Liu et al., 2026d). Embodied benchmarks instead require spatial memory (SM), object-state tracking under partial observability (OS), and task-state or progress tracking (TS) (Hu et al., 2025a; Yadav et al., 2026). These dimensions expose representational failures that long-range or cross-modal labels cannot: an agent may retain an event while losing its precise time, remember an object while failing to preserve its location, or recall a GUI action without the state transition it produced.

The capability annotations should be interpreted together with the benchmark setup recorded in Table 2. The data source distinguishes real, synthetic, simulated, and hybrid settings, which differ in perceptual complexity, environmental control, and the availability of counterfactual conditions. The task field identifies whether memory supports question answering, retrieval, localization, generation, action, planning, learning, or mapping. The ground-truth field records what can be verified, including answers, supporting evidence, regions, timestamps, environment states, actions, provenance, and reliability labels. Answer-only supervision evaluates end-task correctness but provides limited evidence that the intended memory was used. By contrast, evidence, region, timestamp, and provenance annotations support more direct evaluation of memory grounding, while state and action supervision connect memory to interactive behavior. These setup dimensions therefore determine how precisely a benchmark can attribute success or failure to memory, even when two benchmarks nominally cover the same capability.

These dimensions also connect benchmark coverage to the taxonomy introduced earlier. CE and the modality-specific dimensions primarily specify the evidence, attributes, and relations that memory formation must preserve and make accessible, and therefore connect most directly to Axis I. LE, TL, ME, and XR probe the storage, access, and reuse of fixed memory and therefore align with T1 Static Memory, whereas SC, CF, VC, GS, AO, OS, and TS can expose demands for state maintenance or revision under changing observations and therefore inform evaluation of T2 Dynamic Memory. Offline reconstruction of a state history, however, does not by itself establish deployment-time memory updating. UQ and MA primarily evaluate memory utilization through evidence sufficiency and action selection. None of these labels alone establishes T3 Self-Evolving Memory, which requires deployment feedback to produce persistent changes to a reusable memory-management policy. This mapping supports the lifecycle-based coverage analysis in Section 7.4.

7.2 Benchmark Landscape Across Modalities

Table 2 organizes 82 selected benchmark resources and evaluation protocols by their evaluation setting. The selection includes dedicated memory tests and broader tasks used to evaluate systems that retain and reuse multimodal experience. Such tasks can reveal the contribution of memory even when their scores also depend on perception, reasoning, planning, or execution. Selected text-only and semantic-state protocols provide comparisons for shared memory functions; they do not establish visual or acoustic retention. Inclusion therefore indicates evaluation relevance, while the capability annotations describe what each protocol directly or indirectly measures. Beyond differences in sensory input, these groups vary in the experience that must persist, the horizon over which it is reused, and the supervision available to verify its contribution.

Legend. Modalities and applications denote areas receiving substantive discussion rather than isolated citations. Representation families and memory-management tiers are mapped to our taxonomy for consistent comparison. S: source- and modality-specific representations; E: cross-represented explicit memory; L/P: internalized latent or parametric memory; H: hybrid or source-linked composition.

7.2.1 Image Memory

Image benchmarks typically treat memory as a collection of visual episodes associated with dialogue, time, location, or personal context. Their horizons range from multi-turn image sequences to persistent personal archives, with question answering and retrieval as the dominant tasks. These resources differ in whether evaluation uses images directly or substitutes textual descriptions, as well as in the stages of the memory lifecycle they exercise.

LoCoMo (Maharana et al., 2024) evaluates long-term conversational memory through question answering, event summarization, and multimodal dialogue generation. Its conversations include shared images, but the original QA and summarization experiments replace images with captions; images are used directly only in dialogue generation.

Evidence-centered benchmarks provide the clearest tests of visual persistence. Memory-QA (Jiang et al., 2025b) evaluates recall over real-world multimodal records, requiring temporal or location cues and evidence distributed across multiple memories. DISBench (Deng et al., 2026) instead evaluates context-aware retrieval and localization within interconnected visual histories, while MemLens (Ren et al., 2026) stresses whether fine-grained visual evidence survives long-term storage and retrieval.

Personalized and conversational benchmarks extend evaluation from evidence access to memory validity. LCMP (Hong et al., 2026) and ATM-Bench (Mei et al., 2026) distribute user-specific attributes and references across multiple sources and sessions, requiring the agent to track both persistent and changing information. MemEye (Guo et al., 2026b) evaluates image-grounded histories and visual state changes, while Mem-Gallery (Bei et al., 2026) and H2HMem (Zhu et al., 2026b) additionally test conflicting records and insufficient evidence. These benchmarks test whether retrieved memories remain current and mutually consistent, although updates are usually defined by curated histories rather than by an evolving memory-management policy. MemLeak (Wang and Zhang, 2026) examines a complementary requirement: whether facts requested for deletion remain recoverable from retained text or images.

A group connects image memory to action or source verification. SMMBench (Chai et al., 2026) distributes evidence across conversations, screenshots, images, tables, and documents, and provides answer, evidence, and provenance supervision for memory-grounded decisions. EMemBench (Li et al., 2026k) similarly introduces state or action consequences, but such supervision remains uncommon. Conversational benchmarks span broader dialogue evaluation and explicit memory tasks: MMDU (Liu et al., 2024b) evaluates multi-image, multi-turn response quality, whereas MMRC (Xue et al., 2025) includes memory recall, image management, and information updates. Their aggregate scores nevertheless combine memory with other dialogue capabilities. Across these settings, supervision remains predominantly answer- and evidence-based, while provenance, state, reliability, and action labels appear only in a smaller subset. Current image evaluation therefore covers fine-grained evidence retention, entity continuity, and visual state change more thoroughly than extended interaction or adaptive memory use. Provenance and behavioral verification also remain comparatively underdeveloped.

7.2.2 Video Memory

Video-memory evaluation has expanded from offline retrieval over completed recordings to causally constrained streaming and interactive assistance, but it remains dominated by readout-style question answering. We group current benchmarks into three evaluation regimes: fixed-recording or lifelog recall, causal streaming evaluation, and future-oriented interactive assistance. Although all three require access to past video, they differ in whether evidence is available as a completed record, arrives incrementally, or must influence subsequent interaction.

Fixed-recording and lifelog benchmarks provide the clearest tests of long-range temporal access. M3-Bench (Long et al., 2026) and HippoVlog (Lin et al., 2026d) cover extended audiovisual streams, while EgoLifeQA (Yang et al., 2025a) uses week-long multimodal egocentric records to support event recall, personalized question answering, and recommendation. EgoMonth (Chen et al., 2026f) extends the horizon to month-scale experience, and EgoMem (Zuo et al., 2025) emphasizes backtracking through long videos. EgoMemReason (Wang et al., 2026l) further separates entity, event, and behavioral memory. LVOmniBench (Tao et al., 2026) broadens this setting beyond egocentric data by requiring joint reasoning over long-form audio and video. These benchmarks are strong tests of temporal localization and long-range evidence retrieval, with answers established primarily through timestamps or supporting evidence, but they usually assume that the relevant video history is already available as a completed record.

Streaming evaluation introduces causal evidence availability. VStream-QA (Zhang et al., 2025b) and StreamingBench (Lin et al., 2026b) ask questions while the video continues to arrive, whereas StreamBench (Xiong et al., 2025) adds multi-round interaction over hierarchically maintained short- and long-term records. Causal access makes these benchmarks closer to deployment, but does not by itself establish that a model maintains or revises a persistent memory state. Multi-view benchmarks add a distinct challenge: memory must align temporally related evidence across agents, cameras, or viewpoints rather than retrieve from a single chronological stream. MA-EgoQA (Kim et al., 2026) constructs a shared history from synchronized egocentric streams and requires the model to identify and combine agent-specific evidence, while EgoExoMem (Liu et al., 2026i) introduces cross-view temporal and spatial dependencies. These settings therefore test distributed perspective and evidence fusion, not merely access to a longer video, although they still treat memory chiefly as a retained record to be queried rather than as a state that must be actively maintained.

A smaller group evaluates whether video memory supports future behavior. VideoWebArena (Jang et al., 2025) tests factual and skill retention from tutorials during subsequent web tasks, scoring intermediate answers and final environment states. Because video-derived information can remain in context, these scores do not isolate memory-management quality. RIVER Bench (Shi et al., 2026b) and StreamArena (Zhang et al., 2026c) combine historical recall with live or proactive responses under real-time constraints, while StreamMemBench (Liu et al., 2026e) tests whether observed evidence, feedback, and prior interaction are reused in a later task. SLVMBench (Yang et al., 2026) further asks agents to extract procedural knowledge from recorded experience and apply it to a live stream. These tasks move beyond retrospective recall toward stateful assistance, but still mainly test whether prior experience is reused, not whether deployment feedback changes the memory-management policy itself.

Across these settings, supervision remains predominantly answer-, evidence-, and timestamp-based. Overall, video benchmarks evaluate temporal localization, long-range event relations, and multi-event integration more thoroughly than most other modalities. However, conflict resolution, unanswerable queries, and memory-grounded action remain less common in Figure 12; source-level provenance and feedback-driven policy change are also weakly evaluated. This gap contrasts with the architectural trend in Figure 8: memory systems commonly update state, yet video benchmarks rarely verify whether deployment feedback produces persistent improvements in the memory-management policy.

7.2.3 Audio and Speech Memory

Audio and speech occupy a small but differentiated part of the benchmark landscape. Existing evaluations span persistent voice-grounded conversational memory, memory over streaming audiovisual experience, retrieval from a fixed long-form recording, and short-horizon speech state carried or revised across turns. These audio-based settings exercise different parts of the memory lifecycle; text-only preference protocols provide a separate comparison.

ChatMem-Bench, introduced with VoiceMem (Xie et al., 2026c), evaluates long-horizon speech memory using 316 questions over 53 hours of dialogue. Human-curated synthetic histories support tasks covering information updates, abstention, persona, affective attribution, speaker association, and non-lexical acoustic evidence. Its published category scores support direct SC, UQ, SP, and AC coverage, but only indirect CF and RP coverage: the construction pipeline includes contradictions, while a dedicated persistent spoken-repair protocol is not specified. The paper defers detailed annotation methodology to a follow-up report. EgoMem (Yao et al., 2025b) instead combines face and voice identity with asynchronously managed facts, preferences, and social relations for later personalized dialogue.

Streaming audiovisual benchmarks broaden the horizon but make audio attribution less clean. M3-Bench, TeleEgo, and Video-SALMONN S retain or evaluate audiovisual experience over long streams, yet their video-centred tasks rarely isolate speaker, prosodic, or non-speech acoustic retention (Long et al., 2026; Yan et al., 2025a; Sun et al., 2026a). Fixed-recording benchmarks provide more controlled audio evidence: PlanRAG-Audio and LongAudioQA evaluate planned retrieval from long recordings, while ChronosAudio and AudioMarathon stress length generalization and temporal reasoning (Someki et al., 2026; Tang et al., 2026a; Luo et al., 2026b; He et al., 2025a; Gong et al., 2025). Because their memory remains fixed after formation, these settings primarily evaluate access and reuse in T1 Static Memory rather than cross-session memory updating.

Interactive ASR supplies a narrower form of state change. Agentic ASR evaluates whether spoken feedback repairs a transcription state, whereas Beyond Fresh Starts carries or rolls back acoustic and linguistic state across utterance boundaries (Jiang et al., 2026b; Chattopadhyay et al., 2026). CarMem (Kirmayr et al., 2025) provides a text-only comparison for preference extraction, retrieval, and maintenance. Its voice-assistant application does not make these experiments tests of acoustic memory. Current benchmarks therefore cover acoustic retention, long-range retrieval, speaker association, and repair separately, but rarely combine cross-session acoustic memory, conflicting spoken evidence, explicit revision, and memory-grounded action.

7.2.4 GUI Memory

GUI benchmarks occupy a compact but functionally differentiated part of the evaluation landscape. Existing settings span cross-step state retention, cross-session experience reuse, personalized and proactive interaction, and controlled diagnosis of visual memory. All reuse prior interface experience, but differ in whether memory must preserve a transient state, recover an action outcome, transfer a workflow, or retain visual evidence unavailable from text alone.

MemGUI-Bench (Liu et al., 2026d) provides the broadest direct evaluation of mobile-agent memory. Its tasks span multiple applications and use mirrored interactions to separate within-session retention from cross-session improvement, while hierarchical evaluation measures memory fidelity, learning effectiveness, and execution efficiency. MementoGUI-Bench (Zeng et al., 2026) instead formulates long-horizon GUI execution as a memory-control problem: working memory preserves task-relevant interface events and localized visual evidence, whereas episodic memory retrieves reusable trajectories. Together, these benchmarks evaluate GUI-state tracking, action–outcome associations, and experience reuse through regions, task progress, and action supervision.

Personalized and controlled settings test complementary aspects of state maintenance. KnowU-Bench (Chen et al., 2026e) hides the user profile and requires agents to infer preferences from behavioral histories, request missing information, and calibrate proactive intervention in an interactive mobile environment. Its task scores evaluate the use of history in personalized assistance, while also depending on preference inference, interaction, and execution. Memory-World (Wang et al., 2026b), introduced with STAMP, programmatically controls what transient interface information must be stored and later reused, enabling state- and action-verified evaluation in a simulated environment. DataScope (Liu et al., 2026b) combines synthetic task data with real mobile applications in 96 cross-page and cross-application workflows. Terminal success, app-level progress, and Scope-F1 verify which required operations were completed and which distractors were modified from the resulting application states; they do not provide expert action-sequence ground truth or evaluate cross-session reuse. Such benchmarks improve experimental control, although their synthetic states do not fully reproduce the variability of real applications.

DMV-Bench (Tang et al., 2026c) isolates visual memory more directly by injecting pixel-specific incidental cues into products visited across autonomous shopping sessions. Textual descriptions cannot recover the relevant cue, and successful navigation therefore depends on retaining and retrieving visual evidence with its source provenance. Current GUI evaluation consequently provides strong coverage of long-range state tracking, action outcomes, and memory-grounded execution. It remains weaker on conflicting histories, unanswerable or unsafe requests, robustness to interface change, and persistent evolution of memory-management policies. Moreover, endpoint task success alone may still fail to identify whether an error arose during memory formation, retrieval, or action selection.

7.2.5 Embodied Memory

Embodied-memory benchmarks assess whether agents can retain and use experience while navigating, mapping, and manipulating partially observable environments. Because an agent’s actions affect both the environment and the observations available later, memory must preserve spatial structure, maintain object and task states, and associate actions with their consequences. Existing benchmarks can be organized into three complementary regimes: navigation and mapping, memory-dependent manipulation, and long-horizon stateful interaction.

Navigation and mapping benchmarks primarily assess spatial persistence. FindingDory (Yadav et al., 2026) requires agents to retrieve prior visual experience for subsequent navigation and action in tasks that can be procedurally extended to longer horizons, whereas 3DMem-Bench (Hu et al., 2025a) evaluates long-term spatiotemporal question answering, scene captioning, and task execution in dynamic multi-room environments. SpaMEM (Liao et al., 2026) isolates action-conditioned revision of spatial beliefs, while HIOcc (Zhu et al., 2026a) evaluates local occupancy prediction and causal room- and building-level mapping. HIOcc measures map accuracy, online stability, and revisit consistency in static scenes; these support direct spatial-memory coverage but only partial state-change and object-state coverage, without task-progress or downstream-action evaluation. FindingDory, 3DMem-Bench, and SpaMEM additionally assess object histories or updates. Spatial regions, maps, and environment states provide supervision beyond terminal task success.

Other embodied settings assess experience reuse through planning or active perception. MEMORA-Bench (Yu et al., 2026d) pairs recall over participant-specific egocentric histories with generated plans for observed and novel goals. Its explicit information-not-available controls directly evaluate UQ, while plan-order, object-coverage, and preference scores evaluate prospective memory use; spatial, object-state, and task-progress tracking are only indirectly covered by these aggregate tasks. APRS (Tang et al., 2026b) evaluates sequential viewpoint selection and target segmentation within static panoramas, scoring search success, efficiency, and mask quality. Expert action trajectories support training rather than serving as benchmark action ground truth. SIS-Bench (Zou et al., 2026b) combines landmark and action recall with spatial reasoning and multiple-choice path planning over UAV videos. Its planning answers provide indirect MA coverage, not evidence of executed control or online task-progress tracking.

Manipulation benchmarks assess whether policies preserve action-relevant history. LIBERO-Mem (Chung et al., 2026) introduces non-Markovian tasks that require object histories beyond the current observation, with object masks, instance identities, and subgoal annotations. MIKASA-Robo (Cherepanov et al., 2026), evaluated here in its simulated RGB+joints setting, tests delayed object recall, spatial tracking, and sequential action. RMBench (Chen et al., 2026d) and ReMemBench (Shah et al., 2026) provide controlled memory-dependent manipulation tasks. RoboMME (Dai et al., 2026) evaluates temporal, spatial, object, and procedural memory in vision–language–action policies, while RoboMemArena (Lei et al., 2026) extends evaluation to longer tasks with subtask and keyframe annotations and paired real-world tasks. State- and action-level supervision provides direct evidence for memory-grounded action (MA), although endpoint success cannot distinguish memory failures from errors in perception, planning, or control.

Long-horizon and cross-session benchmarks evaluate whether memory remains valid as world state and interaction history evolve. WorldLines (Zhang et al., 2026d) pairs evidence-linked memory questions with embodied planning over semantic household traces; this protocol evaluates state maintenance without establishing raw visual or acoustic retention. WorldMemArena (Liu et al., 2026c) evaluates memory formation, maintenance, retrieval, and use across multi-session action–world interaction. RoboMME-Interference (Rathi, 2026) measures retrieval robustness as unrelated prior sessions accumulate. WorldLines and WorldMemArena therefore extend evaluation from access to T1 Static Memory toward maintenance and revision under T2 Dynamic Memory, whereas RoboMME-Interference isolates interference during access. Most current benchmarks nevertheless rely on simulated or hybrid environments, and few systematically test conflicting memories (CF), unanswerable queries (UQ), lifelong updating in physical environments, or failure attribution across memory-lifecycle stages.

7.3 Application Regimes

Application regimes are distinguished here by the function memory serves rather than by input modality. They consolidate concrete downstream tasks—including multimodal recall, long-form video and audio understanding, personalized interaction, GUI automation, embodied control, and iterative generation—according to the role memory plays in the resulting system. This view is orthogonal to Axis I: the same images, audio, documents, or interaction traces may be retained to support evidence-based decisions, continuity with a user, execution in an environment, or consistency in generated content. Each regime consequently stresses a different property of memory.

7.3.1 Recall and Evidence-Based Reasoning

This regime encompasses multimodal question answering, episodic retrieval, and long-form video or audio understanding. Although these tasks operate over different evidence—from visual details and temporally distributed events to speakers and acoustic cues—they share the requirement that later outputs remain grounded in recoverable observations (Jiang et al., 2025b; Yang et al., 2025a; Someki et al., 2026). Personal archives and long-document assistants require cross-modal retrieval and source grounding, whereas enterprise analysis additionally requires evidence to remain attributable across files, reporting periods, and tool outputs. MARDoc, for example, distils long-document interaction traces into structured evidence and reasoning memories and checks whether the recovered evidence is sufficient (Chen et al., 2026b). Financial reasoning makes the temporal and numerical demands especially explicit: FinAcumen selectively reuses successful strategies and failure-derived cautions while coordinating retrieval, visual decoding, calculation, and verification over text, tables, charts, and time-indexed records (Guo et al., 2026d). The central memory property is therefore not recall volume alone, but recoverable provenance and temporal validity, since persistence without these safeguards can retain stale or weakly grounded evidence.

7.3.2 Personalization and Cross-Session Continuity

Personalization uses memory to maintain a user-specific state across otherwise disconnected encounters. Relevant experience may include stated preferences, prior dialogue, viewed media, corrections, and recurring activity patterns. M2A combines complementary memory layers for long-term multimodal interaction, TAME maintains a state-aware personalized context without additional training, and Persode grounds a visual-journaling assistant in episodic personal history (Feng et al., 2026; Hong et al., 2026; Jin et al., 2025a). In consumer settings, the resulting continuity supports recommendations and assistance; analogous requirements arise in enterprise workflows that carry user, team, or account context across sessions. More recently, LLM-based individual simulation has attracted growing interest. The pioneering Individual Turing Test highlights the value of memory in capturing personal opinions and preferences (Guo et al., 2026c), which could be more faithful through multimodal personal histories beyond text. This regime stresses identity scoping, selective retention, and revision: useful continuity requires the agent to associate evidence with the correct subject and to update preferences without allowing obsolete or cross-user information to persist.

7.3.3 Control and Task Execution

Control applications require memory to change what an agent does, not merely what it can report. They include GUI workflow automation, spatial navigation and mapping, embodied manipulation and planning, and the transfer of procedures learned from demonstrations or prior experience (Wen et al., 2022; Lee et al., 2025; Liang et al., 2022; Mandlekar et al., 2023; Garrett et al., 2025; Hsu et al., 2025; Li et al., 2026f). MAGNET, for instance, evolves visual-semantic and procedural memories so that GUI skills transfer across changing interfaces (Sun et al., 2026d). In embodied settings, remembered layouts, object states, and task progress support navigation and manipulation under partial observability, while environment states and action outcomes provide direct evidence of memory utility (Yadav et al., 2026; Dai et al., 2026; Liu et al., 2026c; Bjorck et al., 2026; Ma et al., 2025). The same principle extends to other domains. FinAgent combines numerical, textual, and visual market evidence with diversified memory retrieval and dual-level reflection to adapt its decisions as market conditions change (Zhang et al., 2024). Enterprise automation adds operational constraints and heterogeneous artifacts: Matrix refines experience-derived memory to specialize agents for recurring business-document extraction, while MemoPhishAgent retrieves episodic reasoning trajectories to orchestrate tools for recurring and novel phishing threats (Liu et al., 2026f; Chen et al., 2026g). The defining property is action relevance under change: stored experience must improve the present decision while remaining revisable when interfaces, environments, document formats, market conditions, or threat patterns evolve.

7.3.4 Generation and World Consistency

Generative applications use memory to preserve constraints, entities, style, and causal state across iterative production. Unlike recall and control, however, this regime remains weakly represented by dedicated memory benchmarks: existing generation tasks commonly evaluate open-ended responses rather than image, video, or speech synthesis with persistent identity and world state. Application systems nevertheless demonstrate the underlying demand. MemSlides separates profile, working, and tool memories to support personalized slide creation and local revision, while agentic video systems retain executable event structures to keep generated scenes aligned with an evolving plan (Jin et al., 2026c; Cudlenco et al., 2026). Financial reporting combines this consistency requirement with evidence grounding. FinSight’s variable memory unifies financial data, tools, and intermediate analyses before synthesizing citation-aware narratives, tables, and charts (Jin et al., 2026a). Here memory is valuable because generation is not a single decoding pass: evidence is collected, transformed, assessed, visualized, and revised over multiple stages. The regime therefore stresses versioned state and recoverable dependencies, so that a change to an underlying fact or analytical result can propagate coherently through the final multimodal artifact.

7.4 Application–Evaluation Alignment and Gaps

The benchmark landscape is broader in modality than in lifecycle coverage. This pattern is partly determined by the downstream tasks used for evaluation. Image, video, and audio benchmarks predominantly use question answering, retrieval, or localization, which expose recall and grounding but provide limited evidence of subsequent memory use. GUI and embodied benchmarks instead evaluate action or planning through interface states, environment transitions, and task success (Liu et al., 2026d; Yadav et al., 2026; Liu et al., 2026c). The task therefore determines which aspects of memory can be observed, whereas the capability dimensions characterize the memory demands being evaluated.

This task dependence leads to uneven coverage across the application regimes in Section 7.3. Long-range and cross-modal evidence access supports many important applications and, correspondingly, constitutes the dominant evaluation target across Table 2. State tracking is increasingly considered in personalized, interactive, and embodied settings. Conflict resolution and evidence insufficiency remain sparsely evaluated, however, and memory-grounded action is concentrated in GUI and embodied benchmarks. Consequently, retrieval-centric applications are better represented in current benchmarks than applications that require persistent state maintenance, conflict handling, or memory-grounded action. Current evaluation therefore establishes whether experience can be recovered more consistently than whether it remains valid, sufficient, and actionable.

Modality-specific evaluation is strongest for structure native to each input stream. Image benchmarks emphasize visual details, entities, and state changes (Jiang et al., 2025b; Ren et al., 2026); video benchmarks emphasize temporal localization and multi-event reasoning (Yang et al., 2025a; Liu et al., 2026e); audio and speech benchmarks examine long-form retrieval, speaker- and acoustic-level retention, and stateful interaction (Someki et al., 2026; Yao et al., 2025b). GUI and embodied benchmarks instead focus on interface, spatial, object, and task states (Yadav et al., 2026; Liu et al., 2026c). Audio evaluation nevertheless remains predominantly recall- or transcription-oriented, with limited coverage of persistent correction, personalized response, speaker conflict, abstention, and memory-grounded action.

Lifecycle coverage remains concentrated on persistence and retrieval in T1 Static Memory; state-update and experience-reuse tasks probe T2 Dynamic Memory, but usually under stable policies. A Learn label therefore does not establish T3 Self-Evolving Memory unless deployment feedback produces persistent changes to a reusable memory-management policy (Zeng et al., 2026; Liu et al., 2026e). Thus, although current benchmarks cover important application-facing retrieval capabilities, their lifecycle coverage remains incomplete. The main gaps lie in conflict and uncertainty handling, source provenance, end-to-end memory attribution, policy adaptation, and memory-grounded visual generation.

8 Challenges and Outlook

The preceding sections show that long-term multimodal agent memory depends on three coupled concerns: what experience is retained and how it is represented, how that memory is maintained as new evidence arrives, and how its logical operations are realized under finite system resources. Their interaction becomes more consequential over long trajectories, changing environments, and shared deployments. We therefore distinguish challenges in formation and representation, challenges in memory management, and tensions between the two, followed by deployment-oriented evaluation and open infrastructure problems. The infrastructure discussion builds on Section 6 and focuses on how the assumptions and guarantees of the analyzed mechanisms can be integrated under sustained deployment.

8.1 Multimodal Memory Formation and Representation

Memory formation determines not only what information is retained, but also what can still be recovered after the original experience has been transformed, what can later be located and changed inside the resulting representation, and how far any of this has been worked out beyond the modalities that dominate current practice. Current systems use a wide range of source, explicit, latent, and parametric representations, yet important questions remain about what each of them should preserve, what structure each should expose, and whether the answers established for visual experience carry over to other modalities.

8.1.1 Recovering Supporting Evidence

Abstraction should move from lossy compression toward recoverable representation. As previous sections show, modality-specific encodings, textual, structured, latent, and parametric memories can all remove information that remains available in source-level evidence. Source-linked designs introduced in Section 3.4 partly address this problem by preserving links from derived memories back to supporting observations. The remaining question is what information should be preserved during formation so that this recoverability is reliable.

Existing systems use different combinations of timestamps (Wang et al., 2025a), source identifiers (Feng et al., 2026), entity links (Long et al., 2026), spatial references (Liu et al., 2025b), and pointers to supporting observations (Lin et al., 2026d). However, a substantial portion of the analyzed architectures do not retain a usable route back to the evidence from which their memories were formed. Among those that do, the route is often simply that the source was never abstracted away, rather than a link deliberately carried by a derived memory, and where such links do exist they differ in what they record. Provenance is therefore seldom a designed property of the representation. A useful direction is to define a minimal provenance bundle for formed memories, containing enough information to identify where a memory came from, how it was produced, and what evidence supports it. The required fields may differ across modalities and representation types, but provenance should be treated as part of the representation itself rather than as optional metadata.

Recoverable representation does not require every source observation to be retained indefinitely. Instead, memory formation should make explicit which information may be discarded and which supporting evidence must remain reachable for later inspection or reinterpretation. This would make abstraction more controlled without requiring all downstream memories to preserve the full original experience. What remains open is how to make this decision. Current systems use different rules (Shen et al., 2026; Liu et al., 2026a), but a better criterion should balance retention cost against the need to verify or correct information later.

8.1.2 Finding and Updating Specific Memory

Formed memory should expose the units that memory management acts on. While recoverability asks whether a memory can be traced back to its supporting evidence, semantic addressability asks whether specific content can be identified and changed within the representation itself. Explicit textual and structured records provide such units by construction, which is why they support revision and conflict resolution most directly (Bo et al., 2026). Semantic addressability remains rare outside these representations. The difficulty is sharpest in latent and parametric memory, which is compact and integrates directly with the model’s computation but does not expose content as separately identifiable units by construction. An update to such memory may therefore reach information it was not intended to change, and the system may not be able to identify which part of memory is responsible for a particular behaviour.

This limitation appears in the corpus as a pattern rather than only as an argument. Section 5.4 shows that every architecture evolving a memory-management policy at deployment time maintains an explicit cross-represented component, and that none whose memory is only latent or only parametric does so. The absence is not a matter of these systems being static: most latent-only architectures do update their memory state under a fixed policy. What they lack is not dynamism, but clearly identifiable units over which memory-management decisions can be attributed, inspected, and revised. Source-preserving memory shows the same absence despite being addressable in another sense, and the contrast is instructive: its units can be identified perceptually, by time or position, yet still not at the semantic level at which a write or retention decision is made. The obstacle is therefore not distributed representation as such, but the absence of units that correspond to what a policy decides about.

Parametric memory can already be divided into separable modules associated with different experiences, modalities, or tasks, as in adapter banks that hold per-task procedures (Si et al., 2026), per-video parameters (Suri et al., 2026), or modality-specific edits (Seong et al., 2025). This makes parts of a distributed representation easier to locate and manipulate, but it has not so far produced semantic addressability: among the analyzed systems that organize memory this way, the few that evolve a management policy during deployment do so through an explicit cross-represented component rather than through the parameter store itself. Structured latent memory that binds persistent state to identifiable slots or entities (Chung et al., 2026) would be a further step in the same direction. What a fuller answer requires is nonetheless clear: a distributed memory is addressable in the sense that management needs only when the same unit can be identified, revised on its own, and afterwards verified to have changed while the rest of memory did not. The open question is whether that combination can be achieved without losing the compactness and direct model integration that motivate latent and parametric memory in the first place.

8.2 Multimodal Memory Management

Section 4 describes how memory changes after formation, from fixed retrieval to persistent maintenance and feedback-driven policy evolution. The challenges collected here concern two questions that arise once memory is allowed to change over time: how the system should keep its memory up to date as new experience arrives, and how the policies that manage memory can themselves improve from feedback.

8.2.1 Maintaining Up-to-Date Memory

Memory maintenance should move from recording change toward keeping track of which state is still valid. New observations can be appended, existing memories can be revised, and repeated evidence can be merged, but these operations do not by themselves tell the system whether an older state still holds, has been replaced, or should be kept only as history. This becomes especially important in multimodal settings, where evidence about the same object, event, or user may arrive at different times and through different modalities. The difficulty here concerns observations of the same state that arrive separately, not disagreement among the stored representations derived from them.

A timestamp records when an observation was made, not how long what it asserts remains true. In multimodal settings the two can come apart in a particular way. A visual observation may show where an object was seen, while a later instruction may report that it has moved. A new image may correct an earlier description, while the older observation remains useful as historical evidence. Where staleness is handled at all, it is often handled by removing or replacing the outdated record (Yeo et al., 2026b), or by checking a cached state against a fresh observation at the moment of use (Liu et al., 2025c), rather than by representing how long a belief remains valid.

A promising direction is to explicitly model this status within the memory management process. A memory may be marked as current, historical, superseded, uncertain, or still valid alongside another state. Such decisions may depend on the source and reliability of new evidence rather than on recency alone. They may also need to distinguish when an observation is received from when the event it describes actually happened. What remains open is how an agent should decide when new evidence replaces an older state, when both should coexist, and when uncertainty should be preserved rather than resolved.

8.2.2 Self-Improving Memory Management

Keeping memory up to date assumes that the rules for doing so are already good enough. The next challenge is to improve those rules from experience. As Section 4.3 describes, feedback-driven memory management changes how memories are written, maintained, or retrieved during deployment. This is different from optimizing a controller during training and then freezing it: once deployment begins, a frozen controller continues to apply a stable policy even as the memory contents change.

Current policy evolution is concentrated on retrieval, while changes to write and maintenance policies remain much less common. One likely reason is that retrieval receives relatively direct feedback: the system can often observe whether retrieved memory helped the next answer or action. The effect of an earlier write, merge, revision, or forgetting decision may appear much later, and in a multimodal agent it may not appear in the same modality at all. A memory written from a visual observation may be revealed as wrong only by a later instruction, by an observation in another modality, or by the failure of an action taken on its basis. Credit therefore has to travel across both time and modality before it can reach the decision that should change.

A useful direction is to connect later success and failure back to the memory decision that most likely caused it. This would allow feedback-driven improvement to extend beyond retrieval without treating every downstream error as evidence that the entire memory policy should change. What remains open is how to assign delayed and cross-modal feedback to earlier memory decisions while separating their effects from the many other operations that occur in between.

8.3 Representation–Management Tensions in Multimodal Memory

Section 5 shows that representation and memory management cannot be designed independently. What a memory preserves limits what later operations can inspect, change, verify, or remove. At the same time, the way memory is managed determines which parts of a representation remain useful over long interactions. The challenges here arise from this coupling. They are not problems of representation or management alone, but cases in which one limits what the other can do.

8.3.1 Editable Representations without Loss of Detail

Multimodal memory should preserve perceptual detail without making later correction difficult. Source observations and modality-specific representations retain information that may disappear after an image, video, trajectory, or other observation is converted into text or structure. Yet the same abstraction that loses detail often makes memory easier to revise. A textual fact or graph entry can be changed directly, while correcting one semantic detail inside a long video, a distributed latent state, or a parameter update is much less straightforward. What a representation preserves therefore limits how precisely memory can later be maintained, both in how narrowly an update can be scoped and in how directly its result can be checked.

Hybrid and source-linked designs reduce this tension by keeping an abstract memory together with a route back to supporting evidence (Wang et al., 2026e; Fan et al., 2026a). But coexistence alone does not settle the problem. An abstract record can become outdated while the source observation remains valid as historical evidence. A new observation may also contradict the current abstraction without making the older evidence useless. The system must then decide which layer should guide current behavior and when the source should be reopened rather than trusting the abstraction.

A useful direction is to combine editable memory state with recoverable modality-specific evidence and an explicit rule for moving between them. Routine updates can operate on compact textual or structured state, while uncertain or high-impact changes can return to the supporting image, video, audio segment, or trajectory for verification. What remains open is how much source detail must stay reachable for later correction without making long-term memory expensive to maintain.

8.3.2 Selective Evidence Retention for Verifiable Memory

Verifiable memory should not require every past observation to remain available forever. Retained source evidence, provenance, version history, and checkpoints make memory changes easier to inspect and reverse. In multimodal memory, however, the cost of keeping this support depends strongly on what form the evidence takes. A short textual record, selected image frames, a long video, an audio segment, a 3D trajectory, and a latent or parametric checkpoint impose very different storage and processing costs. The representation therefore limits how much evidence a management system can realistically keep for later verification.

Current systems usually make this choice implicitly through what their memory architecture happens to retain, rather than through an explicit policy for verification. Some retain source observations (Wang et al., 2026j), some retain only derived records (Gao et al., 2025), and others keep checkpoints or structured state (Gu et al., 2024). What is often missing is a rule for which evidence deserves long-term retention, which evidence can be compressed, and which updates are important enough to justify preserving an earlier version.

A useful direction is selective verification. Lightweight provenance can remain available for most memories, while expensive source evidence or checkpoints are kept only when later inspection is likely to matter. High-impact revisions, uncertain memories, or information supported by conflicting modalities may justify stronger evidence retention than routine observations. The open problem is how to predict this need before the future correction or dispute actually occurs. A system that keeps too little loses the ability to verify its memory, while one that keeps everything gives up much of the benefit of abstraction.

8.3.3 Policy Adaptation with Preserved Evidence Grounding

Self-improving memory management should not silently change what evidence the agent relies on. As Section 4.3 describes, a policy may evolve how memory is written, maintained, or retrieved. In a multimodal system, such a change can also alter the path through which information reaches the agent. A retrieval policy may gradually favor textual summaries over visual evidence, a write policy may compress one modality more aggressively, or a maintenance policy may preserve some forms of memory while discarding others. Management evolution can therefore change not only performance, but also which representations continue to support that performance.

This creates a stability problem that task accuracy alone may not reveal. A revised policy can maintain or even improve short-term answers while relying less on the modality that originally grounded them. For example, a system may answer correctly from a cached textual summary while rarely reopening the visual evidence from which that summary was formed. If the summary later becomes wrong, the loss of that evidence path may only become visible much later. Current policy-evolution methods rarely evaluate whether adaptation changes the balance of evidence used across representations.

A useful direction is to evaluate policy changes together with the evidence paths they alter. Before a new policy is accepted, the system should check not only whether downstream performance improves, but also whether access to supporting modalities and representations has been weakened. This may require tracking which memory paths support important decisions and comparing that use before and after adaptation. What remains open is how much change in evidence use is healthy adaptation and when it becomes a loss of grounding that should block or reverse the policy update.

8.3.4 Sustaining Memory without Losing the Ability to Forget

Long-term memory should remain bounded without losing control over what can be removed. The difficulty depends on representation. Retained images, videos, audio, and trajectories grow with experience; structured memory grows with entities, events, and relations; a bank of adapters can grow with tasks or episodes; and a fixed-size latent state faces a different problem, because new information must compete with what is already encoded. Representation therefore determines what grows, while memory management determines how that growth can be contained.

Existing systems use operations such as consolidation (Rasheed and Kabtoul, 2026), compression (Li et al., 2026a), eviction (Lin et al., 2026c), decay (Sun et al., 2026b), and forgetting (Liu et al., 2025a), but sustained memory growth under a fixed budget remains weakly addressed. A useful direction is representation-aware budgeting. The system should not apply one retention rule to every memory form. Source evidence may be sampled or compressed, structured records may be merged, latent capacity may need to be reallocated, and parametric modules may need consolidation or removal. The hard part is deciding what can be reduced without destroying information whose value will only become clear later.

Capacity is only one reason to remove memory. Some information may need to disappear because it is outdated, corrected, or explicitly deleted. In a multimodal system, that information may already have spread from a source observation into a caption, embedding, graph node, summary, latent state, or parameter update. Removing one copy therefore does not establish that the information is gone. This also creates a direct tension with Section 8.3.2: verification benefits from retaining evidence and version history, while deletion may require the same evidence and history to disappear.

A useful direction is to specify forgetting as a cross-representation outcome rather than as the removal of a single record. Provenance and derivation links can identify the captions, embeddings, structured entries, summaries, or parameter modules that may carry the targeted information, but locating a derivative does not guarantee that the information can be selectively erased from it. The memory policy must determine which information should become unavailable, which unrelated knowledge should remain, and how the outcome can be verified. This also makes the tension with reversibility explicit: retaining an earlier version may support correction but conflict with a removal requirement. The open representational problem is especially difficult for latent states and shared parameters, where the targeted information is not separately addressable. Section 6.6 examines relevant coordination mechanisms, while Section 8.5 addresses how a declared removal reaches replicas and active derivatives and when that operation can be considered complete.

8.4 Deployment-Oriented Memory Evaluation

Evaluation determines which forms of progress become visible. Current benchmarks establish increasingly broad coverage across modalities and tasks, yet long-term memory is ultimately valuable as a process that preserves, updates, accesses, and applies experience under continuing interaction. Future evaluation should expose this process, stress it over sustained horizons and bounded resources, and test it under the operating conditions faced by deployed agents.

8.4.1 Lifecycle-Aware Evaluation

Multimodal memory evaluation should move from outcome-level scoring toward stage-resolved diagnosis. Many existing benchmarks summarize performance with final answer accuracy or task success, including long-video reasoning and interactive visual-recall settings (Long et al., 2026; Tang et al., 2026c). These scores establish whether the complete system succeeded, while leaving the origin of failure unresolved. Recent benchmarks demonstrate the value of richer supervision through evidence annotations, update points, retrieval judgments, feedback-reuse tests, and memory-specific execution metrics (Liu et al., 2026c; Liu et al., 2026e; Chai et al., 2026; Liu et al., 2026d). A lifecycle-aware protocol can organize such signals around four observable stages.

Formation.

Evaluation should determine whether the agent identifies the parts of an experience that will matter later and forms faithful memory from them. Useful process metrics include coverage of gold memory points, correctness and supportedness of written records, rates of hallucinated or irrelevant entries, preservation of source links, and retention of task-relevant visual, acoustic, spatial, and temporal detail. These signals evaluate selective writing and representation quality before later retrieval can conceal their failures.

Maintenance.

Evaluation should track whether persistent memory remains valid as new evidence arrives. Intermediate metrics can measure successful incorporation of current state, rejection or supersession of stale records, conflict resolution, faithfulness of consolidation, and consistency across source, textual, structured, latent, or parametric representations. Versioned memory snapshots and annotated state transitions can further reveal when an initially correct memory becomes corrupted during update, compression, or reorganization.

Access.

Evaluation should assess whether the evidence required by the current decision is recovered from the appropriate memory path. Evidence recall, precision, ranking quality, temporal or spatial localization, cross-modal coverage, and retrieval latency expose distinct access failures. Comparing ordinary retrieval with oracle evidence also separates limitations of memory access from those of the downstream reasoner.

Utilization.

Evaluation should verify that accessed memory actually guides the response, plan, or action. Evidence-conditioned correctness, source attribution, adherence to the current state under conflict, action–outcome consistency, and recovery after feedback provide process-level measures of memory use. Counterfactual removal or replacement of retrieved evidence can further test whether an apparently correct output causally depends on memory. For generative agents, utilization should additionally test whether retrieved constraints preserve entity, style, temporal state, and world consistency, and whether subsequent memory updates propagate coherently through later revisions of the generated artifact.

Few benchmarks expose all four stages, and complete lifecycle supervision is especially scarce for native multimodal evidence. Establishing shared trajectory annotations and stage-level metrics would make memory failures localizable, allow process rewards to target the operation that failed, and support meaningful comparison among architectures whose final accuracy may be similar for very different reasons.

8.4.2 Long-Horizon and Resource-Aware Evaluation

Long-horizon evaluation should measure sustained memory quality under the resource pressure created by accumulating multimodal experience. The field has begun to extend temporal scale: M3-Bench evaluates videos averaging roughly half an hour (Long et al., 2026), StreamArena reaches hour-scale causal streams (Zhang et al., 2026c), EgoLife records a week of shared daily experience (Yang et al., 2025a), and EgoMonth spans 20–120 days of egocentric observations (Chen et al., 2026f). This progression brings evaluation closer to the timescale of persistent assistants. Continuous deployment protocols with interleaved writing, maintenance, querying, and feedback remain much less developed; even long multi-session histories are often frozen and queried offline (Ren et al., 2026).

Resource pressure.

As the observation stream grows, a memory system consumes persistent storage, write-time computation, retrieval bandwidth, model context, and response time. Evaluation should therefore impose explicit budgets and report memory size and growth rate, ingestion throughput and lag, retrieval latency, token or compute cost per unit of experience, and the fraction of source evidence retained at each fidelity level. Early studies already expose large differences in write–read latency and execution cost across memory designs (Liu et al., 2026c; Liu et al., 2026d); standardized budgets would turn these isolated measurements into comparable operating constraints. Systems accounting should also distinguish the size of logical memory from the total physical footprint of its replicas, indices, cached execution states, and temporary buffers. In addition to end-to-end latency and memory quality, experiments should report peak accelerator occupancy, transferred bytes, and the costs of materialization, decompression, and recomputation. Warm-cache and cold-cache conditions should be identified separately, and construction or background write-back should not be treated as free when their costs are shifted outside the query interval. These measurements connect the quality–resource frontier to the mechanisms in Section 6 without reducing memory quality to cache hit rate.

Longitudinal degradation.

A bounded memory must continually decide what to preserve, consolidate, or forget. Its evaluation should trace accuracy, evidence recall, update correctness, grounding, and action quality as a function of elapsed time, accumulated experience, and available memory. Context-length analyses and time-stratified forgetting curves provide useful starting points (Ren et al., 2026; Shi et al., 2026b), while deployment-oriented protocols should also reveal delayed corruption, retrieval slowdown, representation drift, and the compounding effect of early writing errors on later decisions. The central comparison is the quality–resource frontier: how reliably a system preserves useful, verifiable memory as its operating horizon grows under a fixed budget.

8.4.3 Real-World Deployment

Deployment evaluation should reproduce the conditions under which multimodal memory must operate continuously. Real environments combine strict response deadlines, imperfect sensing, asynchronous feedback, open-ended tasks, and changes in users and worlds. Benchmarks increasingly incorporate real egocentric recordings, interactive interfaces, and evolving state (Yang et al., 2025a; Chen et al., 2026f; Liu et al., 2026d; Liu et al., 2026c), while most still isolate these factors within curated data, controlled state transitions, or resettable environments.

Real-time operation.

An agent that recalls the correct evidence after the decision window has passed cannot provide effective assistance. Evaluation should measure end-to-end response latency, deadline-miss rate, proactive trigger timing, and the delay between observation and query-visible memory availability. RIVER couples memory accuracy with temporal distance and response timing, while StreamArena measures query-to-answer latency across routing, recall, external search, and inference (Shi et al., 2026b; Zhang et al., 2026c). Extending these protocols to continuous memory writing and maintenance would expose whether background processing can keep pace with the incoming multimodal stream. For deployments requiring failure recovery, the time to a recoverable commit and behavior after interrupted writes should be measured separately under an explicit failure model; query visibility alone does not establish durability.

Perceptual and environmental robustness.

Deployment streams contain motion blur, occlusion, poor lighting, overlapping speech, sensor dropout, uncertain timestamps, and incomplete or contradictory feedback. Systems also encounter devices, interfaces, locations, speakers, and object appearances outside the distributions represented in curated benchmarks. Evaluation should introduce controlled corruptions alongside naturally noisy trajectories, vary which modalities remain available, and test whether memory preserves uncertainty, requests clarification, or abstains when the evidence is insufficient. Robustness should be measured across both memory formation and later reuse, since an unnoticed perceptual error can persist long after the original observation has disappeared.

Continual user and world change.

Long-running agents must follow changing preferences, identities, routines, goals, interface states, and physical environments. Evaluation should present irregular state changes, delayed corrections, cross-modal disagreement, and multiple users whose memories must remain separated. The resulting metrics should capture adaptation delay, stale-memory use, recovery after correction, cross-user contamination, and retention of valid historical evidence. Such protocols would connect benchmark success to the defining requirement of deployed memory: maintaining reliable continuity while the world that memory describes continues to change.

8.5 Memory Infrastructure and Efficiency

Section 6 examines mechanisms for residency, restoration, reuse, materialization, and coordination. The open systems problem is how their assumptions and guarantees can compose over the lifetime of changing multimodal memory under bounded resources. The directions below concern runtime decisions and guarantees. They complement the representational and management requirements in Section 8.3 and the evaluation protocols in Section 8.4.

8.5.1 Workflow-Responsive Residency and Data Movement

Program- and workflow-aware systems already exploit execution phase, relative idleness, and anticipated reuse (Kang et al., 2026; Xia et al., 2026b; Pan et al., 2025). The next question is how placement and transfer execution should respond when those signals change while data is in flight. TokenCake, for example, already triggers an immediate upload when a tool returns earlier than predicted (Bian et al., 2027). A broader runtime contract should address branch cancellation, overlapping readers, and reprioritization across storage tiers: which transfers can be interrupted, which must drain, when staging buffers can be reused, and whether completed chunks remain useful. Research should connect transfer granularity and queue depth to bounded wasted traffic and resumption delay, rather than treating cancellation as an instantaneous reversal of offload.

Compression should also be optimized over repeated use rather than a single transfer. Building on the transport and compute–load mechanisms in Section 6.3, future designs should jointly account for encoding, decompression, temporary storage, layout conversion, and the number of expected restorations. Incremental restoration may reduce startup delay, but finer transfer units add metadata and scheduling overhead. Quality analysis should distinguish repeatedly decoding one unchanged compressed artifact from repeatedly re-encoding or approximately reconstructing its outputs; only the latter introduces an additional chain of potentially accumulating approximation. The objective is a lifetime cost–quality trade-off, not network-byte reduction alone.

8.5.2 Dependency-Aware Reuse and Scheduling

The dependency assumptions of prefix caching and selective recomputation in Section 6.4 become harder to track when memory is revised, reordered, cropped, or materialized through a different model interface. Future work should investigate compact dependency descriptions that connect execution artifacts to their source versions and relevant transformations, such as preprocessing, encoder or adapter versions, prompt composition, positions, and attention configuration. These descriptions should distinguish exact reuse from empirically validated approximation and permit selective invalidation or reconstruction. An unresolved trade-off is granularity: coarse dependencies cause unnecessary recomputation, while fine-grained tracking can cost more than the computation it saves.

Scheduling should use this information without collapsing relevance into cache locality. Preble establishes the value of balancing prefix reuse with worker load (Srivatsa et al., 2025); multimodal memory introduces additional choices among available evidence representations and materialization paths. Research should optimize worker selection, cache admission, prefetching, and reconstruction jointly, subject to the evidence, freshness, and quality requirements of the task. Reuse value must include validation overhead, expected remaining uses before invalidation, and interference with other programs. A high hit rate is not a sufficient objective when the hits are expensive to restore, likely to be invalidated before further use, or available only for evidence that is less appropriate to the current decision.

8.5.3 Coordinating Materialization and Write-back

Read-side acceleration can shift cost into memory construction rather than eliminate it. Existing workload characterization separates construction, retrieval, and generation costs, while FOLIO illustrates asynchronous construction with a recent-evidence path (Omri et al., 2026; Fan et al., 2026a). Future systems should jointly schedule foreground inference and background encoding, summarization, indexing, or adapter production. The key decisions are when to materialize expensive artifacts, when to defer work until demand is known, and how to prevent persistent write backlogs from making new experience unavailable. These decisions should consider expected reuse and invalidation, not just instantaneous GPU utilization.

Such orchestration also needs representation-specific fallbacks. When a compact record is not ready, a runtime may expose retained source evidence; when a model-specific artifact is incompatible, it may reconstruct a compatible form from an authorized source. These alternatives are not semantically interchangeable and should operate within the memory policy’s requirements, rather than silently replacing evidence to meet a latency target. Research should determine when eager materialization, lazy reconstruction, or incremental publication provides the best balance of responsiveness, groundedness, and recoverability. Systems evaluation in Section 8.4 should account for the costs incurred on both sides of this read–write boundary.

8.5.4 Cross-Layer Consistency, Recovery, and Revocation

The scoped guarantees in Section 6.6 motivate explicit publication and recovery contracts across artifacts. Future work should specify whether a read requires a consistent snapshot, read-your-writes visibility, or bounded staleness, and how these requirements span source media, semantic records, retrieval indices, and derived execution state. The coordination granularity need not be the entire memory bank; a versioned set of dependent artifacts may be sufficient. The challenge is to expose a coherent read path while expensive derivatives are being rebuilt, without globally blocking unrelated requests.

Durability should be specified separately from logical retention and query visibility. For deployments that require recovery from failures, research should identify which source records, versions, and transformation results must be committed, which caches can be reconstructed, and how interrupted write-back can resume without duplicating or exposing incomplete records. Preserving source media alone may not reproduce a prior stochastic summary or parameter update. The required guarantee must therefore distinguish recovery of the exact previous artifact from reconstruction of a newly generated alternative.

Deletion and permission changes extend coordination to already materialized state. Whereas Section 8.3.4 asks what must be forgotten and how to verify that outcome, the infrastructure question is how a declared removal propagates through replicas, indices, summaries, contexts, caches, and retained versions. Future work should define a completion boundary and distinguish preventing new retrieval from retiring active derivatives and reclaiming storage. Collaborative Memory’s retrieval-time access control provides one boundary, not a demonstrated solution for all downstream artifacts (Rezazadeh et al., 2025). Dependency metadata can identify retirement targets, but cannot by itself establish semantic erasure from a distributed latent state or shared model parameters.

MemOS proposes common abstractions for heterogeneous memory resources, while MemoryOS organizes hierarchical memory for agent interaction (Li et al., 2025b; Kang et al., 2025). Building on such efforts, the objective should be interoperable contracts for identity, version, provenance, compatibility, and lifetime—not a universal representation that erases modality-specific semantics. The central open problem is to make their guarantees compose under finite resources and partial failure while preserving the evidence needed for future reasoning and correction.

9 Conclusion

The central question of multimodal agent memory is what evidence to retain from heterogeneous multimodal experience, how to represent it for subsequent reasoning and action, and how to keep it valid over time. Our framework characterizes multimodal agent memory along two axes: Formation and Representation, and Memory Management. Axis I classifies the operative representation through which retained experience becomes available to downstream computation; Axis II distinguishes T1 Static Memory, T2 Dynamic Memory and T3 Self-Evolving Memory. Joint analysis of the two axes shows that representation constrains the accessibility, revisability, verifiability, and recoverability of memory. The coexistence of multiple representations further requires maintaining provenance links, temporal validity, and cross-representation consistency, and updating memory state is not equivalent to evolving the management policy. Current systems remain dominated by fixed memory and memory updating through stable policies. Online policy evolution, bounded lifelong memory, governed forgetting, and lifecycle-aware evaluation remain underdeveloped. At the systems level, current infrastructure lacks unified mechanisms for efficiently and consistently storing, restoring, and reusing memory under finite resource constraints; while existing benchmarks remain insufficient to establish whether memories remain valid, sufficient, and actionable throughout the full memory lifecycle. Together, these challenges point toward a multimodal memory system for long-term operation that preserves verifiable and recoverable multimodal evidence under bounded resources, maintains temporally evolving and cross-representation-consistent memory states, and uses continual feedback to improve its memory-management policies.

References

1. Aamodt and Plaza (1994) Agnar Aamodt and Enric Plaza. Case-based reasoning: Foundational issues, methodological variations, and system approaches. AI communications, 7(1):39–59, 1994.
2. Abhyankar et al. (2024) Reyna Abhyankar, Zijian He, Vikranth Srivatsa, Hao Zhang, and Yiying Zhang. InferCept: Efficient intercept support for augmented large language model inference. In Proceedings of the 41st International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, pages 81–95, 2024.
3. Agashe et al. (2025) Saaket Agashe, Jiuzhou Han, Shuyu Gan, Jiachen Yang, Ang Li, and Xin Wang. Agent S: An open agentic framework that uses computers like a human. In International Conference on Learning Representations, 2025.
4. Alam et al. (2026) Samiul Alam, Shakhrul Iman Siam, Michael J. Proulx, James Fort, Richard Newcombe, Hyo Jin Kim, and Mi Zhang. Supermemory-vqa: An egocentric visual question-answering benchmark for long-horizon memory, 2026. URL https://arxiv.org/abs/2606.00825.
5. Ali et al. (2025) Muhammad Qasim Ali, Saeejith Nair, Alexander Wong, Yuchen Cui, and Yuhao Chen. Graphpad: Inference-time 3d scene graph updates for embodied question answering, 2025. URL https://arxiv.org/abs/2506.01174.
6. Anderson et al. (2004) John R Anderson, Daniel Bothell, Michael D Byrne, Scott Douglass, Christian Lebiere, and Yulin Qin. An integrated theory of the mind. Psychological review, 111(4):1036–1060, 2004.
7. Anthropic (2026) Anthropic. Introducing Claude Opus 5. https://www.anthropic.com/news/claude-opus-5, July 2026.
8. Anwar et al. (2025) Abrar Anwar, John Welsh, Joydeep Biswas, Soha Pouya, and Yan Chang. ReMEmbR: Building and reasoning over long-horizon spatio-temporal memory for robot navigation. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 2838–2845. IEEE, 2025.
9. Atkinson and Shiffrin (1968) Richard C Atkinson and Richard M Shiffrin. Human memory: A proposed system and its control processes. In Psychology of learning and motivation, volume 2, pages 89–195. Elsevier, 1968.
10. Baddeley and Hitch (1974) Alan D Baddeley and Graham Hitch. Working memory. In Psychology of Learning and Motivation, volume 8, pages 47–89. Elsevier, 1974.
11. Bärmann et al. (2026) Leonard Bärmann, Joana Plewnia, Alex Waibel, and Tamim Asfour. Learning to forget – hierarchical episodic memory for lifelong robot deployment, 2026. URL https://arxiv.org/abs/2604.11306.
12. Bei et al. (2026) Yuanchen Bei, Tianxin Wei, Xuying Ning, Yanjun Zhao, Zhining Liu, Xiao Lin, Yada Zhu, Hendrik Hamann, Jingrui He, and Hanghang Tong. Mem-Gallery: Benchmarking multimodal long-term conversational memory for MLLM agents. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 40750–40784, 2026.
13. Bian et al. (2027) Zhuohang Bian, Feiyang Wu, Zhuoran Li, Teng Ma, and Youwei Zhuo. Tokencake: A kv-cache-centric serving framework for llm-based multi-agent applications. In Proceedings of the Twenty-Second European Conference on Computer Systems (EuroSys ’27), 2027. URL https://arxiv.org/abs/2510.18586. To appear.
14. Bjorck et al. (2026) Johan Bjorck, Zhiqi Li, Yunze Man, Jing Wang, An-Chieh Cheng, Sifei Liu, Shihao Wang, Zhiding Yu, Abhishek Badki, Stan Birchfield, Valts Blukis, Yevgen Chebotar, Siyi Chen, Sicong Leng, Yu-Cheng Chou, Tianli Ding, Boyi Li, Zhengyi Luo, Hang Su, Jonathan Tremblay, Tingwu Wang, Bowen Wen, Jimmy Wu, Xianghui Xie, Hanrong Ye, Hongxu Yin, K. R. Zentner, Liangyan Gui, Yu-Xiong Wang, Yuke Zhu, Linxi Fan, and Jan Kautz. Vesta: A generalist embodied reasoning model, 2026. URL https://arxiv.org/abs/2606.20905.
15. Bo et al. (2026) Weihao Bo, Shan Zhang, Yanpeng Sun, Jingjing Wu, Qunyi Xie, Xiao Tan, Kunbin Chen, Wei He, Xiaofan Li, Na Zhao, Jingdong Wang, and Zechao Li. Vilomem: Agentic learner with grow-and-refine multimodal semantic memory. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5476–5486, 2026. URL https://openaccess.thecvf.com/content/CVPR2026/html/Bo_ViLoMem_Agentic_Learner_with_Grow-and-Refine_Multimodal_Semantic_Memory_CVPR_2026_paper.html.
16. Cai et al. (2026) Zhongyi Cai, Yi Du, Chen Wang, and Yu Kong. Vision to geometry: 3D spatial memory for sequential embodied MLLM reasoning and exploration. In Advances in Neural Information Processing Systems, 2026. URL https://arxiv.org/abs/2512.02458. To appear.
17. Chai et al. (2026) Huacan Chai, Yukai Wang, Yingxuan Yang, Dan Peng, Yuanyi Song, Zhihui Fu, Weiwen Liu, Jianghao Lin, Jun Wang, and Weinan Zhang. SMMBench: A benchmark for source-distributed multimodal agent memory. In Advances in Neural Information Processing Systems, 2026. URL https://arxiv.org/abs/2605.15710. To appear.
18. Chang and Ren (2026) Jiale Chang and Yuxiang Ren. Scrapmem: A bio-inspired framework for on-device personalized agent memory via optical forgetting, 2026. URL https://arxiv.org/abs/2605.03804.
19. Chattopadhyay et al. (2026) Sameep Chattopadhyay, Alexander Erdmann, and Mari Ostendorf. Beyond fresh starts: Stateful inference for streaming ASR in conversational voice agents, 2026. URL https://arxiv.org/abs/2608.22101.
20. Chen et al. (2026a) Guo Chen, Lidong Lu, Yicheng Liu, Liangrui Dong, Lidong Zou, Jixin Lv, Zhenquan Li, Xinyi Mao, Baoqi Pei, Shihao Wang, Zhiqi Li, Karan Sapra, Fuxiao Liu, Yin-Dong Zheng, Yifei Huang, Limin Wang, Zhiding Yu, Andrew Tao, Guilin Liu, and Tong Lu. Towards multimodal lifelong understanding: A dataset and agentic baseline, 2026a. URL https://arxiv.org/abs/2603.05484.
21. Chen et al. (2026b) Kaifeng Chen, Hongtao Liu, Qiyao Peng, Jian Yang, Yongqiang Liu, Xiaochen Zhang, and Qing Yang. Mardoc: A memory-aware refinement agent framework for multimodal long document qa, 2026b. URL https://arxiv.org/abs/2606.05749.
22. Chen et al. (2026c) Tao Chen, Kun Zhang, Qiong Wu, Xiao Chen, Chao Chang, Xiaoshuai Sun, Yiyi Zhou, and Rongrong Ji. Scaling the long video understanding of multimodal large language models via visual memory mechanism. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 31877–31888, 2026c. URL https://openaccess.thecvf.com/content/CVPR2026/html/Chen_Scaling_the_Long_Video_Understanding_of_Multimodal_Large_Language_Models_CVPR_2026_paper.html.
23. Chen et al. (2026d) Tianxing Chen, Yuran Wang, Mingleyang Li, Yan Qin, Hao Shi, Zixuan Li, Yifan Hu, Yingsheng Zhang, Kaixuan Wang, et al. RMBench: Memory-dependent robotic manipulation benchmark with insights into policy design, 2026d. URL https://arxiv.org/abs/2603.01229.
24. Chen et al. (2026e) Tongbo Chen, Zhengxi Lu, Zhan Xu, Guocheng Shao, Shaohan Zhao, Fei Tang, Yong Du, Kaitao Song, Yizhou Liu, et al. Knowu-bench: Towards interactive, proactive, and personalized mobile agent evaluation, 2026e. URL https://arxiv.org/abs/2604.08455.
25. Chen et al. (2026f) Weitao Chen, Jiaxin Hu, Tianyidan Xie, Li Yang, Yuyi Qian, Banghao Xu, Ziheng Tang, Shenyi Wang, Mingyue Yu, et al. Egomonth: A month-level egocentric video benchmark for long-term spatiotemporal memory, 2026f. URL https://arxiv.org/abs/2608.13113.
26. Chen et al. (2026g) Xuan Chen, Hao Liu, Tao Yuan, Mehran Kafai, Piotr Habas, and Xiangyu Zhang. MemoPhishAgent: Memory-augmented multi-modal LLM agent for phishing URL detection. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), pages 1182–1196, San Diego, California, USA, 2026g. Association for Computational Linguistics. 10.18653/v1/2026.acl-industry.84. URL https://aclanthology.org/2026.acl-industry.84/.
27. Chen et al. (2026h) Zhisheng Chen, Tingyu Wu, Zijie Zhou, Zhengwei Xie, Jinhan Li, Ziyan Weng, Liang Lin, Jingwei Song, Zikai Xiao, and Yingwei Zhang. Polarmem: A training-free polarized latent graph memory for verifiable vision-language models, 2026h. URL https://arxiv.org/abs/2602.00415.
28. Chen, Celia (2026) Chen, Celia. Unlocking the Codex harness: how we built the App Server. https://openai.com/index/unlocking-the-codex-harness/, 2026.
29. Cherepanov et al. (2026) Egor Cherepanov, Nikita Kachaev, Alexey K. Kovalev, and Aleksandr I. Panov. Memory, benchmark & robots: A benchmark for solving complex tasks with reinforcement learning. In The Fourteenth International Conference on Learning Representations, pages 9759–9815, 2026. URL https://proceedings.iclr.cc/paper_files/paper/2026/hash/1088fffae5f2aa8c8324e4f45f62248d-Abstract-Conference.html.
30. Chhikara et al. (2025) Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory. In ECAI 2025, volume 413 of Frontiers in Artificial Intelligence and Applications, pages 2993–3000. IOS Press, 2025. 10.3233/FAIA251160.
31. Choi et al. (2026) Seoyoung Choi, Minseok Ko, Hyunseok Lee, Kunwoong Kim, Woomin Song, Chanseok Jeon, and Jinwoo Shin. Naive visual memory is not enough: A failure-mode study of gui agents, 2026. URL https://arxiv.org/abs/2606.14106.
32. Chung et al. (2026) Nhat Chung, Taisei Hanyu, Toan Nguyen, Huy Le, Frederick Bumgarner, Duy Minh Ho Nguyen, Khoa Vo, Kashu Yamazaki, Chase Rainwater, Tung Kieu, Anh Nguyen, and Ngan Le. Rethinking progression of memory state in robotic manipulation: An object-centric perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, 2026. URL https://ojs.aaai.org/index.php/AAAI/article/view/37337.
33. Cudlenco et al. (2026) Nicolae Cudlenco, Mihai Masala, and Marius Leordeanu. Authoring for living worlds: Tool-constrained llm agents for executable multi-actor scenarios, 2026. URL https://arxiv.org/abs/2604.10383.
34. Dai et al. (2026) Yinpei Dai, Hongze Fu, Jayjun Lee, Yuejiang Liu, Haoran Zhang, Jianing Yang, Chelsea Finn, Nima Fazeli, and Joyce Chai. Robomme: Benchmarking and understanding memory for robotic generalist policies. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 22396–22450. PMLR, 2026. URL https://proceedings.mlr.press/v306/dai26c.html.
35. DeepSeek-AI et al. (2026) DeepSeek-AI, Anyi Xu, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chenchen Ling, et al. DeepSeek-V4: Towards highly efficient million-token context intelligence. arXiv preprint arXiv:2606.19348, 2026.
36. Deng et al. (2026) Chenlong Deng, Mengjie Deng, Junjie Wu, Dun Zeng, Teng Wang, Qingsong Xie, Jiadeng Huang, Shengjie Ma, Changwang Zhang, Zhaoxiang Wang, Jun Wang, Yutao Zhu, and Zhicheng Dou. Deepimagesearch: Benchmarking multimodal agents for context-aware image retrieval in visual histories. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 23935–23952. PMLR, 2026. URL https://proceedings.mlr.press/v306/deng26g.html.
37. Di et al. (2025) Shangzhe Di, Zhelun Yu, Guanghao Zhang, Haoyuan Li, Tao Zhong, Hao Cheng, Bolin Li, Wanggui He, Fangxun Shu, and Hao Jiang. Streaming video question-answering with in-context video KV-cache retrieval. In The Thirteenth International Conference on Learning Representations, 2025.
38. Diko et al. (2025) Anxhelo Diko, Tinghuai Wang, Wassim Swaileh, Shiyan Sun, and Ioannis Patras. ReWind: Understanding long videos with instructed learnable memory. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13734–13743, 2025. 10.1109/CVPR52734.2025.01282. URL https://openaccess.thecvf.com/content/CVPR2025/html/Diko_ReWind_Understanding_Long_Videos_with_Instructed_Learnable_Memory_CVPR_2025_paper.html.
39. Dorbala and Manocha (2026) Vishnu Sashank Dorbala and Dinesh Manocha. MemCtrl: Using MLLMs as active memory controllers on embodied agents. arXiv preprint arXiv:2601.20831, 2026. 10.48550/arXiv.2601.20831. URL https://arxiv.org/abs/2601.20831.
40. Du et al. (2025) Yiming Du, Wenyu Huang, Danna Zheng, Zhaowei Wang, Sebastien Montella, Mirella Lapata, Kam-Fai Wong, and Jeff Z. Pan. Rethinking memory in LLM based agents: Representations, operations, and emerging topics, 2025. URL https://arxiv.org/abs/2505.00675.
41. El Hattami et al. (2026) Amine El Hattami, Nicolas Chapados, and Christopher Pal. Skill.nb: Selective formalization and gated execution for durable agent workflows. arXiv preprint arXiv:2606.08049, 2026. URL https://arxiv.org/abs/2606.08049.
42. Fan et al. (2026a) Haoyang Fan, Dhruv Parikh, Anvitha Ramachandran, Sameh Gobriel, Nilesh Jain, Rajgopal Kannan, and Viktor Prasanna. Folio: Focused semantic memory for streaming video understanding, 2026a. URL https://arxiv.org/abs/2607.13298.
43. Fan et al. (2026b) Jingxiang Fan, Junbao Zhuo, and Bochao Zou. RRM: Experience-driven reflective retrieval memory for long-horizon multimodal reasoning. arXiv preprint arXiv:2607.28156, 2026b. URL https://arxiv.org/abs/2607.28156.
44. Fan et al. (2024) Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li. VideoAgent: A memory-augmented multimodal agent for video understanding. In European Conference on Computer Vision, pages 75–92. Springer, 2024.
45. Fan et al. (2025) Yue Fan, Xiaojian Ma, Rongpeng Su, Jun Guo, Rujie Wu, Xi Chen, and Qing Li. Embodied videoagent: Persistent memory from egocentric videos and embodied sensors enables dynamic scene understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 6342–6352, 2025. URL https://openaccess.thecvf.com/content/ICCV2025/html/Fan_Embodied_VideoAgent_Persistent_Memory_from_Egocentric_Videos_and_Embodied_Sensors_ICCV_2025_paper.html.
46. Feng et al. (2026) Junyu Feng, Binxiao Xu, Jiayi Chen, Mengyu Dai, Cenyang Wu, Haodong Li, Bohan Zeng, Yunliu Xie, Hao Liang, Ming Lu, and Wentao Zhang. M2A: Multimodal memory agent with dual-layer hybrid memory for long-term personalized interactions, 2026. URL https://arxiv.org/abs/2602.07624.
47. Gao et al. (2024) Bin Gao, Zhuomin He, Puru Sharma, Qingxuan Kang, Djordje Jevdjic, Junbo Deng, Xingkun Yang, Zhou Yu, and Pengfei Zuo. Cost-efficient large language model serving for multi-turn conversations with CachedAttention. In 2024 USENIX Annual Technical Conference (USENIX ATC 24), pages 111–126. USENIX Association, 2024.
48. Gao et al. (2025) Xinzge Gao, Chuanrui Hu, Bin Chen, and Teng Li. Chain-of-memory: Enhancing gui agents for cross-application navigation, 2025. URL https://arxiv.org/abs/2506.18158.
49. Garrett et al. (2025) Caelan Reed Garrett, Ajay Mandlekar, Bowen Wen, and Dieter Fox. SkillMimicGen: Automated demonstration generation for efficient skill learning and deployment. In Proceedings of the 8th Conference on Robot Learning, volume 270 of Proceedings of Machine Learning Research, pages 2750–2790, 2025. URL https://proceedings.mlr.press/v270/garrett25a.html.
50. Gim et al. (2024) In Gim, Guojun Chen, Seung-seob Lee, Nikhil Sarda, Anurag Khandelwal, and Lin Zhong. Prompt cache: Modular attention reuse for low-latency inference. In P. Gibbons, G. Pekhimenko, and C. De Sa, editors, Proceedings of Machine Learning and Systems, volume 6, pages 325–338, 2024. URL https://proceedings.mlsys.org/paper_files/paper/2024/file/a66caa1703fe34705a4368c3014c1966-Paper-Conference.pdf.
51. Gong et al. (2025) Ziwei Gong, Lin Ai, Harsh Deshpande, Alexander Johnson, Emmy Phung, Zehui Wu, Ahmad Emami, and Julia Hirschberg. Comparison-Based Automatic Evaluation for Meeting Summarization. In Interspeech 2025, pages 291–295, 2025. 10.21437/Interspeech.2025-2771.
52. Google DeepMind (2026) Google DeepMind. Introducing gemini 3.7 flash. https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-3-7-flash/, 2026. Blog post, August 13, 2026.
53. Graves et al. (2014) Alex Graves, Greg Wayne, and Ivo Danihelka. Neural Turing machines. arXiv preprint arXiv:1410.5401, 2014.
54. Gu et al. (2024) Qiao Gu, Alihusein Kuwajerwala, Sacha Morin, Krishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, Chuang Gan, Celso Miguel de Melo, Joshua B. Tenenbaum, Antonio Torralba, Florian Shkurti, and Liam Paull. Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning. In IEEE International Conference on Robotics and Automation (ICRA), pages 5021–5028, 2024. 10.1109/ICRA57147.2024.10610243. URL https://doi.org/10.1109/ICRA57147.2024.10610243.
55. Guo et al. (2026a) Jiarui Guo, Haojia Wei, Yiming Zhang, Yifei Liu, Yuning Gong, Hongjie Zhang, Xue Yang, and Zhihang Zhong. Photoflow: Agentic 3d virtual photography missions. In Advances in Neural Information Processing Systems, 2026a. URL https://arxiv.org/abs/2605.23771. To appear.
56. Guo et al. (2026b) Minghao Guo, Qingyue Jiao, Zeru Shi, Yihao Quan, Boxuan Zhang, Danrui Li, Liwei Che, Wujiang Xu, Shilong Liu, et al. Memeye: A visual-centric evaluation framework for multimodal agent memory, 2026b. URL https://arxiv.org/abs/2605.15128.
57. Guo et al. (2026c) Minghao Guo, Ziyi Ye, Wujiang Xu, Xi Zhu, Wenyue Hua, and Dimitris N Metaxas. Individual turing test: A case study of llm-based simulation using longitudinal personal data. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 3770–3775. Association for Computing Machinery, 2026c. 10.1145/3805712.3809868.
58. Guo et al. (2026d) Pianran Guo, Pengcheng Zhou, Yuchen Jian, Shuhua Chen, Zhongliang Yang, and Linna Zhou. FinAcumen: Financial multimodal reasoning via self-evolving experience memory harness, 2026d. URL https://arxiv.org/abs/2606.17642.
59. Guo et al. (2022) Rentong Guo, Xiaofan Luan, Long Xiang, Xiao Yan, Xiaomeng Yi, Jigao Luo, Qianya Cheng, Weizhi Xu, Jiarui Luo, Frank Liu, Zhenshan Cao, Yanliang Qiao, Ting Wang, Bo Tang, and Charles Xie. Manu: A cloud native vector database management system. Proceedings of the VLDB Endowment, 15(12):3548–3561, 2022. 10.14778/3554821.3554843. URL https://www.vldb.org/pvldb/vol15/p3548-yan.pdf.
60. Han et al. (2026) Qiushi Han, Keya Hu, Linlu Qiu, Cathy Wu, and Kaiming He. Vista: A visual harness for reasoning in an interactive world, 2026. URL https://arxiv.org/abs/2610.02200.
61. He et al. (2024) Bo He, Hengduo Li, Young Kyun Jang, Menglin Jia, Xuefei Cao, Ashish Shah, Abhinav Shrivastava, and Ser-Nam Lim. Ma-lmm: Memory-augmented large multimodal model for long-term video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13504–13514, 2024. URL https://openaccess.thecvf.com/content/CVPR2024/html/He_MA-LMM_Memory-Augmented_Large_Multimodal_Model_for_Long-Term_Video_Understanding_CVPR_2024_paper.html.
62. He et al. (2025a) Peize He, Zichen Wen, Yubo Wang, Yuxuan Wang, Xiaoqian Liu, Jiajie Huang, Zehui Lei, Zhuangcheng Gu, Xiangqi Jin, Jiabing Yang, Kai Li, Zhifei Liu, Weijia Li, Cunxiang Wang, Conghui He, and Linfeng Zhang. Audiomarathon: A comprehensive benchmark for long-context audio understanding and efficiency in audio LLMs, 2025a. URL https://arxiv.org/abs/2510.07293.
63. He et al. (2026) Zefeng He, Siyuan Huang, Xiaoye Qu, Yafu Li, Tong Zhu, Yu Cheng, and Yang Yang. Gems: Agent-native multimodal generation with memory and skills, 2026. URL https://arxiv.org/abs/2603.28088.
64. He et al. (2025b) Zifan He, Yingqi Cao, Zongyue Qin, Neha Prakriya, Yizhou Sun, and Jason Cong. Hmt: Hierarchical memory transformer for efficient long context language processing. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8068–8089, Albuquerque, New Mexico, 2025b. Association for Computational Linguistics. 10.18653/v1/2025.naacl-long.410. URL https://aclanthology.org/2025.naacl-long.410/.
65. Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997. 10.1162/neco.1997.9.8.1735.
66. Honerkamp et al. (2024) Daniel Honerkamp, Martin Büchner, Fabien Despinoy, Tim Welschehold, and Abhinav Valada. Language-grounded dynamic scene graphs for interactive object search with mobile manipulation. IEEE Robotics and Automation Letters (RA-L), 9(10):8298–8305, 2024. 10.1109/LRA.2024.3441495. URL https://doi.org/10.1109/LRA.2024.3441495.
67. Hong et al. (2026) Rongpei Hong, Jian Lang, Ting Zhong, Yong Wang, and Fan Zhou. TAMEing long contexts in personalization: Towards training-free and state-aware MLLM personalized assistant. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, pages 452–463. Association for Computing Machinery, 2026. 10.1145/3770854.3780214. URL https://doi.org/10.1145/3770854.3780214.
68. Hsu et al. (2025) Cheng-Chun Hsu, Bowen Wen, Jie Xu, Yashraj Narang, Xiaolong Wang, Yuke Zhu, Joydeep Biswas, and Stan Birchfield. SPOT: SE(3) pose trajectory diffusion for object-centric manipulation. In IEEE International Conference on Robotics and Automation, pages 4853–4860, 2025. 10.1109/ICRA55743.2025.11127562. URL https://doi.org/10.1109/ICRA55743.2025.11127562.
69. Hu et al. (2025a) Wenbo Hu, Yining Hong, Yanjun Wang, Leison Gao, Zibu Wei, Xingcheng Yao, Nanyun Peng, Yonatan Bitton, Idan Szpektor, and Kai-Wei Chang. 3DLLM-Mem: Long-term spatial-temporal memory for embodied 3D large language model. In Advances in Neural Information Processing Systems, volume 38, pages 67856–67884, 2025a.
70. Hu et al. (2025b) Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, Senjie Jin, Jiejun Tan, Yanbin Yin, Jiongnan Liu, Zeyu Zhang, Zhongxiang Sun, Yutao Zhu, Hao Sun, Boci Peng, Zhenrong Cheng, Xuanbo Fan, Jiaxin Guo, Xinlei Yu, Zhenhong Zhou, Zewen Hu, Jiahao Huo, Junhao Wang, Yuwei Niu, Yu Wang, Zhenfei Yin, Xiaobin Hu, Yue Liao, Qiankun Li, Kun Wang, Wangchunshu Zhou, Yixin Liu, Dawei Cheng, Qi Zhang, Tao Gui, Shirui Pan, Yan Zhang, Philip Torr, Zhicheng Dou, Ji-Rong Wen, Xuanjing Huang, Yu-Gang Jiang, and Shuicheng Yan. Memory in the age of AI agents, 2025b. URL https://arxiv.org/abs/2512.13564.
71. Hu et al. (2023) Ziniu Hu, Ahmet Iscen, Chen Sun, Zirui Wang, Kai-Wei Chang, Yizhou Sun, Cordelia Schmid, David A. Ross, and Alireza Fathi. REVEAL: Retrieval-augmented visual-language pre-training with multi-source multimodal knowledge memory. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23369–23379, 2023. 10.1109/CVPR52729.2023.02238.
72. Huang et al. (2025) Chenguang Huang, Oier Mees, Andy Zeng, and Wolfram Burgard. Multimodal spatial language maps for robot navigation and manipulation. The International Journal of Robotics Research, 2025. 10.1177/02783649251351658. URL https://doi.org/10.1177/02783649251351658.
73. Huang et al. (2026a) Jie Huang, Ruixun Liu, Sirui Sun, Xinyi Yang, Yin Li, Yixin Zhu, and Yiwu Zhong. M3eval: Multi-modal memory evaluation through cognitively-grounded video tasks, 2026a. URL https://arxiv.org/abs/2606.05008.
74. Huang et al. (2026b) Wei-Chieh Huang, Weizhi Zhang, Yueqing Liang, Yuanchen Bei, Yankai Chen, Tao Feng, Zhen Tan, Yu Wang, Tianxin Wei, Shanglin Wu, et al. A survey of agent memory in the second half: Towards self-evolving and long-horizon agents. Transactions on Machine Learning Research, 2026b.
75. Huang et al. (2026c) Xun Huang, Shijia Zhao, Yunxiang Wang, Xin Lu, Wanfa Zhang, Rongsheng Qu, Weixin Li, Yunhong Wang, and Chenglu Wen. Msgnav: Unleashing the power of multi-modal 3d scene graph for zero-shot embodied navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 37154–37163, 2026c. URL https://openaccess.thecvf.com/content/CVPR2026/html/Huang_MSGNav_Unleashing_the_Power_of_Multi-modal_3D_Scene_Graph_for_CVPR_2026_paper.html.
76. Huang et al. (2026d) Zhengjun Huang, Wenxuan Liu, Zhoujin Tian, Wei Chen, Junle Chen, Yuqian Wu, Fangyuan Zhang, Qintian Guo, and Xiaofang Zhou. M^{3}Exam: Benchmarking multimodal memory for realistic user-agent interactions, 2026d. URL https://arxiv.org/abs/2606.07402.
77. Jaeckle et al. (2023) Florian Jaeckle, Fartash Faghri, Ali Farhadi, Oncel Tuzel, and Hadi Pouransari. FastFill: Efficient compatible model update. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=rnRiiHw8Vy.
78. Jain et al. (2025) Jitesh Jain, Shubham Maheshwari, Ning Yu, Wen-mei Hwu, and Humphrey Shi. Augustus: An llm-driven multimodal agent system with contextualized user memory, 2025. URL https://arxiv.org/abs/2510.15261.
79. Jang et al. (2026) Hakbeom Jang, Younghoon Min, Sunwoong Kim, Taeyoung Ahn, Hanyee Kim, Youngpyo Joo, Hoshik Kim, and Jongryool Kim. Itme: Inference tiered memory expansion with disaggregated cxl-hybrid memories, 2026. URL https://arxiv.org/abs/2606.12556.
80. Jang et al. (2025) Lawrence Jang, Yinheng Li, Dan Zhao, Charles Ding, Justin Lin, Paul Pu Liang, Rogerio Bonatti, and Kazuhito Koishida. Videowebarena: Evaluating long context multimodal agents with video understanding web tasks. In International Conference on Learning Representations (ICLR), pages 36934–36958, 2025. URL https://proceedings.iclr.cc/paper_files/paper/2025/hash/5b555804d495321df2e3208cc27f4fbc-Abstract-Conference.html.
81. Ji et al. (2026) Li Ji, Siyin Wang, Pengfang Qian, Xiaopeng Yu, Yihai Tian, Zhaoye Fei, Jingjing Gong, and Xipeng Qiu. Hime: Hierarchical embodied memory for long-horizon vision-language-action control. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 51611–51631. PMLR, 2026. URL https://proceedings.mlr.press/v306/ji26k.html.
82. Jia et al. (2025) Zixia Jia, Jiaqi Li, Yipeng Kang, Yuxuan Wang, Tong Wu, Quansen Wang, Xiaobo Wang, Shuyi Zhang, Junzhe Shen, Qing Li, Siyuan Qi, Yitao Liang, Di He, Zilong Zheng, and Song-Chun Zhu. The AI hippocampus: How far are we from human memory? Transactions on Machine Learning Research, 2025. URL https://openreview.net/forum?id=Sk7pwmLuAY.
83. Jiang et al. (2025a) Hanxiao Jiang, Binghao Huang, Ruihai Wu, Zhuoran Li, Shubham Garg, Hooshang Nayyeri, Shenlong Wang, and Yunzhu Li. RoboEXP: Action-conditioned scene graph via interactive exploration for robotic manipulation. In Proceedings of the 8th Conference on Robot Learning, volume 270 of Proceedings of Machine Learning Research, pages 3027–3052. PMLR, 2025a. URL https://proceedings.mlr.press/v270/jiang25c.html.
84. Jiang et al. (2025b) Hongda Jiang, Xinyuan Zhang, Siddhant Garg, Rishab Arora, Shiun-Zu Kuo, Jiayang Xu, Aaron Colak, and Xin Luna Dong. Memory-QA: Answering recall questions based on multimodal memories. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 24244–24266, 2025b. 10.18653/v1/2025.emnlp-main.1234. URL https://aclanthology.org/2025.emnlp-main.1234/.
85. Jiang et al. (2026a) Rongjie Jiang, Jianwei Wang, Gengda Zhao, Chengyang Luo, Kai Wang, and Wenjie Zhang. Advancing multimodal agent reasoning with long-term neuro-symbolic memory. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 2132–2143, 2026a. 10.1145/3770855.3817643.
86. Jiang et al. (2026b) Zixuan Jiang, Yanqiao Zhu, Peng Wang, Qinyuan Chen, Xinjian Zhao, Xipeng Qiu, Wupeng Wang, Zhifu Gao, Xiangang Li, Kai Yu, and Xie Chen. Towards human-like interactive speech recognition with agentic correction and semantic evaluation, 2026b. URL https://arxiv.org/abs/2605.29430.
87. Jin et al. (2026a) Jiajie Jin, Yuyao Zhang, Yimeng Xu, Hongjin Qian, Yutao Zhu, and Zhicheng Dou. FinSight: Towards real-world financial deep research. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5868–5894, San Diego, California, United States, 2026a. Association for Computational Linguistics. 10.18653/v1/2026.acl-long.265. URL https://aclanthology.org/2026.acl-long.265/.
88. Jin et al. (2025a) Seokho Jin, Manseo Kim, Sungho Byun, Hansol Kim, Jungmin Lee, Sujeong Baek, Semi Kim, Sanghum Park, and Sung Park. Persode: Personalized visual journaling with episodic memory-aware AI agent. arXiv preprint arXiv:2508.20585, 2025a. 10.48550/arXiv.2508.20585. URL https://arxiv.org/abs/2508.20585.
89. Jin et al. (2026b) Shilong Jin, Lanjun Wang, and Zhuosheng Zhang. SE-GA: Memory-augmented self-evolution for GUI agents. In Forty-third International Conference on Machine Learning, 2026b. URL https://openreview.net/forum?id=9S9V5ntQJM.
90. Jin et al. (2025b) Shuowei Jin, Xueshen Liu, Qingzhao Zhang, and Zhuoqing Mao. Compute or load KV cache? why not both? In Proceedings of the 42nd International Conference on Machine Learning (ICML), volume 267 of Proceedings of Machine Learning Research, pages 28031–28043, 2025b.
91. Jin et al. (2026c) Ye Jin, Yangyang Xu, Jun Zhu, and Yibo Yang. Memslides: A hierarchical memory driven agent framework for personalized slide generation with multi-turn local revision, 2026c. URL https://arxiv.org/abs/2606.17162.
92. Kang et al. (2026) Hao Kang, Ziyang Li, Weili Xu, Xinyu Yang, Yinfang Chen, Junxiong Wang, Beidi Chen, Tushar Krishna, Chenfeng Xu, and Simran Arora. ThunderAgent: A fast, simple, and program-aware agentic inference system. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 55630–55657. PMLR, 2026. URL https://proceedings.mlr.press/v306/kang26j.html.
93. Kang et al. (2025) Jiazheng Kang, Mingming Ji, Zhe Zhao, and Ting Bai. Memory os of ai agent. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 25961–25970, 2025. 10.18653/v1/2025.emnlp-main.1318. URL https://aclanthology.org/2025.emnlp-main.1318/.
94. Kim et al. (2026) Kangsan Kim, Yanlai Yang, Suji Kim, Woongyeong Yeo, Youngwan Lee, Mengye Ren, and Sung Ju Hwang. MA-EgoQA: Question answering over egocentric videos from multiple embodied agents, 2026. URL https://arxiv.org/abs/2603.09827.
95. Kirmayr et al. (2025) Johannes Kirmayr, Lukas Stappen, Phillip Schneider, Florian Matthes, and Elisabeth Andre. CarMem: Enhancing long-term memory in LLM voice assistants through category-bounding. In Proceedings of the 31st International Conference on Computational Linguistics: Industry Track, pages 343–357. Association for Computational Linguistics, 2025. URL https://aclanthology.org/2025.coling-industry.29/.
96. Kwon et al. (2023) Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, pages 611–626. Association for Computing Machinery, 2023. 10.1145/3600006.3613165.
97. Lang et al. (2026) Shiqiang Lang, Jing Liu, Haoyang He, Peiwen Sun, Yuanteng Chen, Tao Liu, Lan Yang, Longteng Guo, and Honggang Zhang. Longspace: Exploring long-horizon spatial memory from perception to recall in video, 2026. URL https://arxiv.org/abs/2606.05677.
98. Lee et al. (2024) Sunjae Lee, Junyoung Choi, Jungjae Lee, Munim Hasan Wasi, Hojun Choi, Steven Y Ko, Sangeun Oh, and Insik Shin. Mobilegpt: Augmenting llm with human-like app memory for mobile task automation. In Proceedings of the 30th Annual International Conference on Mobile Computing and Networking, pages 1119–1133. Association for Computing Machinery, 2024. 10.1145/3636534.3690682.
99. Lee et al. (2026) Taekbeom Lee, Youngseok Jang, Jeonghwa Heo, Jeongjun Choi, and H. Jin Kim. Obsgraph: Hierarchical observation representation for embodied reasoning and exploration, 2026. URL https://arxiv.org/abs/2606.24068.
100. Lee et al. (2025) Taeyeop Lee, Gyuree Kang, Bowen Wen, Youngho Kim, Seunghyeok Back, In So Kweon, David Hyunchul Shim, and Kuk-Jin Yoon. DeLTa: Demonstration and language-guided novel transparent object manipulation, 2025. URL https://arxiv.org/abs/2510.05662.
101. Lei et al. (2026) Huashuo Lei, Wenxuan Song, Huarui Zhang, Jieyuan Pei, Jiayi Chen, Haodong Yan, Han Zhao, Pengxiang Ding, Zhipeng Zhang, et al. Robomemarena: A comprehensive and challenging robotic memory benchmark, 2026. URL https://arxiv.org/abs/2605.10921.
102. Lei et al. (2025) Mingcong Lei, Honghao Cai, Yuyuan Yang, Yimou Wu, Jinke Ren, Zezhou Cui, Liangchen Tan, Junkun Hong, Gehan Hu, Shuangyu Zhu, Shaohan Jiang, Ge Wang, Junyuan Tan, Zhenglin Wan, Zheng Li, Zhen Li, Shuguang Cui, Yiming Zhao, and Yatong Han. Robomemory: A brain-inspired multi-memory agentic framework for interactive environmental learning in physical embodied systems, 2025. URL https://arxiv.org/abs/2508.01415.
103. Lewis et al. (2020) Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in neural information processing systems, volume 33, pages 9459–9474, 2020.
104. Li et al. (2026a) Bingxuan Li, Rui Yang, Cheng Qian, Jiateng Liu, Jeonghwan Kim, Zhenhailong Wang, Manling Li, Tong Zhang, and Heng Ji. Long-horizon embodied decision-making via multimodal memory compression. arXiv preprint arXiv:2608.01456, 2026a. 10.48550/arXiv.2608.01456. URL https://arxiv.org/abs/2608.01456.
105. Li et al. (2026b) Borui Li, Xingcai Zhang, Tianen Liu, Shuai Wang, and Yun Cheng. Digimongpt: An evolvable agent with hierarchical human-like memory for video question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 6001–6009, 2026b.
106. Li et al. (2026c) Chenghao Li, Jun Liu, Songbo Zhang, Huadong Jian, Hao Ni, Lik-Hang Lee, Sung-Ho Bae, Guoqing Wang, Yang Yang, and Chaoning Zhang. Experience transfer for multimodal llm agents in minecraft game. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 37143–37153, 2026c. URL https://openaccess.thecvf.com/content/CVPR2026/html/Li_Experience_Transfer_for_Multimodal_LLM_Agents_in_Minecraft_Game_CVPR_2026_paper.html.
107. Li et al. (2025a) Hanchen Li, Runyuan He, Qiuyang Mang, Qizheng Zhang, Huanzhi Mao, Xiaokun Chen, Hangrui Zhou, Huanchen Zhang, Alvin Cheung, Joseph Gonzalez, and Ion Stoica. Continuum: Efficient and robust multi-turn llm agent scheduling with kv cache time-to-live. arXiv preprint arXiv:2511.02230, 2025a.
108. Li et al. (2026d) Haoyuan Li, Zhengdong Hu, Jun Wang, Hehe Fan, and Yi Yang. Skill-3D: Evolving scene-aware skills for agentic 3D spatial reasoning. arXiv preprint arXiv:2606.07436, 2026d. URL https://arxiv.org/abs/2606.07436.
109. Li et al. (2026e) Haozhou Li, Xiangyu Dong, Huiyan Jiang, Yaoming Zhou, and Xiaoguang Ma. Cmmr-vln: Vision-and-language navigation via continual multimodal memory retrieval, 2026e. URL https://arxiv.org/abs/2603.07997.
110. Li et al. (2026f) Hongyu Li, Bowen Wen, Xinghao Zhu, Yixuan Wang, Yilun Du, Yunzhu Li, George Konidaris, Stan Birchfield, Soha Pouya, Chenran Li, and Yan Chang. Hydra-0: Action flow for generalist world modeling and control, 2026f. URL https://arxiv.org/abs/2608.18077.
111. Li et al. (2026g) Ji Li, Bo Wang, Jing Xia, Mingyi Li, and Shiyan Hu. Himm: Human-inspired long-term memory modeling for embodied exploration and question answering. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2026g. URL https://arxiv.org/abs/2602.15513.
112. Li et al. (2026h) Jinze Li, Yang Zhang, Xin Yang, Jiayi Qu, Jinfeng Xu, Shuo Yang, Junhua Ding, and Edith Cheuk-Han Ngai. OCR-memory: Optical context retrieval for long-horizon agent memory. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10409–10420, San Diego, California, United States, 2026h. Association for Computational Linguistics. 10.18653/v1/2026.acl-long.474. URL https://aclanthology.org/2026.acl-long.474/.
113. Li et al. (2026i) Pengteng Li, Weiyu Guo, He Zhang, Tiefu Cai, Xiao He, Yandong Guo, and Hui Xiong. Spatial memory for out-of-vision manipulation in vision-language-action. In International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 67119–67142. PMLR, 2026i. URL https://proceedings.mlr.press/v306/li26ay.html.
114. Li et al. (2026j) Xiang Li, Ya-Li Li, Yuan Wang, and Shengjin Wang. Trm-vla: Temporal-aware chain-of-thought reasoning and memorization for vision-language-action models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10943–10953, 2026j. URL https://openaccess.thecvf.com/content/CVPR2026/html/Li_TRM-VLA_Temporal-Aware_Chain-of-Thought_Reasoning_and_Memorization_for_Vision-Language-Action_Models_CVPR_2026_paper.html.
115. Li et al. (2026k) Xinze Li, Ziyue Zhu, Siyuan Liu, Yubo Ma, Yuhang Zang, Yixin Cao, and Aixin Sun. EMemBench: Interactive benchmarking of episodic memory for VLM agents. In Findings of the Association for Computational Linguistics: EMNLP 2026, 2026k. URL https://arxiv.org/abs/2601.16690. To appear.
116. Li et al. (2024) Zaijing Li, Yuquan Xie, Rui Shao, Gongwei Chen, Dongmei Jiang, and Liqiang Nie. Optimus-1: Hybrid multimodal memory empowered agents excel in long-horizon tasks. In Advances in neural information processing systems, volume 37, pages 49881–49913, 2024.
117. Li et al. (2025b) Zhiyu Li, Chenyang Xi, Chunyu Li, Ding Chen, Boyu Chen, Shichao Song, Simin Niu, Hanyu Wang, Jiawei Yang, Chen Tang, Qingchen Yu, Jihao Zhao, Yezhaohui Wang, Peng Liu, Zehao Lin, Pengyuan Wang, Jiahao Huo, Tianyi Chen, Kai Chen, Kehang Li, Zhen Tao, Huayi Lai, Hao Wu, Bo Tang, Zhengren Wang, Zhaoxin Fan, Ningyu Zhang, Linfeng Zhang, Junchi Yan, Mingchuan Yang, Tong Xu, Wei Xu, Huajun Chen, Haofen Wang, Hongkang Yang, Wentao Zhang, Zhi-Qin John Xu, Siheng Chen, and Feiyu Xiong. Memos: A memory os for ai system, 2025b. URL https://arxiv.org/abs/2507.03724.
118. Lian et al. (2026) Niu Lian, Yuting Wang, Hanshu Yao, Jinpeng Wang, Bin Chen, Yaowei Wang, Min Zhang, and Shu-Tao Xia. From verbatim to gist: Distilling pyramidal multimodal memory via semantic information bottleneck for long-horizon video agents. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11601–11617, 2026.
119. Liang et al. (2022) Junchi Liang, Bowen Wen, Kostas Bekris, and Abdeslam Boularias. Learning sensorimotor primitives of sequential manipulation tasks from visual demonstrations. In 2022 IEEE International Conference on Robotics and Automation, pages 8591–8597, 2022. 10.1109/ICRA46639.2022.9811703. URL https://doi.org/10.1109/ICRA46639.2022.9811703.
120. Liao et al. (2026) Chih-Ting Liao, Xi Xiao, Chunlei Meng, Zhangquan Chen, Yitong Qiao, Weilin Zhou, Tianyang Wang, Xu Zheng, and Xin Cao. Spamem: Benchmarking dynamic spatial reasoning via perception-memory integration in embodied environments. In Computer Vision – ECCV 2026, Lecture Notes in Computer Science, pages 425–442. Springer, 2026. 10.1007/978-3-032-37235-2_25.
121. Lin et al. (2026a) Jingjie Lin, Bingbing Wang, Zihan Wang, Zhengda Jin, Weiming Qiao, Jing Li, and Ruifeng Xu. Connecting the dots: Benchmarking reflective memory in long-horizon dialogue, 2026a. URL https://arxiv.org/abs/2606.01223.
122. Lin et al. (2026b) Junming Lin, Zheng Fang, Chi Chen, Haoxuan Cheng, Zihao Wan, Fuwen Luo, Ziyue Wang, Peng Li, Yang Liu, and Maosong Sun. StreamingBench: Assessing the gap for MLLMs to achieve streaming video understanding. In ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 12147–12151. IEEE, 2026b.
123. Lin et al. (2026c) Min Lin, Xiwen Liang, Bingqian Lin, Jingzhi Liu, Zijian Jiao, Kehan Li, Ziang Yan, Yu Sun, Weijia Liufu, Yuhan Ma, Jiarui Hu, Yuecheng Liu, Shen Zhao, Yuzheng Zhuang, and Xiaodan Liang. Echovla: Robotic vision-language-action model with synergistic declarative memory for mobile manipulation. In Computer Vision – ECCV 2026, Lecture Notes in Computer Science, pages 517–534. Springer, 2026c. 10.1007/978-3-032-37225-3_28.
124. Lin et al. (2026d) Yueqian Lin, Jingyang Zhang, Qinsi Wang, Hancheng Ye, Yuzhe Fu, Yudong Liu, Hai Helen Li, and Yiran Chen. HippoMM: Hippocampal-inspired multimodal memory for long audiovisual event understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings, pages 5968–5977, 2026d. URL https://openaccess.thecvf.com/content/CVPR2026F/html/Lin_HippoMM_Hippocampal-inspired_Multimodal_Memory_for_Long_Audiovisual_Event_Understanding_CVPRF_2026_paper.html.
125. Lin et al. (2026e) Zichuan Lin, Feiyu Liu, Yijun Yang, Jiafei Lyu, Yiming Gao, Yicheng Liu, Zhicong Lu, Yangbin Yu, Mingyu Yang, Junyou Li, et al. UI-Voyager: A self-evolving GUI agent learning via failed experience. arXiv preprint arXiv:2603.24533, 2026e.
126. Liu et al. (2026a) An Liu, Bingxi Liu, Hongyu Ding, Yixuan Jiang, Yaran Chen, Fulin Tang, Cong Leng, Hong Zhang, and Jian Cheng. Ham-vln: Harnessing hierarchical agentic memory for zero-shot vision-and-language navigation, 2026a. URL https://arxiv.org/abs/2607.29600.
127. Liu et al. (2026b) Chen Liu, Ling Chen, Hanzhang Zhou, Xu Zhang, Quyu Kong, Panrong Tong, Wenhao Wang, Xin Yu, Steven Hoi, and Yue Wang. What memory do GUI agents really need? from passive records to active task-driving states, 2026b. URL https://arxiv.org/abs/2606.31612.
128. Liu et al. (2026c) Chengzhi Liu, Yuzhe Yang, Sophia Xiao Pu, Yepeng Liu, Lin Long, Yichen Guo, Nuo Chen, Zhaotian Weng, Elena Kochkina, Simerjot Kaur, Charese Smiley, Xiaomo Liu, James Zou, Sheng Liu, Yuheng Bu, Songyou Peng, and Xin Eric Wang. Worldmemarena: Evaluating multimodal agent memory through action-world interaction. In Advances in Neural Information Processing Systems, 2026c. URL https://arxiv.org/abs/2605.29341. To appear.
129. Liu et al. (2026d) Guangyi Liu, Pengxiang Zhao, Yaozhen Liang, Qinyi Luo, Shunye Tang, Yuxiang Chai, Weifeng Lin, Han Xiao, Wenhao Wang, et al. Memgui-bench: Benchmarking memory of mobile GUI agents in dynamic environments. In Proceedings of the 34th ACM International Conference on Multimedia, 2026d. URL https://arxiv.org/abs/2602.06075. To appear.
130. Liu et al. (2026e) Guanming Liu, Yuqi Ren, Hansu Gu, Peng Zhang, Weihang Wang, Jiahao Liu, Ning Gu, and Tun Lu. Streammembench: Streaming evaluation of agent memory for future-oriented assistance. In Findings of the Association for Computational Linguistics: EMNLP 2026, 2026e. URL https://arxiv.org/abs/2606.14571. To appear.
131. Liu et al. (2026f) Jiale Liu, Yifan Zeng, Malte Højmark-Bertelsen, Marie Normann Gadeberg, Huazheng Wang, and Qingyun Wu. Memory-augmented agent training for business document understanding. In AI Agent for Information Retrieval: Generating and Ranking, volume 2695 of Communications in Computer and Information Science, pages 39–53. Springer, 2026f. 10.1007/978-3-032-11477-8_5.
132. Liu et al. (2026g) Jiaqi Liu, Zipeng Ling, Shi Qiu, Yanqing Liu, Siwei Han, Peng Xia, Haoqin Tu, Zeyu Zheng, Cihang Xie, Charles Fleming, Mingyu Ding, and Huaxiu Yao. Omni-simplemem: Autoresearch-guided discovery of lifelong multimodal agent memory, 2026g. URL https://arxiv.org/abs/2604.01007.
133. Liu et al. (2025a) Junming Liu, Yifei Sun, Weihua Cheng, Haodong Lei, Yirong Chen, Licheng Wen, Xuemeng Yang, Daocheng Fu, Pinlong Cai, Nianchen Deng, Yi Yu, Shuyue Hu, Botian Shi, and Ding Wang. Memverse: Multimodal memory for lifelong learning agents, 2025a. URL https://arxiv.org/abs/2512.03627.
134. Liu et al. (2026h) Kang Liu, Zijing Wang, Yongkang Liu, Mengjie Zhao, Xiaocui Yang, Shi Feng, Yifei Zhang, and Daling Wang. Tram: Enhancing multimodal reasoning with trajectory-derived auxiliary memory, 2026h. URL https://arxiv.org/abs/2608.01922.
135. Liu et al. (2025b) Peiqi Liu, Zhanqiu Guo, Mohit Warke, Soumith Chintala, Chris Paxton, Nur Muhammad Mahi Shafiullah, and Lerrel Pinto. DynaMem: Online dynamic spatio-semantic memory for open world mobile manipulation. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 13346–13355. IEEE, 2025b.
136. Liu et al. (2026i) Ruiping Liu, Junwei Zheng, Yufan Chen, Di Wen, Shaofang Quan, Chengzhi Wu, Jiaming Zhang, Kailun Yang, Kunyu Peng, and Rainer Stiefelhagen. Egoexomem: Cross-view memory reasoning over synchronized egocentric and exocentric videos, 2026i. URL https://arxiv.org/abs/2605.18734.
137. Liu et al. (2026j) Runtao Liu, Ziyi Liu, Jiaqi Tang, Yue Ma, Renjie Pi, Jipeng Zhang, and Qifeng Chen. LongVideoAgent: Multi-agent reasoning with long videos. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 40404–40416, 2026j.
138. Liu et al. (2024a) Yuhan Liu, Hanchen Li, Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Ananthanarayanan, Michael Maire, Henry Hoffmann, Ari Holtzman, and Junchen Jiang. CacheGen: KV cache compression and streaming for fast large language model serving. In Proceedings of the ACM SIGCOMM 2024 Conference, pages 38–56. ACM, 2024a. 10.1145/3651890.3672274.
139. Liu et al. (2026k) Yuhan Liu, Yuyang Huang, Jiayi Yao, Shaoting Feng, Zhuohan Gu, Kuntai Du, Hanchen Li, Yihua Cheng, Junchen Jiang, Shan Lu, Madan Musuvathi, and Esha Choukse. DroidSpeak: KV cache sharing across fine-tuned model variants. In 23rd USENIX Symposium on Networked Systems Design and Implementation (NSDI 26), pages 319–338. USENIX Association, 2026k. URL https://www.usenix.org/conference/nsdi26/presentation/liu-yuhan.
140. Liu et al. (2025c) Zibin Liu, Cheng Zhang, Xi Zhao, Yunfei Feng, Bingyu Bai, Dahu Feng, Erhu Feng, Yubin Xia, and Haibo Chen. Beyond training: Enabling self-evolution of agents with MobiMem, 2025c. URL https://arxiv.org/abs/2512.15784.
141. Liu et al. (2025d) Zikang Liu, Junyi Li, Wayne Xin Zhao, Dawei Gao, Yaliang Li, and Ji rong Wen. Pal-ui: Planning with active look-back for vision-based gui agents, 2025d. URL https://arxiv.org/abs/2510.00413.
142. Liu et al. (2024b) Ziyu Liu, Tao Chu, Yuhang Zang, Xilin Wei, Xiaoyi Dong, Pan Zhang, Zijian Liang, Yuanjun Xiong, Yu Qiao, Dahua Lin, et al. MMDU: A multi-turn multi-image dialog understanding benchmark and instruction-tuning dataset for LVLMs. In Advances in Neural Information Processing Systems, volume 37, pages 8698–8733, 2024b.
143. Long et al. (2026) Lin Long, Yichen He, Wentao Ye, Yiyuan Pan, Yuan Lin, Hang Li, Junbo Zhao, and Wei Li. Seeing, listening, remembering, and reasoning: A multimodal agent with long-term memory. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=PMz29A7Muq.
144. Lu et al. (2026) Yihao Lu, Wanru Cheng, Zeyu Zhang, and Hao Tang. Mma: Multimodal memory agent, 2026. URL https://arxiv.org/abs/2602.16493.
145. Luo et al. (2026a) Jinghao Luo, Yuchen Tian, Chuxue Cao, Ziyang Luo, Hongzhan Lin, Kaixin Li, Chuyi Kong, Ruichao Yang, and Jing Ma. From storage to experience: A survey on the evolution of LLM agent memory mechanisms. In Findings of the Association for Computational Linguistics: ACL 2026, pages 41622–41652, San Diego, California, United States, 2026a. Association for Computational Linguistics. 10.18653/v1/2026.findings-acl.2069. URL https://aclanthology.org/2026.findings-acl.2069/.
146. Luo et al. (2026b) Kaiwen Luo, Liang Lin, Yibo Zhang, Moayad Aloqaily, Jialiang Tao, Dexian Wang, Zhenhong Zhou, Junwei Zhang, Kun Wang, Li Sun, et al. ChronosAudio: A comprehensive long-audio benchmark for evaluating audio-large language models. arXiv preprint arXiv:2601.04876, 2026b.
147. Ma et al. (2025) Bolei Ma, Yuting Li, Wei Zhou, Ziwei Gong, Yang Janet Liu, Katja Jasinskaja, Annemarie Friedrich, Julia Hirschberg, Frauke Kreuter, and Barbara Plank. Pragmatics in the era of large language models: A survey on datasets, evaluation, opportunities and challenges. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8679–8696, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-251-0. 10.18653/v1/2025.acl-long.425. URL https://aclanthology.org/2025.acl-long.425/.
148. Ma et al. (2026) Xiaokang Ma, Yifan Sun, Zhihong Jin, Jie Gu, Yudong Luo, Shenyi Shao, Chu Tang, Jingmin Chen, and Li Pu. ReflectWorld-MM: An entity-oriented multimodal memory system for open-ended video streams. arXiv preprint arXiv:2607.09759, 2026. 10.48550/arXiv.2607.09759. URL https://arxiv.org/abs/2607.09759.
149. Maharana et al. (2024) Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of LLM agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pages 13851–13870, 2024. 10.18653/v1/2024.acl-long.747. URL https://aclanthology.org/2024.acl-long.747/.
150. Mandlekar et al. (2023) Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, and Dieter Fox. MimicGen: A data generation system for scalable robot learning using human demonstrations. In Proceedings of the 7th Conference on Robot Learning, volume 229 of Proceedings of Machine Learning Research, pages 1820–1864, 2023. URL https://proceedings.mlr.press/v229/mandlekar23a.html.
151. Mei et al. (2026) Jingbiao Mei, Jinghong Chen, Guangyu Yang, Xinyu Hou, Margaret Li, and Bill Byrne. According to me: Long-term personalized referential memory QA, 2026. URL https://arxiv.org/abs/2603.01990.
152. Mi et al. (2026) Hongze Mi, Yibo Feng, WenJie Lu, Song Cao, Jinyuan Li, Yanming Li, Xuelin Zhang, Haotian Luo, Songyang Peng, He Cui, Tengfei Tian, Jun Fang, Hua Chai, and Naiqiang Tan. Darwinian memory: A training-free self-regulating memory system for GUI agent evolution. In Forty-third International Conference on Machine Learning, 2026. URL https://openreview.net/forum?id=tNn9bikyXZ.
153. Mitash et al. (2020) Chaitanya Mitash, Rahul Shome, Bowen Wen, Abdeslam Boularias, and Kostas E. Bekris. Task-driven perception and manipulation for constrained placement of unknown objects. IEEE Robotics and Automation Letters, 5(4):5605–5612, 2020. 10.1109/LRA.2020.3006816. URL https://doi.org/10.1109/LRA.2020.3006816.
154. Mokaria et al. (2026) Neel Mokaria, Rishie Raj, Dheeraj Baiju, Xiaoqian Shen, Shraman Pramanick, Kevin Qinghong Lin, Arda Senocak, Mike Zheng Shou, Philip Torr, Mohamed Elhoseiny, Yapeng Tian, Ruohan Gao, Salman Khan, Sayan Nag, Sanjoy Chowdhury, and Dinesh Manocha. A survey on foundations and frontiers of multimodal agentic frameworks: Techniques and applications. Transactions on Machine Learning Research, 2026. URL https://openreview.net/forum?id=eaVoaI7f8v.
155. Mondal et al. (2026) Ishani Mondal, Yiwen Song, Mihir Parmar, Palash Goyal, Jordan Boyd-Graber, Tomas Pfister, and Yale Song. CANVAS: Continuity-aware narratives via visual agentic storyboarding, 2026. URL https://arxiv.org/abs/2604.13452.
156. Mu et al. (2026) Chenyu Mu, Xin He, Qu Yang, Wanshun Chen, Jiadi Yao, Huang Liu, Zihao Yi, Bo Zhao, Xingyu Chen, Ruotian Ma, Fanghua Ye, Erkun Yang, Cheng Deng, Zhaopeng Tu, Xiaolong Li, and Linus. The script is all you need: An agentic framework for long-horizon dialogue-to-cinematic video generation, 2026. URL https://arxiv.org/abs/2601.17737.
157. Newell (1990) Allen Newell. Unified theories of cognition. Harvard University Press, 1990.
158. Nguyen et al. (2026a) Thao Nguyen, Krishna Kumar Singh, Donghyun Kim, Yong Jae Lee, and Yuheng Li. Personal ai agent for camera roll vqa, 2026a. URL https://arxiv.org/abs/2606.05275.
159. Nguyen et al. (2026b) Viet Nguyen, Thao Nguyen, Vishal M. Patel, and Yuheng Li. Personal visual memory from explicit and implicit evidence. In Advances in Neural Information Processing Systems, 2026b. URL https://arxiv.org/abs/2605.28806. To appear.
160. Nie et al. (2026a) Chang Nie, Chaoyou Fu, Yifan Zhang, Haihua Yang, and Caifeng Shan. Personavlm: Long-term personalized multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15000–15009, 2026a.
161. Nie et al. (2026b) Chang Nie, Jiaju Wei, Junlan Feng, Chaoyou Fu, and Caifeng Shan. Light-omni: Reflex over reasoning in agentic video understanding with long-term memory, 2026b. URL https://arxiv.org/abs/2607.05511.
162. Omri et al. (2026) Yasmine Omri, Ziyu Gan, Zachary Broveak, Robin Geens, Zexue He, Alex Pentland, Marian Verhelst, Tsachy Weissman, and Thierry Tambe. Agent memory: Characterization and system implications of stateful long-horizon workloads, 2026. URL https://arxiv.org/abs/2606.06448.
163. OpenAI (2026) OpenAI. GPT-5.6: Frontier Intelligence That Scales with Your Ambition. https://openai.com/index/gpt-5-6/, July 2026.
164. Packer et al. (2023) Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. MemGPT: Towards LLMs as operating systems. arXiv preprint arXiv:2310.08560, 2023. URL https://arxiv.org/abs/2310.08560.
165. Pan et al. (2025) Zaifeng Pan, Ajjkumar Patel, Yipeng Shen, Zhengding Hu, Yue Guan, Wan-Lu Li, Lianhui Qin, Yida Wang, and Yufei Ding. KVFlow: Efficient prefix caching for accelerating LLM-based multi-agent workflows. In Advances in Neural Information Processing Systems (NeurIPS), volume 38, 2025.
166. Park et al. (2023) Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22, 2023.
167. Plaat et al. (2025) Aske Plaat, Max J. van Duijn, Niki van Stein, Mike Preuss, Peter van der Putten, and Kees Joost Batenburg. Agentic Large Language Models, a Survey. Journal of Artificial Intelligence Research, 84, 2025. 10.1613/JAIR.1.18675. URL https://www.jair.org/index.php/jair/article/view/18675.
168. Qian et al. (2024) Rui Qian, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Shuangrui Ding, Dahua Lin, and Jiaqi Wang. Streaming long video understanding with large language models. In Advances in Neural Information Processing Systems, volume 37, pages 119336–119360, 2024. 10.52202/079017-3792. URL https://proceedings.neurips.cc/paper_files/paper/2024/hash/d7ce06e9293c3d8e6cb3f80b4157f875-Abstract-Conference.html.
169. Qin et al. (2025) Ruoyu Qin, Zheming Li, Weiran He, Jialei Cui, Feng Ren, Mingxing Zhang, Yongwei Wu, Weimin Zheng, and Xinran Xu. Mooncake: trading more storage for less computation — a kvcache-centric architecture for serving llm chatbot. In Proceedings of the 23rd USENIX Conference on File and Storage Technologies, FAST ’25, USA, 2025. USENIX Association. ISBN 978-1-939133-45-8.
170. Qin et al. (2026) Zerui Qin, Sheng Yue, Xingyuan Hua, Yongjian Fu, and Ju Ren. Executable agentic memory for gui agent. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 101815–101840. PMLR, 2026. URL https://proceedings.mlr.press/v306/qin26m.html.
171. Qiu et al. (2026) Yilun Qiu, Jiahe Wang, Cilin Yan, Jiayin Cai, Xiaolong Jiang, Yan Hu, and Chun Yuan. Agentcvr: Active multi-agent cross-video reasoning via script-simulated reinforcement learning, 2026. URL https://arxiv.org/abs/2605.29643.
172. Qu et al. (2026) Hongyu Qu, Jianzhe Gao, Xiaobin Hu, Shaohuan Yang, Xinlei Yu, Rui Yan, Wenguan Wang, Xiangbo Shu, and Shuicheng Yan. Dual latent memory in vision-language-action models for robotic manipulation, 2026. URL https://arxiv.org/abs/2607.07608.
173. Rasheed and Kabtoul (2026) A. Haroon Rasheed and Maria Kabtoul. eMEM: A hybrid spatio-temporal memory system for embodied agents, 2026. URL https://arxiv.org/abs/2606.03374.
174. Rathi (2026) Soumil Rathi. Robomme-interference: Benchmarking robot memory under interference, 2026. URL https://arxiv.org/abs/2606.22338.
175. Rege et al. (2026) Aniket Rege, Arka Sadhu, Yuliang Li, Kejie Li, Ramya Korlakai Vinayak, Yuning Chai, Yong Jae Lee, and Hyo Jin Kim. Agentic very long video understanding. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 46575–46602, San Diego, California, United States, 2026. Association for Computational Linguistics. 10.18653/v1/2026.acl-long.2161. URL https://aclanthology.org/2026.acl-long.2161/.
176. Ren et al. (2026) Xiyu Ren, Zhaowei Wang, Yiming Du, Zhongwei Xie, Chi Liu, Xinlin Yang, Haoyue Feng, Wenjun Pan, Tianshi Zheng, et al. Memlens: Benchmarking multimodal long-term memory in large vision-language models, 2026. URL https://arxiv.org/abs/2605.14906.
177. Rezazadeh et al. (2025) Alireza Rezazadeh, Zichao Li, Ange Lou, Yuying Zhao, Wei Wei, and Yujia Bao. Collaborative memory: Multi-user memory sharing in llm agents with dynamic access control, 2025. URL https://arxiv.org/abs/2505.18279.
178. Riva et al. (2026) Paolo Riva, Leonardo Gargani, Matteo Frosi, and Matteo Matteucci. Embodiedlgr: Integrating lightweight graph representation and retrieval for semantic-spatial memory in robotic agents. In 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2026. URL https://arxiv.org/abs/2604.18271. To appear.
179. Saxena et al. (2025) Saumya Saxena, Blake Buchanan, Chris Paxton, Peiqi Liu, Bingqing Chen, Narunas Vaskevicius, Luigi Palmieri, Jonathan Francis, and Oliver Kroemer. Grapheqa: Using 3d semantic scene graphs for real-time embodied question answering. In Proceedings of The 9th Conference on Robot Learning, volume 305 of Proceedings of Machine Learning Research, pages 2714–2742. PMLR, 2025. URL https://proceedings.mlr.press/v305/saxena25a.html.
180. Seong et al. (2025) Jin Seong, Jiyun Park, Wencke Liermann, Hongseok Choi, Yoonji Nam, Hyun Kim, Soojong Lim, and Namhoon Lee. MemEIC: A step toward continual and compositional knowledge editing. In Advances in Neural Information Processing Systems, volume 38, 2025. 10.52202/085713-4304. URL https://proceedings.neurips.cc/paper_files/paper/2025/hash/bbdd7f4acc745b59708b5cb61258d92e-Abstract-Conference.html.
181. Shah et al. (2026) Rutav Shah, Rajat Kumar Jenamani, Xiaohan Zhang, Lingfeng Sun, Roberto Martín-Martín, Yuke Zhu, Deva Ramanan, and Karl Schmeckpeper. Scaling short-term memory of visuomotor policies for long-horizon tasks, 2026. URL https://arxiv.org/abs/2606.16178.
182. Shen et al. (2025) Xiaoqian Shen, Wenxuan Zhang, Jun Chen, and Mohamed Elhoseiny. Vgent: Graph-based retrieval-reasoning-augmented generation for long video understanding. In Advances in Neural Information Processing Systems, volume 38, 2025. URL https://proceedings.neurips.cc/paper_files/paper/2025/hash/91b75715a2ae8f2542b78be31941eac3-Abstract-Conference.html.
183. Shen et al. (2026) Yucheng Shen, Jiulong Wu, Jizhou Huang, Dawei Yin, Lingyong Yan, and Min Cao. Visor: Agentic visual retrieval-augmented generation via iterative search and over-horizon reasoning. In Proceedings of the 34th ACM International Conference on Multimedia, 2026. URL https://arxiv.org/abs/2604.09508. To appear.
184. Shi et al. (2026a) Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, Erjin Zhou, Haoqiang Fan, Xiangyu Zhang, and Gao Huang. Memoryvla: Perceptual-cognitive memory in vision-language-action models for robotic manipulation. In The Fourteenth International Conference on Learning Representations, 2026a. URL https://openreview.net/forum?id=54U3XHf7qq.
185. Shi et al. (2026b) Yansong Shi, Qingsong Zhao, Tianxiang Jiang, Xiangyu Zeng, Yi Wang, and Limin Wang. RIVER: A real-time interaction benchmark for video LLMs. In The Fourteenth International Conference on Learning Representations, pages 9447–9471, 2026b. URL https://proceedings.iclr.cc/paper_files/paper/2026/hash/1022661f3f43406065641f16ce25eafa-Abstract-Conference.html.
186. Shinn et al. (2023) Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, pages 8634–8652, 2023. 10.52202/075280-0377. URL https://proceedings.neurips.cc/paper_files/paper/2023/hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html.
187. Shlapentokh-Rothman et al. (2026) Michal Shlapentokh-Rothman, Prachi Garg, Yu-Xiong Wang, and Derek Hoiem. Decomposing queries into tool calls for long-video keyframe retrieval, 2026. URL https://arxiv.org/abs/2605.23826.
188. Si et al. (2026) Shengyu Si, Yuanzhuo Lu, Ruimeng Yang, Ziyi Ye, Zuxuan Wu, and Yu-Gang Jiang. Vla-pro: Cross-task procedural memory transfer for vision-language-action models. In Advances in Neural Information Processing Systems, 2026. URL https://arxiv.org/abs/2605.29562. To appear.
189. Someki et al. (2026) Masao Someki, Chien-yu Huang, Siddhant Arora, Samuele Cornell, Markus Müller, Nathan Susanj, Rupak Vignesh Swaminathan, Grant Strimel, Jing Liu, and Shinji Watanabe. PlanRAG-Audio: Planning and retrieval augmented generation for long-form audio understanding. In Findings of the Association for Computational Linguistics: ACL 2026, pages 26167–26183. Association for Computational Linguistics, 2026. 10.18653/v1/2026.findings-acl.1304. URL https://aclanthology.org/2026.findings-acl.1304/.
190. Song et al. (2024) Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, Yan Lu, Jenq-Neng Hwang, and Gaoang Wang. MovieChat: From dense token to sparse memory for long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18221–18232, 2024. URL https://openaccess.thecvf.com/content/CVPR2024/html/Song_MovieChat_From_Dense_Token_to_Sparse_Memory_for_Long_Video_CVPR_2024_paper.html.
191. Song et al. (2026) Enxin Song, Wenhao Chai, Tian Ye, Jenq-Neng Hwang, Xi Li, and Gaoang Wang. Moviechat+: Question-aware sparse memory for long video question answering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(1):374–389, 2026. 10.1109/TPAMI.2025.3604614.
192. Squire (1992) Larry R. Squire. Memory and the hippocampus: A synthesis from findings with rats, monkeys, and humans. Psychological Review, 99(2):195–231, 1992. 10.1037/0033-295X.99.2.195.
193. Srivatsa et al. (2025) Vikranth Srivatsa, Zijian He, Reyna Abhyankar, Dongming Li, and Yiying Zhang. Preble: Efficient distributed prompt scheduling for LLM serving. In The Thirteenth International Conference on Learning Representations, 2025. URL https://proceedings.iclr.cc/paper_files/paper/2025/hash/5bc342f48de8264779952fac378f96dc-Abstract-Conference.html.
194. Su et al. (2026) Encheng Su, Jianyu Wu, Jinouwen Zhang, Qiucheng Yu, Chen Tang, Pengze Li, Lintao Wang, Aoran Wang, Xinzhu Ma, Shixiang Tang, Yizhou Wang, and Houqiang Li. S3mem: Structured spatiotemporal scene-event memory for long-horizon interactive question answering, 2026. URL https://arxiv.org/abs/2605.28831.
195. Sun et al. (2026a) Guangzhi Sun, Yixuan Li, Xiaodong Wu, Yudong Yang, Wei Li, Zejun Ma, and Chao Zhang. video-salmonn s: Memory-enhanced streaming audio-visual llm. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 116776–116791. PMLR, 2026a. URL https://proceedings.mlr.press/v306/sun26s.html.
196. Sun et al. (2026b) Haoran Sun, Wenjie Li, Yujie Zhang, Zekai Lin, Fanrui Zhang, Kaitao Chen, Xingqi He, Yichen Li, Mianxin Liu, Lei Liu, and Yankai Jiang. Experience makes skillful: Enabling generalizable medical agent reasoning via self-evolving skill memory. In Advances in Neural Information Processing Systems, 2026b. URL https://arxiv.org/abs/2606.09365. To appear.
197. Sun et al. (2026c) Jingyu Sun, Yan Lin, Yuyang Xue, Yifan Wang, Zhengtao Yao, Rui Qian, Zefeng Xu, Jiachen Li, Xianyang Liu, Jiancheng Pan, Jingyuan Sun, Syed Murtuza Baker, and Hongpeng Zhou. PMMC: Prospective multimodal memory compilation for long-term LVLM agents. arXiv preprint arXiv:2608.00962, 2026c. 10.48550/arXiv.2608.00962. URL https://arxiv.org/abs/2608.00962.
198. Sun et al. (2026d) Libo Sun, Jiwen Zhang, Siyuan Wang, and Zhongyu Wei. MAGNET: Towards adaptive GUI agents with memory-driven knowledge evolution. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 28181–28206, San Diego, California, United States, 2026d. Association for Computational Linguistics. 10.18653/v1/2026.acl-long.1299. URL https://aclanthology.org/2026.acl-long.1299/.
199. Sun et al. (2026e) Xuefei Sun, Xujia Zhang, Brendan Crowe, Doncey Albin, and Christoffer Heckman. Scenegraphgrounder: Zero-shot 3d visual grounding via structured scene graph matching, 2026e. URL https://arxiv.org/abs/2605.21788.
200. Suri et al. (2026) Manan Suri, Sarvesh Baskar, and Dinesh Manocha. Frames2lora: Parametric video internalization for vision-language models, 2026. URL https://arxiv.org/abs/2606.04351.
201. Tang et al. (2026a) Quanwei Tang, Dong Zhang, Shoushan Li, and Guodong Zhou. Don’t just listen, try planning: Graph-based retrieval-generation agent for long-form audio meeting understanding. In Findings of the Association for Computational Linguistics: ACL 2026, pages 20715–20742. Association for Computational Linguistics, 2026a. 10.18653/v1/2026.findings-acl.1038. URL https://aclanthology.org/2026.findings-acl.1038/.
202. Tang et al. (2026b) Song Tang, Shuming Hu, Xincheng Shuai, Henghui Ding, and Yu-Gang Jiang. Seek to segment: Active perception for panoramic referring segmentation. In Computer Vision – ECCV 2026, Lecture Notes in Computer Science, pages 99–116. Springer, 2026b. 10.1007/978-3-032-37044-0_6.
203. Tang et al. (2026c) Yujin Tang, Chenming Shang, Ruize Xu, and Nikhil Singh. DMV-Bench: Diagnosing long-horizon multimodal agents’ visual memory with incidental cue injection. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, 2026c. URL https://arxiv.org/abs/2606.27499. To appear.
204. Tao et al. (2026) Keda Tao, Yuhua Zheng, Jia Xu, Wenjie Du, Kele Shao, Hesong Wang, Xueyi Chen, Xin Jin, Junhan Zhu, Bohan Yu, Weiqiang Wang, Jian Liu, Can Qin, Yulun Zhang, Ming-Hsuan Yang, and Huan Wang. LVOmniBench: Pioneering long audio-video understanding evaluation for omnimodal LLMs, 2026. URL https://arxiv.org/abs/2603.19217.
205. Tian et al. (2026) Jiayi Tian, Shiao Liu, Yuting Xu, Jia Lu, Zihao Guan, Honglin Han, Di Yang, Minqi Gu, Yifei Qian, Tianlin Zhang, Yanqing Zhu, Zeqian Ye, Menglin Yang, Fei Wang, Xu Hu, Xiuxian Li, Wei Zhang, Shihui Su, Yiyan Ji, Jingbo Wang, Ziteng Feng, Jiaheng Liu, Zhaoxiang Zhang, Xiaolong Wu, Zixiao Tang, Zhining Gu, Yang Cai, Linbo Zheng, Jingjing Ma, Mingyang Yin, Zedong Chu, Wenbin Tang, and Mu Xu. ABot-AgentOS: A general robotic agent OS with lifelong multi-modal memory. arXiv preprint arXiv:2607.10350, 2026. URL https://arxiv.org/abs/2607.10350.
206. Tu et al. (2026) Haoqin Tu, Jianwen Chen, Zijun Wang, Siwei Han, Juncheng Wu, Hardy Chen, Haonian Ji, Kaiwen Xiong, Jiaqi Liu, Peng Xia, Jieru Mei, Hongliang Fei, Jason Eshraghian, Zeyu Zheng, Yuyin Zhou, Huaxiu Yao, and Cihang Xie. Visualclaw: A real-time, personalized agent for the physical world, 2026. URL https://arxiv.org/abs/2606.16295.
207. Tulving (1972) Endel Tulving. Episodic and semantic memory. In Endel Tulving and Wayne Donaldson, editors, Organization of Memory, pages 381–403. Academic Press, New York, 1972.
208. Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html.
209. Wang et al. (2026a) Feng Wang, Canmiao Fu, Zhipeng Huang, Chen Li, Jing Lyu, and Ge Li. Cognitive-structured multimodal agent for multimodal understanding, generation, and editing, 2026a. URL https://arxiv.org/abs/2607.08497.
210. Wang et al. (2024a) Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research, 2024a. URL https://openreview.net/forum?id=ehfRiF0R3a.
211. Wang et al. (2024b) Jiuniu Wang, Zehua Du, Yuyuan Zhao, Bo Yuan, Kexiang Wang, Jian Liang, Yaxi Zhao, Yihen Lu, Gengliang Li, Junlong Gao, Xin Tu, and Zhenyu Guo. AesopAgent: Agent-driven evolutionary system on story-to-video production, 2024b. URL https://arxiv.org/abs/2403.07952.
212. Wang et al. (2024c) Junyang Wang, Haiyang Xu, Haitao Jia, Xi Zhang, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. Mobile-agent-v2: Mobile device operation assistant with effective navigation via multi-agent collaboration. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, 2024c.
213. Wang et al. (2026b) Junyang Wang, Haiyang Xu, Xi Zhang, Zhaoqing Zhu, Ming Yan, Jieping Ye, and Jitao Sang. STAMP: Training explicit memory for mobile GUI agents in controllable and scalable virtual environments, 2026b. URL https://arxiv.org/abs/2605.29324.
214. Wang and Zhang (2026) Kuan Wang and Chao Zhang. Memleak: Diagnosing information leaks in multimodal agent memory. In Advances in Neural Information Processing Systems, 2026. URL https://arxiv.org/abs/2606.29788. To appear.
215. Wang and Li (2026) Meiqi Wang and Shichao Li. Compiling and benchmarking task-state horizons for embodied agents, 2026. URL https://arxiv.org/abs/2608.08036.
216. Wang et al. (2026c) Pan Wang, Yihao Hu, Xiujin Liu, Jingchu Yang, Hang Wang, and Zhihao Wen. Atlasva: Self-evolving visual skill memory for teacher-free vlm agents, 2026c. URL https://arxiv.org/abs/2605.17933.
217. Wang et al. (2026d) Qian Wang, Ziqi Huang, Ruoxi Jia, Paul Debevec, and Ning Yu. MAViS: A multi-agent framework for long-sequence video storytelling. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2273–2295, Rabat, Morocco, 2026d. Association for Computational Linguistics. 10.18653/v1/2026.eacl-long.101.
218. Wang et al. (2026e) Qiuchen Wang, Shihang Wang, Yu Zeng, Qiang Zhang, Fanrui Zhang, Zhuoning Guo, Bosi Zhang, Wenxuan Huang, Lin Chen, Zehui Chen, Pengjun Xie, and Ruixue Ding. Vimrag: Navigating massive visual context in retrieval-augmented generation via multimodal memory graph, 2026e. URL https://arxiv.org/abs/2602.12735.
219. Wang et al. (2026f) Sen Wang, Bangwei Liu, Zhenkun Gao, Lizhuang Ma, Xuhong Wang, Yuan Xie, and Xin Tan. Explore with long-term memory: A benchmark and multimodal llm-based reinforcement learning framework for embodied exploration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 37098–37108, 2026f.
220. Wang et al. (2026g) Shijian Wang, Jiarui Jin, Runhao Fu, Zexuan Yan, Xingjian Wang, Mengkang Hu, Eric Wang, Xiaoxi Li, Kangning Zhang, Li Yao, Wenxiang Jiao, Xuelian Cheng, Yuan Lu, and Zongyuan Ge. Museagent: A multimodal reasoning agent with stateful experiences, 2026g. URL https://arxiv.org/abs/2603.27813.
221. Wang et al. (2026h) Weijie Wang, Haoyu Zhao, Yifan Yang, Feng Chen, Zeyu Zhang, Yefei He, Zicheng Duan, Donny Y. Chen, Yuqing Yang, and Bohan Zhuang. Latent spatial memory for video world models, 2026h. URL https://arxiv.org/abs/2606.09828.
222. Wang et al. (2026i) Xiaodong Wang, Xuanyi Zhao, Pedro Rodriguez, Devendra Singh Sachan, Barlas Oguz, Seungwhan Moon, Shang-Wen Li, Gargi Ghosh, Xin Dong, and Wen-Tau Yih. S-EMBER: A large-scale benchmark for streaming egocentric memory retrieval, 2026i. URL https://arxiv.org/abs/2607.02689.
223. Wang et al. (2026j) Yichuan Wang, Zhifei Li, Zirui Wang, Paul Teiletche, Lesheng Jin, Matei Zaharia, Joseph E. Gonzalez, and Sewon Min. Pixelrag: Web screenshots beat text for retrieval-augmented generation, 2026j. URL https://arxiv.org/abs/2606.28344.
224. Wang et al. (2026k) Yiran Wang, Chenyi Xiong, Ziyue Qin, Miao Zhang, Kui Xiao, and Zhifei Li. Hylovqa: Dynamic hypernetwork-generated low-rank adaptation for continual visual question answering. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, pages 1794–1802. International Joint Conferences on Artificial Intelligence Organization, 2026k. 10.24963/ijcai.2026/200.
225. Wang and Chen (2025) Yu Wang and Xi Chen. Mirix: Multi-agent memory system for llm-based agents, 2025. URL https://arxiv.org/abs/2507.07957.
226. Wang et al. (2025a) Yun Wang, Long Zhang, Jingren Liu, Jiaqi Yan, Zhanjie Zhang, Jiahao Zheng, Ao Ma, Run Ling, Xun Yang, Dapeng Wu, Xiangyu Chen, and Xuelong Li. Video-em: Event-centric episodic memory for long-form video understanding, 2025a. URL https://arxiv.org/abs/2508.09486.
227. Wang et al. (2025b) Yuxuan Wang, Yiqi Song, Cihang Xie, Yang Liu, and Zilong Zheng. Videollamb: Long streaming video understanding with recurrent memory bridges. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 24170–24181, 2025b. 10.1109/ICCV51701.2025.02240. URL https://openaccess.thecvf.com/content/ICCV2025/html/Wang_VideoLLaMB_Long_Streaming_Video_Understanding_with_Recurrent_Memory_Bridges_ICCV_2025_paper.html.
228. Wang et al. (2025c) Zixuan Wang, Bo Yu, Junzhe Zhao, Wenhao Sun, Sai Hou, Shuai Liang, Xing Hu, Yinhe Han, and Yiming Gan. KARMA: Augmenting embodied AI agents with long-and-short term memory systems. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 1–8. IEEE, 2025c. 10.1109/ICRA55743.2025.11128047. URL https://doi.org/10.1109/ICRA55743.2025.11128047.
229. Wang et al. (2026l) Ziyang Wang, Yue Zhang, Shoubin Yu, Ce Zhang, Zengqi Zhao, Jaehong Yoon, Hyunji Lee, Gedas Bertasius, and Mohit Bansal. Egomemreason: A memory-driven reasoning benchmark for long-horizon egocentric video understanding. In Third Conference on Language Modeling, 2026l. URL https://arxiv.org/abs/2605.09874. To appear.
230. Wang et al. (2026m) Zun Wang, Jialu Li, Han Lin, Jaehong Yoon, and Mohit Bansal. DreamRunner: Fine-grained compositional story-to-video generation with retrieval-augmented motion adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 10503–10511, 2026m. 10.1609/aaai.v40i13.38022.
231. Wei et al. (2026) Tianxin Wei, Ting-Wei Li, Zhining Liu, Xuying Ning, Ze Yang, Jiaru Zou, Zhichen Zeng, Ruizhong Qiu, Xiao Lin, Dongqi Fu, Zihao Li, Mengting Ai, Duo Zhou, Wenxuan Bao, Yunzhe Li, Gaotang Li, Cheng Qian, Yu Wang, Xiangru Tang, Yin Xiao, Liri Fang, Hui Liu, Xianfeng Tang, Yuji Zhang, Chi Wang, Jiaxuan You, Heng Ji, Hanghang Tong, and Jingrui He. A survey of agentic reasoning for large language models: Towards recursively self-improving and collective agents. Transactions on Machine Learning Research, 2026. URL https://arxiv.org/abs/2601.12538. To appear.
232. Wen and Bekris (2021) Bowen Wen and Kostas E. Bekris. BundleTrack: 6d pose tracking for novel objects without instance or category-level 3d models. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 8067–8074, 2021. 10.1109/IROS51168.2021.9635991. URL https://doi.org/10.1109/IROS51168.2021.9635991.
233. Wen et al. (2022) Bowen Wen, Wenzhao Lian, Kostas Bekris, and Stefan Schaal. You only demonstrate once: Category-level manipulation from single visual demonstration. In Robotics: Science and Systems, 2022. URL https://arxiv.org/abs/2201.12716.
234. Wen et al. (2023) Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas Müller, Alex Evans, Dieter Fox, Jan Kautz, and Stan Birchfield. BundleSDF: Neural 6-dof tracking and 3d reconstruction of unknown objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 606–617, 2023. URL https://openaccess.thecvf.com/content/CVPR2023/html/Wen_BundleSDF_Neural_6-DoF_Tracking_and_3D_Reconstruction_of_Unknown_Objects_CVPR_2023_paper.html.
235. Wen et al. (2024) Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu, Yaqin Zhang, and Yunxin Liu. Autodroid: Llm-powered task automation in android. In Proceedings of the 30th Annual International Conference on Mobile Computing and Networking, ACM MobiCom ’24, pages 543–557, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400704895. 10.1145/3636534.3649379. URL https://doi.org/10.1145/3636534.3649379.
236. Wen et al. (2026) Siwei Wen, Zhangcheng Wang, Xingjian Zhang, Lei Huang, and Wenjun Wu. EventMemAgent: Hierarchical event-centric memory for online video understanding with adaptive tool use. In Computer Vision – ECCV 2026, Lecture Notes in Computer Science, pages 318–334. Springer, 2026. 10.1007/978-3-032-37624-4_20.
237. Weng et al. (2024) Yijia Weng, Bowen Wen, Jonathan Tremblay, Valts Blukis, Dieter Fox, Leonidas Guibas, and Stan Birchfield. Neural implicit representation for building digital twins of unknown articulated objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3141–3150, 2024. URL https://openaccess.thecvf.com/content/CVPR2024/html/Weng_Neural_Implicit_Representation_for_Building_Digital_Twins_of_Unknown_Articulated_CVPR_2024_paper.html.
238. Weston et al. (2015) Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. In 3rd International Conference on Learning Representations, ICLR 2015, 2015. URL https://arxiv.org/abs/1410.3916.
239. Wu et al. (2026a) Jingwen Wu, Xijun Zhang, and Ge Song. Rethinking visual neglect: Steering via context-preference for mllm hallucination mitigation, 2026a. URL https://arxiv.org/abs/2605.27993.
240. Wu et al. (2026b) Peiran Wu, Yunze Liu, Chi-Hao Wu, Chen Chen, and Junxiao Shen. O-marc: Omni memory-augmented compression distillation for efficient video understanding, 2026b. URL https://arxiv.org/abs/2605.26584.
241. Wu et al. (2025a) Weijia Wu, Zeyu Zhu, and Mike Zheng Shou. Automated movie generation via multi-agent cot planning, 2025a. URL https://arxiv.org/abs/2503.07314.
242. Wu et al. (2025b) Wenyi Wu, Kun Zhou, Ruoxin Yuan, Vivian Yu, Stephen Wang, Zhiting Hu, and Biwei Huang. Auto-scaling continuous memory for gui agent, 2025b. URL https://arxiv.org/abs/2510.09038.
243. Wu et al. (2025c) Yaxiong Wu, Sheng Liang, Chen Zhang, Yichao Wang, Yongyue Zhang, Huifeng Guo, Ruiming Tang, and Yong Liu. From human memory to AI memory: A survey on memory mechanisms in the era of LLMs, 2025c. URL https://arxiv.org/abs/2504.15965.
244. Wu et al. (2022) Yuhuai Wu, Markus N. Rabe, DeLesley Hutchins, and Christian Szegedy. Memorizing transformers. In The Tenth International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=TrjbxzRcnf-.
245. Xi et al. (2025) Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and potential of large language model based agents: A survey. Science China information sciences, 68(2):121101, 2025.
246. Xia et al. (2025) Peng Xia, Kangyu Zhu, Haoran Li, Tianze Wang, Weijia Shi, Sheng Wang, Linjun Zhang, James Zou, and Huaxiu Yao. MMed-RAG: Versatile multimodal RAG system for medical vision-language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=s5epFPdIW6.
247. Xia et al. (2026a) Shao-Jun Xia, Yizhuo He, Jiashen Liu, Yuner Zhang, Yifan Jiang, Xiaoyang Chen, Liangxi Liu, Chenwen Luo, and Jinbao Wang. Memory in vision-language models: Taxonomy, mechanisms, and applications. Preprints, July 2026a. 10.20944/preprints202607.1539.v1. URL https://doi.org/10.20944/preprints202607.1539.v1.
248. Xia et al. (2026b) Tian Xia, Hanchen Li, Zhifei Li, Xiaokun Chen, Hao Kang, Yifan Qiao, Yi Xu, and Ion Stoica. Idleness is relative: Exploiting tool-call idle windows for offloading in agentic systems with mori. arXiv preprint arXiv:2606.00866, 2026b.
249. Xiao et al. (2026a) Han Xiao, Guozhi Wang, Hao Wang, Shilong Liu, Yuxiang Chai, Yue Pan, Yufeng Zhou, Xiaoxin Chen, Yafei Wen, and Hongsheng Li. UI-Mem: Self-evolving experience memory for online reinforcement learning in mobile GUI agents. arXiv preprint arXiv:2602.05832, 2026a. URL https://arxiv.org/abs/2602.05832.
250. Xiao et al. (2026b) Junbin Xiao, Jiajun Chen, Tianxiang Sun, Xun Yang, and Angela Yao. Mukv: Multi-grained kv cache compression for long streaming video question-answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11381–11391, 2026b.
251. Xie et al. (2025) Junlin Xie, Zhihong Chen, Ruifei Zhang, and Guanbin Li. Large multimodal agents: A survey. Visual Intelligence, 3(1):24, 2025. 10.1007/s44267-025-00093-y.
252. Xie et al. (2026a) Xianghui Xie, Bowen Wen, Yan Chang, Hesam Rabeti, Jiefeng Li, Ye Yuan, Gerard Pons-Moll, and Stan Birchfield. CARI4D: Category agnostic 4d reconstruction of human-object interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14006–14016, 2026a. URL https://openaccess.thecvf.com/content/CVPR2026/html/Xie_CARI4D_Category_Agnostic_4D_Reconstruction_of_Human-Object_Interaction_CVPR_2026_paper.html.
253. Xie et al. (2026b) Yiweng Xie, Bo He, Junke Wang, Xiangyu Zheng, Ziyi Ye, and Zuxuan Wu. Fluxmem: Adaptive hierarchical memory for streaming video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 31272–31282, 2026b. URL https://openaccess.thecvf.com/content/CVPR2026/html/Xie_FluxMem_Adaptive_Hierarchical_Memory_for_Streaming_Video_Understanding_CVPR_2026_paper.html.
254. Xie et al. (2026c) Zhifei Xie, Jiaqi Lang, Ze An, Yifan Zhao, Dongchao Yang, Kai Li, Ziyang Ma, Mingbao Lin, Chunyan Miao, and Shuicheng Yan. Voicemem: Streaming dual-brain memory for real-time interaction, 2026c. URL https://arxiv.org/abs/2608.26005.
255. Xiong et al. (2025) Haomiao Xiong, Zongxin Yang, Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Jiawen Zhu, and Huchuan Lu. Streaming video understanding and multi-round interaction with memory-enhanced knowledge. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=JbPb6RieNC.
256. Xu et al. (2025) Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-mem: Agentic memory for llm agents. In Advances in Neural Information Processing Systems (NeurIPS), volume 38, pages 17577–17604, 2025. 10.52202/085713-0593. URL https://proceedings.neurips.cc/paper_files/paper/2025/hash/19909c36f51abc4856b4560aff3d36d6-Abstract-Conference.html.
257. Xu et al. (2026) Yunzhe Xu, Yiyuan Pan, and Zhe Liu. Dream to recall: Imagination-guided experience retrieval for memory-persistent vision-and-language navigation. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 9035–9049, 2026. 10.1109/TPAMI.2026.3679426. URL https://doi.org/10.1109/TPAMI.2026.3679426.
258. Xue et al. (2025) Haochen Xue, Feilong Tang, Ming Hu, Yexin Liu, Qidong Huang, Yulong Li, Chengzhi Liu, Zhongxing Xu, Chong Zhang, et al. MMRC: A large-scale benchmark for understanding multimodal large language model in real-world conversation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, pages 22477–22503, 2025. 10.18653/v1/2025.acl-long.1096. URL https://aclanthology.org/2025.acl-long.1096/.
259. Yadav et al. (2026) Karmesh Yadav, Yusuf Ali, Gunshi Gupta, Yarin Gal, and Zsolt Kira. Findingdory: A benchmark to evaluate memory in embodied agents. In Computer Vision – ECCV 2026, Lecture Notes in Computer Science, pages 232–249. Springer, 2026. 10.1007/978-3-032-37086-0_13.
260. Yan et al. (2025a) Jiaqi Yan, Ruilong Ren, Jingren Liu, Shuning Xu, Ling Wang, Yiheng Wang, Xinlin Zhong, Yun Wang, Long Zhang, Xiangyu Chen, Changzhi Sun, Jixiang Luo, Dell Zhang, Hao Sun, Chi Zhang, and Xuelong Li. Teleego: Benchmarking egocentric AI assistants in the wild, 2025a. URL https://arxiv.org/abs/2510.23981.
261. Yan et al. (2026) Sikuan Yan, Sicheng Dong, Haotong Wang, Ercong Nie, Yilun Liu, Jinhe Bi, Yingjie Xu, Susanna Schwarzmann, Riccardo Trivisonno, Volker Tresp, and Yunpu Ma. Pyravid: Hierarchical multimodal memory for long-horizon video reasoning, 2026. URL https://arxiv.org/abs/2605.17065.
262. Yan et al. (2025b) Zhijie Yan, Shufei Li, Zuoxu Wang, Lixiu Wu, Han Wang, Jun Zhu, Lijiang Chen, and Jihong Liu. Dynamic open-vocabulary 3d scene graphs for long-term language-guided mobile manipulation. IEEE Robotics and Automation Letters (RA-L), 10(5):4252–4259, 2025b. 10.1109/LRA.2025.3547643. URL https://doi.org/10.1109/LRA.2025.3547643.
263. Yang et al. (2025a) Jingkang Yang, Shuai Liu, Hongming Guo, Yuhao Dong, Xiamengwei Zhang, Sicheng Zhang, Pengyun Wang, Zitang Zhou, et al. Egolife: Towards egocentric life assistant. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 28885–28900, 2025a. 10.1109/CVPR52734.2025.02690.
264. Yang et al. (2026) Yudong Yang, Guangzhi Sun, Yixuan Li, Wei Li, Zejun Ma, and Chao Zhang. SLVMBench: Skill learning from video memory. In Advances in Neural Information Processing Systems, 2026. URL https://arxiv.org/abs/2607.11312. To appear.
265. Yang et al. (2025b) Yuncong Yang, Han Yang, Jiachen Zhou, Peihao Chen, Hongxin Zhang, Yilun Du, and Chuang Gan. 3d-mem: 3d scene memory for embodied exploration and reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17294–17303, 2025b. 10.1109/CVPR52734.2025.01612.
266. Yao et al. (2025a) Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, and Junchen Jiang. CacheBlend: Fast large language model serving for RAG with cached knowledge fusion. In Proceedings of the Twentieth European Conference on Computer Systems, pages 94–109. ACM, 2025a. 10.1145/3689031.3696098. URL https://doi.org/10.1145/3689031.3696098.
267. Yao et al. (2025b) Yiqun Yao, Naitong Yu, Xiang Li, Xin Jiang, Xuezhi Fang, Wenjia Ma, Xuying Meng, Jing Li, Aixin Sun, and Yequan Wang. Egomem: Lifelong memory agent for full-duplex omnimodal models, 2025b. URL https://arxiv.org/abs/2509.11914.
268. Yeo et al. (2026a) Woongyeong Yeo, Kangsan Kim, Soyeong Jeong, Jinheon Baek, and Sung Ju Hwang. UniversalRAG: Retrieval-augmented generation over corpora of diverse modalities and granularities. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3843–3871, San Diego, California, United States, July 2026a. Association for Computational Linguistics. 10.18653/v1/2026.acl-long.177. URL https://aclanthology.org/2026.acl-long.177/.
269. Yeo et al. (2026b) Woongyeong Yeo, Kangsan Kim, Jaehong Yoon, and Sung Ju Hwang. WorldMM: Dynamic multimodal memory agent for long video reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 25599–25609, 2026b. URL https://openaccess.thecvf.com/content/CVPR2026/html/Yeo_WorldMM_Dynamic_Multimodal_Memory_Agent_for_Long_Video_Reasoning_CVPR_2026_paper.html.
270. Yin et al. (2026) Yufei Yin, Qianke Meng, Minghao Chen, Jiajun Ding, Zhenwei Shao, and Zhou Yu. Videoarm: Agentic reasoning over hierarchical memory for long-form video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24042–24051, 2026.
271. Yu et al. (2025a) Jiwen Yu, Jianhong Bai, Yiran Qin, Quande Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Xihui Liu. Context as memory: Scene-consistent interactive long video generation with memory retrieval. In SIGGRAPH Asia 2025 Conference Papers, pages 1–11. ACM, 2025a. 10.1145/3757377.3763833.
272. Yu et al. (2025b) Ming-Ming Yu, Fei Zhu, Wenzhuo Liu, Yirong Yang, Qunbo Wang, Wenjun Wu, and Jing Liu. C-nav: Towards self-evolving continual object navigation in open world. In Advances in Neural Information Processing Systems, volume 38, pages 126181–126207, 2025b. 10.52202/085713-4206. URL https://proceedings.neurips.cc/paper_files/paper/2025/hash/b78fef5ae6944767b9c1a5a2ae392cde-Abstract-Conference.html.
273. Yu et al. (2026a) Mingxian Yu, Siqi Luo, and Xu Chen. Graphpilot: Gui task automation with one-step llm reasoning powered by knowledge graph. Journal of Intelligent Computing and Networking, 2(1):13–21, 2026a. 10.64509/jicn.21.45.
274. Yu et al. (2025c) Shi Yu, Chaoyue Tang, Bokai Xu, Junbo Cui, Junhao Ran, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, and Maosong Sun. VisRAG: Vision-based retrieval-augmented generation on multi-modality documents. In The Thirteenth International Conference on Learning Representations, 2025c. URL https://openreview.net/forum?id=zG459X3Xge.
275. Yu et al. (2026b) Xinlei Yu, Chengming Xu, Zhangquan Chen, Bo Yin, Cheng Yang, Yongbo He, Yihao Hu, Jiangning Zhang, Cheng Tan, Xiaobin Hu, and Shuicheng Yan. Dual latent memory for visual multi-agent system. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 151378–151399. PMLR, 2026b. URL https://proceedings.mlr.press/v306/yu26bf.html.
276. Yu et al. (2026c) Xinlei Yu, Chengming Xu, Guibin Zhang, Zhangquan Chen, Yudong Zhang, Yongbo He, Peng-Tao Jiang, Jiangning Zhang, Xiaobin Hu, and Shuicheng Yan. Vismem: Latent vision memory unlocks potential of vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 31544–31555, 2026c. URL https://openaccess.thecvf.com/content/CVPR2026/html/Yu_VisMem_Latent_Vision_Memory_Unlocks_Potential_of_Vision-Language_Models_CVPR_2026_paper.html.
277. Yu et al. (2026d) Zihao Yu, Xiu Yuan, and Chongjie Zhang. MEMORA: Embodied action memory from egocentric videos for reasoning and planning, 2026d. URL https://arxiv.org/abs/2607.14252. Accepted to EMNLP 2026.
278. Zeng et al. (2026) Ziyun Zeng, Hang Hua, Bocheng Zou, Mu Cai, Rogerio Feris, and Jiebo Luo. Mementogui: Learning agentic multimodal memory control for long-horizon GUI agents, 2026. URL https://arxiv.org/abs/2605.18652.
279. Zhang et al. (2026a) Chenghao Zhang, Guanting Dong, Yufan Liu, Tong Zhao, Xiaoxi Li, and Zhicheng Dou. Towards verifiable multimodal deep research: A multi-agent harness for interleaved report generation, 2026a. URL https://arxiv.org/abs/2605.29861.
280. Zhang et al. (2025a) Chi Zhang, Zhao Yang, Jiaxuan Liu, Yanda Li, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. Appagent: Multimodal agents as smartphone users. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–20. ACM, 2025a. 10.1145/3706598.3713600.
281. Zhang et al. (2026b) Guibin Zhang, Yaohui Ling, Fanci Meng, Kun Wang, and Shuicheng Yan. Mem-w: Latent memory-native gui agents, 2026b. URL https://arxiv.org/abs/2605.09317.
282. Zhang et al. (2025b) Haoji Zhang, Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, and Xiaojie Jin. Flash-VStream: Efficient real-time understanding for long video streams. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 21059–21069, 2025b. 10.1109/ICCV51701.2025.01957. URL https://openaccess.thecvf.com/content/ICCV2025/html/Zhang_Flash-VStream_Efficient_Real-Time_Understanding_for_Long_Video_Streams_ICCV_2025_paper.html.
283. Zhang et al. (2025c) Hongxin Zhang, Zheyuan Zhang, Zeyuan Wang, Zunzhe Zhang, Lixing Fang, Qinhong Zhou, and Chuang Gan. Ella: Embodied social agents with lifelong memory, 2025c. URL https://arxiv.org/abs/2506.24019.
284. Zhang et al. (2024) Wentao Zhang, Lingxuan Zhao, Haochong Xia, Shuo Sun, Jiaze Sun, Molei Qin, Xinyi Li, Yuqing Zhao, Yilei Zhao, Xinyu Cai, Longtao Zheng, Xinrun Wang, and Bo An. A multimodal foundation agent for financial trading: Tool-augmented, diversified, and generalist. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4314–4325. Association for Computing Machinery, 2024. 10.1145/3637528.3671801. URL https://doi.org/10.1145/3637528.3671801.
285. Zhang et al. (2025d) Xiaoyi Zhang, Zhaoyang Jia, Zongyu Guo, Jiahao Li, Bin Li, Houqiang Li, and Yan Lu. Deep video discovery: Agentic search with tool use for long-form video understanding. In Advances in Neural Information Processing Systems, volume 38, 2025d. URL https://proceedings.neurips.cc/paper_files/paper/2025/hash/8190b210e9808e54ee16263b673a847d-Abstract-Conference.html.
286. Zhang et al. (2026c) Xichen Zhang, Guankai Li, Yinghao Zhu, Shijian Wang, Sitong Wu, Shaozuo Yu, Meng Chu, Yuan Lu, and Jiaya Jia. Streamarena: Toward continuous, interactive, and long-horizon agentic streaming video understanding, 2026c. URL https://arxiv.org/abs/2608.05703.
287. Zhang et al. (2026d) Yehang Zhang, Jianchong Su, Haojian Huang, Yifan Chang, Tianhao Zhou, Xinli Xu, Yingjie Xu, Yinchuan Li, Zexi Li, and Ying-Cong Chen. Worldlines: Benchmarking and modeling long-horizon stateful embodied agents, 2026d. URL https://arxiv.org/abs/2606.18847. Accepted to EMNLP 2026.
288. Zhang et al. (2026e) Yi Zhang, Jiawen Zhu, Lele Fu, and Guansong Pang. Anomalyagent: Training-free agentic models for zero-/few-shot anomaly detection, 2026e. URL https://arxiv.org/abs/2605.30140.
289. Zhang et al. (2025e) Zeyu Zhang, Quanyu Dai, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey on the memory mechanism of large language model based agents. ACM Transactions on Information Systems, 43(6):1–47, 2025e. 10.1145/3748302.
290. Zhao et al. (2025a) Canyu Zhao, Mingyu Liu, Wen Wang, Weihua Chen, Fan Wang, Hao Chen, Bo Zhang, and Chunhua Shen. MovieDreamer: Hierarchical generation for coherent long visual sequences. In The Thirteenth International Conference on Learning Representations, 2025a.
291. Zhao et al. (2025b) Shiju Zhao, Junhao Hu, Rongxiao Huang, Jiaqi Zheng, and Guihai Chen. Mpic: Position-independent multimodal context caching system for efficient mllm serving, 2025b. URL https://arxiv.org/abs/2502.01960.
292. Zheng et al. (2024) Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. SGLang: Efficient execution of structured language model programs. In Advances in Neural Information Processing Systems, volume 37, 2024. 10.52202/079017-2000. URL https://proceedings.neurips.cc/paper_files/paper/2024/hash/724be4472168f31ba1c9ac630f15dec8-Abstract-Conference.html.
293. Zhong et al. (2024) Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. MemoryBank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 19724–19731, 2024. 10.1609/aaai.v38i17.29946. URL https://doi.org/10.1609/aaai.v38i17.29946.
294. Zhu et al. (2026a) Hu Zhu, Bohan Li, Xianda Guo, Yanlun Peng, Hongsi Liu, Baorui Peng, Xiaofeng Wang, Mingqi Yuan, Xin Jin, Wenjun Zeng, and Chang Wen Chen. GEM-Occ: From visual geometry evidence to embodied semantic occupancy memory, 2026a. URL https://arxiv.org/abs/2607.05543.
295. Zhu et al. (2026b) Shiping Zhu, Yibo Yang, Zhengyang Wang, Tiancheng Shen, Dandan Guo, and Ming-Hsuan Yang. H2HMem: A multimodal memory benchmark for agents in human-human interactions, 2026b. URL https://arxiv.org/abs/2606.09461.
296. Zhu et al. (2026c) Sibo Zhu, Wenyi Wu, Kun Zhou, Stephen Wang, and Biwei Huang. Hybrid self-evolving structured memory for computer-use agents. In Findings of the Association for Computational Linguistics: ACL 2026, pages 11287–11304, San Diego, California, United States, 2026c. Association for Computational Linguistics. 10.18653/v1/2026.findings-acl.549. URL https://aclanthology.org/2026.findings-acl.549/.
297. Zou et al. (2026a) Tao Zou, Yichen He, Tian Qiu, Yuan Lin, and Hang Li. Task-focused memorization for multimodal agents, 2026a. URL https://arxiv.org/abs/2605.31075.
298. Zou et al. (2026b) Zhishan Zou, Guoyan Sun, Zhiwei Wei, Jiancheng Pan, Yujie Li, Mugen Peng, and Wenjia Xu. Self in space: Benchmarking self-awareness and spatial cognition in UAV embodied intelligence, 2026b. URL https://arxiv.org/abs/2607.12477.
299. Zuo et al. (2025) Jialong Zuo, Yongtai Deng, Lingdong Kong, Jingkang Yang, Rui Jin, Yiwei Zhang, Nong Sang, Liang Pan, Ziwei Liu, and Changxin Gao. Videolucy: Deep memory backtracking for long video understanding. In Advances in Neural Information Processing Systems (NeurIPS), volume 38, pages 25660–25691, 2025. 10.52202/085713-0864. URL https://proceedings.neurips.cc/paper_files/paper/2025/hash/24a2b1e2d4c3b20f09eb006d3d610777-Abstract-Conference.html.