Keep information that may only become relevant to a future task.
EmbodiedMemory-Bench
EmbodiedMemory-Bench: Benchmarking Embodied Memory
for Long-Horizon Embodied Tasks
Remember the world. Act beyond the moment. 记得住，才走得远。
OmniAI Group of ZJU ACES Lab1 Zhejiang University2 Central South University3 Institute of Software, Chinese Academy of Sciences
Motivation
Why Do Embodied Agents Need Memory?
Long-horizon interaction requires more than recognizing the current scene. Agents must connect what they have seen, what they have done, and what happened next.
Update locations and states as observations and interactions unfold.
Use action outcomes and corrections to guide what happens next.
Understanding the failures
When memory fails,
actions fail too.
We manually inspected 100 failed trajectories on EmbodiedBench. Four recurring deficiencies reveal what an embodied agent needs to remember.
91%
of the inspected failures fall into these four memory categories.
Analysis of Gemini-3-Flash and Qwen3-VL-32B on EB-ALF and EB-Hab.
Read the abstract
Long-horizon embodied interaction requires agents to retain and continually update information about the environment as they observe, act, and encounter change. Yet current agents struggle to maintain such memory reliably. Our analysis traces this limitation to four key deficiencies: weak fine-grained visual memory, unreliable dynamic world-state tracking, failing to record world state revealed by interaction outcomes, and limited generalization from prior experience. However, existing benchmarks do not directly assess these memory capabilities during long-horizon embodied interaction. To address this gap, we introduce EmbodiedMemory-Bench (EMem-Bench), comprising 2,554 interactive episodes across four task families. EMem-Bench requires agents to build and update memory from interaction history, then use it to complete a later task by acting in the environment. We further present Embodied-Memorizer (EMem), an external memory system that organizes embodied experience into spatial, event, and scene memories. We also train EMem-8B, an 8B policy that manages and uses these memories. We evaluate a diverse range of open-source and proprietary MLLMs and representative multimodal memory systems. Results show that current models remain weak and uneven across the four challenges. Under matched backbones, EMem achieves the best overall performance among the evaluated memory systems and improves both open-source and proprietary models, while EMem-8B further improves over its backbone.
EMem-Bench
Four Challenges for Embodied Memory
Build memory from interaction history, then use it to complete a later task in the environment.
2,554Episodes
4Task families
1,118Scenes
Explore the benchmark
From a Past Observation to the Right Action
Choose a memory challenge and follow an illustrative example from the paper.
FINE-GRAINED VISUAL MEMORY
Remember the detail before it matters.
The future task is unknown during observation. Can the agent retain an object’s location amid a busy scene and unrelated activity?
OBSERVED HISTORYTime →
THE LATER TASK
Benchmark construction
Grounded in a World the Agent Can Act In
Multi-room scenes, task-specific memory cues, and distractor trajectories. Every retained episode passes automated execution checks and manual review.
Embodied-Memorizer
Three Memories. One Continually Updated World.
EMem organizes embodied experience into complementary scene, spatial, and event memories.
Scene Memory
Ground observations in their rooms and scenes. Distinguish the mug in the kitchen from the one in the bedroom.
Spatial Memory
Maintain the latest locations and states of objects. Replace outdated relations when the world changes.
Event Memory
Retain actions, feedback, and corrections. Consolidate recurring experiences into reusable knowledge.
Observe→Write→Retrieve→Act↺
EMem-8B learns to write, retrieve, and use these memories through supervised fine-tuning of Qwen3-VL-8B.
Results
EMem-Bench Leaderboard
Memory improves task success across matched backbones.
All reported results
| # | Model / system | Setting |
|---|
Outlined scores: best in the reported full-set results
WorseBetter
SR ↑ Success Rate. ERR ↓ Error Recurrence Rate. Average gives equal weight to each task family. Rank follows average SR; clicking a heading changes display order.
* Robotics-ER 1.5 uses a 10% subset due to API instability. It is shown for reference and excluded from full-set ranks and best-score highlights. Full-context baselines use text history and the final visual observation.
Matched-backbone improvements
What Changes When We Add Memory?
Absolute gains in average success rate over each corresponding full-context backbone.
35.3 → 58.9 average SR
40.9 → 57.3 average SR
25.4 → 46.0 average SR
Resources
Explore. Evaluate. Build on EMem.
Inspect the dataset, explore the method, and build on the released code and evaluation resources.
Paper
Benchmark design, memory architecture, experiments, and supplementary material.
Read on arXiv →GitHub
Task construction, evaluation, memory tools, and inference utilities.
Open on GitHub →Dataset
All 2,554 episodes, frozen manifests, metadata, and verification reports.
Open on Hugging Face →Get started
Try the Memory Demo
From the provided code directory. No model, simulator, API key, or GPU required.
Terminal
python -m pip install -e .
ememCitation
Reference
Lizhou Liang, Xinyu Zhong, Miao Pan, Xiaohe Zhou, Xuanyu Liu, Qinfeng Li, Peng Li, Jintao Chen, Xuhong Zhang, Wenqi Zhang. EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Tasks. 2026. arXiv:2609.28236.