ESCAPE: Episodic Spatial Memory and Adaptive Execution Policy for Long-Horizon Mobile Manipulation
Jingjing Qian, Zeyuan He, Chen Shi, Lei Xiao, Li Jiang
TLDR
ESCAPE uses episodic spatial memory and an adaptive policy for robust long-horizon mobile manipulation, achieving SOTA on ALFRED.
Key contributions
- Spatio-Temporal Fusion Mapping creates persistent, depth-free 3D spatial memory for robust perception.
- Memory-Driven Target Grounding generates precise interaction masks using the spatial memory.
- Adaptive Execution Policy dynamically combines global navigation and local manipulation for flexible action.
- Achieves state-of-the-art performance on ALFRED, improving efficiency and robustness in long-horizon tasks.
Why it matters
This paper tackles the critical challenge of long-horizon mobile manipulation in complex environments, addressing issues like forgetting and rigid execution. ESCAPE's novel memory and adaptive policy significantly advance embodied AI, enabling more robust and efficient task completion. Its strong performance on ALFRED highlights its potential for real-world robotic applications.
Original Abstract
Coordinating navigation and manipulation with robust performance is essential for embodied AI in complex indoor environments. However, as tasks extend over long horizons, existing methods often struggle due to catastrophic forgetting, spatial inconsistency, and rigid execution. To address these issues, we propose ESCAPE (Episodic Spatial Memory Coupled with an Adaptive Policy for Execution), operating through a tightly coupled perception-grounding-execution workflow. For robust perception, ESCAPE features a Spatio-Temporal Fusion Mapping module to autoregressively construct a depth-free, persistent 3D spatial memory, alongside a Memory-Driven Target Grounding module for precise interaction mask generation. To achieve flexible action, our Adaptive Execution Policy dynamically orchestrates proactive global navigation and reactive local manipulation to seize opportunistic targets. ESCAPE achieves state-of-the-art performance on the ALFRED benchmark, reaching 65.09% and 60.79% success rates in test seen and unseen environments with step-by-step instructions. By reducing redundant exploration, our ESCAPE attains substantial improvements in path-length-weighted metrics and maintains robust performance (61.24% / 56.04%) even without detailed guidance for long-horizon tasks.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.