Premover: Fast Vision-Language-Action Control by Acting Before Instructions Are Complete
Joonha Park, Jiseung Jeong, Taesik Gong
TLDR
Premover speeds up Vision-Language-Action policies by enabling robots to start acting before user instructions are fully complete, reducing idle time.
Key contributions
- Converts idle time during user instruction input into useful precomputation.
- Uses projection heads to create a focus map, reweighting image tokens for next steps.
- A trained readiness threshold decides when the VLA policy should begin acting.
- Reduces VLA task completion time by 13.6% on LIBERO, matching baseline success.
Why it matters
Current VLA policies waste time waiting for full user instructions. Premover solves this by enabling proactive computation during input, significantly boosting interaction efficiency. This approach maintains high success rates, making robots more responsive in real-world scenarios.
Original Abstract
Vision-Language-Action (VLA) policies are typically evaluated as if the user had finished typing or speaking before the robot begins acting. In real deployment, however, users take several seconds to enter a request, leaving the policy idle for a substantial fraction of the interaction. We introduce Premover, a lightweight module that converts this idle window into useful precomputation. Premover keeps the VLA backbone frozen and attaches two small projection heads, one for image patches, one for language tokens, that map an intermediate layer of the backbone into a shared space. The resulting focus map is supervised by simulator-rendered target-object segmentation masks and applied as a per-patch reweighting of the next step's image tokens. A single scalar readiness threshold, trained jointly from streaming prefixes, decides when the policy should begin acting. On the LIBERO benchmark suite, Premover reduces mean wall-clock time from 34.0 to 29.4 seconds, a 13.6% reduction, while matching the full-prompt baseline's success rate (95.1% vs. 95.0%); naive premoving, by contrast, collapses to 66.4%.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.