Prop-Chromeleon: Adaptive Haptic Props in Mixed Reality through Generative Artificial Intelligence
Haoyu Wang, Fengyuan Zhu, Bingjian Huang, Zhecheng Wang, Ludwig Sidenmark
TLDR
Prop-Chromeleon uses generative AI to transform everyday objects into adaptive haptic props in Mixed Reality, enhancing realism and immersion.
Key contributions
- Introduces Prop-Chromeleon, an MR system using generative AI for adaptive haptic props.
- Dynamically transforms everyday objects into passive haptic props via user text prompts.
- AI pipeline generates and anchors virtual assets that align with physical prop shapes.
- User studies confirm improved perceived realism, immersion, and enjoyment in MR experiences.
Why it matters
This paper addresses a core challenge in Mixed Reality: the lack of consistent haptic feedback. Prop-Chromeleon's generative AI approach allows for dynamic, shape-aware haptic props, significantly boosting realism and user engagement. This innovation paves the way for more believable and interactive MR experiences.
Original Abstract
Mixed Reality (MR) aims to blend digital and physical worlds, but the absence of haptic feedback often breaks visual-tactile consistency. We introduce Prop-Chromeleon, a MR system based on generative artificial intelligence (AI) that dynamically transforms everyday objects into adaptive passive haptic props through user-provided text prompts. Our AI pipeline performs generation and anchoring of virtual assets that align with the shape of physical props, allowing us to study how virtual content generation behaves under geometric and prompt-based constraints. We evaluate Prop-Chromeleon's effectiveness through a generation study using varied object shapes and user prompts, combining quantitative shape similarity metrics with qualitative prompt fidelity analysis. Our user study further showcases Prop-Chromeleon's improvements in perceived realism, immersion, and enjoyment compared to static baselines. These results show that shape-aware generation can support both believable haptic interaction and creative engagement in MR.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.