InvestChat: Exploring Multimodal Interaction via Natural Language, Touch, and Pen in an Investment Dashboard
Sarah Lykke Tost, Adson Lucas de Paiva Sales, Henrik Østergaard, Vaishali Dhanoa, Gabriela Molina León
TLDR
InvestChat is a multimodal tablet app combining natural language, touch, and pen for stock market exploration, enhancing user engagement.
Key contributions
- Developed InvestChat, a tablet application for stock market exploration with an LLM-powered chat.
- Integrates natural language, touch, and pen input for a rich multimodal interaction experience.
- Evaluation with novice investors showed enhanced user engagement and complementary use of modalities.
- Participants found natural language most effective and valued the freedom of choice among input methods.
Why it matters
This paper introduces InvestChat, demonstrating how multimodal interaction can significantly improve user engagement in complex data exploration tasks like stock analysis. It highlights the power of combining natural language with touch and pen, offering valuable insights for future UI/UX design in financial tools.
Original Abstract
We designed and implemented InvestChat, a multimodal tablet-based application that supports stock market exploration with multiple coordinated views and an LLM-powered chat. We evaluated the application with 12 novice investors. Our findings suggest that combining natural language, touch, and pen input during stock market exploration facilitates user engagement. Participants leveraged the modalities in complementary ways, enjoying the freedom of choice and finding natural language most effective.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.