RTMS: A Real-Time Multimodal Scaffolding System for Improving Debugging in Computing Education
Anahita Golrang, Kshitij Sharma
TLDR
A real-time multimodal system (RTMS) uses cognitive load and stress indicators to provide adaptive feedback, significantly improving debugging for students.
Key contributions
- RTMS delivers real-time multimodal feedback based on cognitive load and stress during debugging.
- Feedback significantly improved debugging success and efficiency for 120 CS students.
- Combined cognitive load and stress feedback produced the largest performance gains.
- The system effectively reduced the novice-expert debugging performance gap.
Why it matters
Debugging is a major challenge in programming education, especially for novices. This research demonstrates that real-time, adaptive feedback, triggered by cognitive and affective states, can significantly improve debugging performance. It also helps bridge the gap between novice and expert programmers, suggesting new directions for adaptive learning environments.
Original Abstract
Debugging is a demanding aspect of programming yet guidance on how to teach it effectively remains limited. Novices often struggle to recognize impasses regulate their problem solving and manage cognitive load and stress. This study investigates whether real time multimodal feedback triggered by indicators of cognitive load and physiological stress can improve debugging performance narrow expert novice gaps and reduce the influence of prior programming experience on success. We conducted a between subjects experiment with 120 undergraduate computer science students who debugged a medium sized Python program. Participants were assigned to one of four conditions no feedback cognitive load triggered feedback stress triggered feedback or combined trigger feedback. Eye tracking and heart rate variability data were used to detect moments of struggle and automatically deliver brief context sensitive hints. All three feedback conditions significantly improved debugging success and efficiency compared with the control group. Cognitive load triggered feedback produced stronger gains than stress triggered feedback and the combined trigger condition yielded the largest improvements. Programming expertise predicted performance only in the control condition and in all feedback conditions the novice expert gap was markedly reduced. Adaptive feedback that responds to learners cognitive and affective states can help manage debugging demands and reduce performance differences linked to prior experience highlighting opportunities for physiologically aware adaptive learning environments.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.