The Role of LLMs in Collaborative Software Design

Original Abstract

While much prior work examines Large Language Models (LLMs) for solo development tasks (e.g., coding), far less is known about how LLMs shape collaborative group work in software engineering. This study focuses on one such collaborative task, namely software design. It presents the results of an exploratory laboratory study of 18 pairs of software professionals who could use an LLM however they saw fit, to design a University campus bicycle parking application. Our findings reveal that introducing an LLM leads to distinct patterns of joint use: shared-instance use facilitated shared understanding, whereas parallel use across separate instances sometimes led to ''context drift''. We also observe wide variation in reliance, from non-use to treating the LLM as an information source or producer. Across these modes, professionals scrutinized and reflected on LLM responses, often yielding design insights; however, early anchoring sometimes curtailed exploration. We provide implications for tools to aid designers while retaining the human-centricity important to design.