Just an error? Prototyping embodied experiences of LLM hallucinations

Large language model (LLM) hallucination is an intrinsic flaw leading to the response deviating from factual knowledge and conversational context. This thesis explores the method of prototyping the complex interplay of human interaction and LLM hallucinations in embodied medium, focusing on reflecting hallucination’s impact on user experience and socio-technical implications with research through design (RtD) method, especially addressing the new potential of multimodality and speculative design in conveying discourse. The research creates three prototypes, Moodie Assistant, Whisper Web, and Mindscape, to probe the manifestations of LLM hallucinations in various contexts. The thesis uses these prototypes to conduct paired interviews and workshops to capture user reactions and encourage reflective and creative thinking on the algorithmic feature. The the?sis challenges the conventional view of LLM hallucinations as mere technical glitches, proposing a deeper investigation into their interactive characteristics. It also expands the methodology of algorithm experience prototyping by gearing towards design speculation and including a broader spectrum of embodied mediums. Both contributions lead to design insights for designers and researchers as a reference for engaging with LLM hallucination and negative algorithm experience in their design/research journey