GenAI Workflows
Co-designing Characters and Storyboards with GenAI and people with Parkinson's
2024-2025
Prompt Engineering, Human-AI interaction, Digital Health, Storyboarding, TORUS project
GenAI tools like Midjourney can rapidly produce large volumes of visual content from text prompts, including photorealistic scenes with multiple consistent characters. As such, they are being actively explored as tools for character and storyboard creation by UX professionals and researchers. In principle, GenAI tools offers clear opportunities for cognitive offloading and job effort reduction to rapidly create materials for design teams. Moreover, as GenAI opens new avenues for collaborative content generation through co-shaping prompts and output, further opportunities for teamwork emerge that include non-experts.
As part of doing research within digital the health domain, I involved people living with Parkinson’s in co-creating GenAI characters (personas) to ensure their values and experiences are better reflected, leading to richer, more respectful and empowering depictions.
We co-designed these GenAI-generated characters with the purpose of developing an interactive storyboard depicting different interaction scenarios of a smart home monitoring system (as part of EPSRC-funded TORUS project). In total, I generated ~2900 images, with ~85 of them featured in the final storyboards used to conduct online studies with Parkinson’s patients and carers. The storyboards feature a day in the life of each character, and different potential interactions with the system - including voice control via voice, wearable reminders, data logging/journalling in a dedicated smart phone app. The purpose of the tool and the associated studies were to investigate acceptability of different interaction scenarios, discuss what interaction options seem more appropriate, and unpack any ethical concerns (privacy, data processing) that might surface amongst participants.
We interviewed 15 participants using the tool, and deployed it as an online survey (using Maze to record user flows) capturing insights from another 10 participants. Results from the studies formed design recommendations to guide the design of TORUS smart home health monitoring system.
Above: The interactive storyboard (Figma) used as a tool to conduct online studies, investigating interaction scenarios with the home monitoring system. The storyboard was essentially an online survey, whereby one can nagivate different interaction scenarios using buttons and leave feedback on an embedded survey.
Above: Short video-clip showcasing part of the storyboard
Myself and my co-authors author a paper (in process), presenting key findings from the three-week design experiment of generating the characters (personas) and the storyboards, reflecting on key learnings from the design process and unpacking some of the complexities at play in the use of GenAI tools for character and storyboard creation. We seek to form implications for cognitive tools and workflows using GenAI for characters and storyboards.
In the paper, we discuss how cognitive loading/ offloading took place within different content generation phases; and talk about AI hallucination and how it impacted cognitive load, design fixation, and creativity. Moreover, we unpack the dynamics and complexities of human-AI collaboration as a creative co-regulation process that evolved as part of employing diverse prompting and tool-use strategies. We reflect on how GenAI can support, distort, or reconfigure human values, and the potential cognitive biases at play. We highlight the mismatch between human values, intentions, and the intuitive understanding of social context, and the language and output of the algorithm, and point to potential solutions and tools to mitigate issues.
Above: A diagram showing the amount of content generated and key design phases.
Above: An example of MidJourney prompting exploration.