Lenia Margariti, PhD

Human-Robot Interactions

Designing Conversational Experiences for a Retail Service Robot
2018-2019

Duration: 4 deployments spanning accross 12 months

Role: Digital Transformation Analyst / UX Designer, COSMOTE-T

Team: IT D&I - Software Engineers, Marketing

Methods: Agile Sprints, Prototyping, Human-Robot Interaction Design, Voice Interaction design, Usability Testing, Deployment, Observation / Field Study

Impact: Designed and evaluated multimodal conversational experiences for a humanoid retail robot, combining voice, gesture, and touchscreen interactions to improve customer engagement and in-store assistance. Research insights informed the development of more natural and accessible human-robot interactions for real-world retail deployments. Accelerated growth through POCs design, development and evaluation, informing future product/service development.

Overview

As part of the COSMOTE-T IT Innovation Team, I contributed to the design and evaluation of customer-facing interactions for Pepper, a humanoid service robot deployed in telecommunications retail stores.

Pepper is a customisable humanoid Robot provided by Softbank. Pepper supports programming intuitive interactions that involve natural language, gestures, face recognition and screen interaction. In COSMOTE-T IT Innovation Team, we customized the Robot to engage, entertain and assist customers in retail shops. In collaboration with Athena Research center, our team was working towards integrating Pepper services with a custom NLP to improve speech recognition and support a wider range of conversations in Greek language with diverse users, as well as improving gestural and voice response sequences.

The project explored how conversational AI, multimodal interactions, and social robotics could improve customer engagement, assist with information discovery, and create memorable in-store experiences. Working alongside software engineers and researchers from Athena Research Center, we developed and tested interactions that combined voice, gesture, facial recognition, and touchscreen interfaces.

Our aim was ultimately the long term deployment of Pepper in key COSMOTE-T stores to support and entertain customers - e.g. advertise products, provide instructions on where to find a product and answer questions on prices and special offers.



Above: A member of our team interacting with the robot.

The Challenge

Retail customers have diverse needs, expectations, and levels of comfort when interacting with emerging technologies. The challenge was to design intuitive interactions that would help customers engage with the robot naturally while supporting practical retail tasks such as product discovery, store navigation, and promotional communication.

An additional challenge involved designing interactions that worked effectively in Greek, requiring improvements to speech recognition, conversational flows, and multimodal feedback.

My Role

I worked as a Human-Robot Interaction and Voice Interaction Designer, contributing to both product design and user research activities.

My responsibilities included:

  • Designing conversational flows and dialogue structures.
  • Creating tablet-based User Interfaces and content.
  • Developing and testing interaction logic using Python.
  • Supporting deployment, testing, and debugging activities.
  • Conducting field observations and customer interviews to evaluate user experiences.





Above: Human-Robot Choreography. Examples from deployments at the flagiship store and in the central office building, and our efforts in designing interactions that appeal to kids. Verbal consent was obtained for the images, blurred for de-identification purposes.

Research & Design Approach

To understand how people interacted with the robot in real retail environments, I conducted observational studies and short contextual interviews during deployments. These activities helped identify usability challenges, behavioural patterns, and opportunities for improving engagement.

One area of focus was multimodal interaction design. Through iterative testing, I explored how voice and touchscreen interactions could work together to improve task completion, user confidence, and engagement.

I also investigated how children interacted with the robot, designing playful experiences and interaction patterns that incorporated gestures, movement imitation, and simple voice-based games.

Impact

The project informed the design of customer-facing conversational experiences for retail environments and contributed to the development of more natural, engaging, and accessible human-robot interactions. Research findings directly influenced dialogue design, multimodal interaction patterns, and engagement strategies used in pilot deployments across COSMOTE-T retail locations.