PenGUIn

Scope of the Project

In January 2024, RE:LAB was selected among 54 participating organisations for the successful submission of the PenGUIn project, part of UTTER’s Financial Support to Third Parties – First Open Call. UTTER (Unified Transcription and Translation for Extended Reality, https://he-utter.eu/ ) is an Horizon Europe project focused on advancing online and hybrid interaction through extended reality. UTTER is developing conversation platforms where human agents interact with the assistance of AI agents, demonstrated in two primary use cases: a meeting assistant and a multilingual customer support assistant.

The PenGUIn project, spanning 9 months, aims to enhance user experience through an innovative, inclusive, adaptive, and usable Graphical User Interface (GUI) for XR platforms. It seeks to study the most appropriate information design framework to support agent tasks and manage the cognitive load in UTTER’s use cases and beyond, such as virtual learning and virtual healthcare.

Objectives of the Project

The main goal of PenGUIn is to converge innovation, usability, and content design to achieve functionality, effectiveness, and ergonomics, aligning with RE:LAB’s methodology, “Interaction Engineering.” The project will design and test a user-centred approach to prototype a library of graphical elements that are intuitive, cross-cutting, and compatible with usability criteria.

PenGUIn will support transparent, inclusive, and task-oriented dialogue and interaction between agents and users on virtual platforms. This will be achieved by considering various use cases to develop adaptable XR interaction logics. The approach will simulate use in different contexts, examine interaction and communication types between operators and users, integrate emotion-tracking and AI inputs, and manage adaptable information and content based on detected emotions. It will also focus on customization flexibility and validate the developed UI through expert analysis, focus groups, and iterative user testing.

The project will work on two use cases to develop a novel User Interface:

– Multilingual customer service platform

– Multilingual virtual meeting assistant

RE:LAB will lead the evaluation of interactions through expert analysis, focus groups, and iterative user testing. The graphic design development will ensure a product tailored to different interaction needs while maintaining a consistent and standardizable structure.