Multimodal collaborative interactions for medical training in virtual reality

Fastelli, Francesco (2021). Multimodal collaborative interactions for medical training in virtual reality. Thesis (Master thesis), E.T.S. de Ingenieros Informáticos (UPM).


Title: Multimodal collaborative interactions for medical training in virtual reality
  • Fastelli, Francesco
  • Chellali, Amine
  • Román Díez, Guillermo
Item Type: Thesis (Master thesis)
Masters title: Digital Innovation: Human Computer Interaction and Design
Date: 20 October 2021
Faculty: E.T.S. de Ingenieros Informáticos (UPM)
Department: Lenguajes y Sistemas Informáticos e Ingeniería del Software
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (2MB) | Preview


The purpose of this work was to explore multimodal collaboration techniques to improve teaching technical medical skills. The methodology used was a user-centered design approach. Local anesthesia was chosen as a case study. After four observation sessions in the hospital, it was possible to identify three different modalities used to teach the skills needed for local anesthesia: voice, gestural and haptic. Based on that, a research question was defined: "Which modality among voice, gestural and haptic is the most effective to teach spatial information in a VR collaborative training system?". To address this research question, a prototype of an immersive virtual environment for teaching spatial information was developed. Users could interact with the prototype using a haptic device. Twenty-one volunteers participated in a user study to assess the impact of the teacher-learner communication modality on learning spatial information. The analysis of the results shows that instructions given through the gestural modality permitted the participants to reach a distance estimation error significantly lower than the two other modalities. On the other hand, receiving the instructions through a haptic modality helped participants perform the task faster. This suggests that users were more confident with this modality. At the same time, the co-presence questionnaire results show that users found the voice modality to be more accurate for a collaborative experience. In the questionnaire on comparison of modalities, the gestural and haptic modalities were perceived as more adapted to learning and memorizing spatial information. These results contribute to understanding the importance of the suitable modality for transferring spatial technical skills in a healthcare learning environment. Besides, the research suggests that a combination of modalities could be the most appropriate way to teach technical skills such as spatial awareness by improving both performance and user experience. This opens new research perspectives for further investigations on the effectiveness of multimodal interaction to support technical medical skills teaching in VR.

More information

Item ID: 69116
DC Identifier:
OAI Identifier:
Deposited by: Biblioteca Facultad de Informatica
Deposited on: 23 Nov 2021 14:30
Last Modified: 23 Nov 2021 14:30
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM