Emotion transplantation through adaptation in HMM-based speech synthesis

Lorenzo Trueba, Jaime and Barra Chicote, Roberto and San Segundo Hernández, Rubén and Ferreiros López, Javier and Yamagishi, Junichi and Montero Martínez, Juan Manuel (2015). Emotion transplantation through adaptation in HMM-based speech synthesis. "Computer Speech & Language", v. 34 (n. 1); pp. 292-307. ISSN 0885-2308.


Title: Emotion transplantation through adaptation in HMM-based speech synthesis
  • Lorenzo Trueba, Jaime
  • Barra Chicote, Roberto
  • San Segundo Hernández, Rubén
  • Ferreiros López, Javier
  • Yamagishi, Junichi
  • Montero Martínez, Juan Manuel
Item Type: Article
Título de Revista/Publicación: Computer Speech & Language
Date: November 2015
Volume: 34
Freetext Keywords: Statistical parametric speech synthesis; Expressive speech synthesis; Cascade adaptation; Emotion transplantation
Faculty: E.T.S.I. Telecomunicación (UPM)
Department: Ingeniería Electrónica
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

[img] PDF - Users in campus UPM only until November 2017 - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (7MB)


This paper proposes an emotion transplantation method capable of modifying a synthetic speech model through the use of CSMAPLR adaptation in order to incorporate emotional information learned from a different speaker model while maintaining the identity of the original speaker as much as possible. The proposed method relies on learning both emotional and speaker identity information by means of their adaptation function from an average voice model, and combining them into a single cascade transform capable of imbuing the desired emotion into the target speaker. This method is then applied to the task of transplanting four emotions (anger, happiness, sadness and surprise) into 3 male speakers and 3 female speakers and evaluated in a number of perceptual tests. The results of the evaluations show how the perceived naturalness for emotional text significantly favors the use of the proposed transplanted emotional speech synthesis when compared to traditional neutral speech synthesis, evidenced by a big increase in the perceived emotional strength of the synthesized utterances at a slight cost in speech quality. A final evaluation with a robotic laboratory assistant application shows how by using emotional speech we can significantly increase the students’ satisfaction with the dialog system, proving how the proposed emotion transplantation system provides benefits in real applications.

Funding Projects

Comunidad de MadridS2009/TIC-1542MA2VICMRUnspecifiedUnspecified

More information

Item ID: 40458
DC Identifier: http://oa.upm.es/40458/
OAI Identifier: oai:oa.upm.es:40458
Deposited by: Memoria Investigacion
Deposited on: 23 May 2016 16:50
Last Modified: 23 May 2016 16:52
  • Open Access
  • Open Access
  • Sherpa-Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Recolecta
  • e-ciencia
  • Observatorio I+D+i UPM
  • OpenCourseWare UPM