Proposing a speech to gesture translation architecture for Spanish deaf people.

San Segundo Hernández, Rubén and Montero Martínez, Juan Manuel and Macías Guarasa, Javier and Córdoba Herralde, Ricardo de and Ferreiros López, Javier and Pardo Muñoz, José Manuel (2008). Proposing a speech to gesture translation architecture for Spanish deaf people.. "Journal of Visual Languages and Computing", v. 19 (n. 5); pp. 523-538. ISSN 1045-926X. https://doi.org/10.1016/j.jvlc.2007.06.002.

Description

Title: Proposing a speech to gesture translation architecture for Spanish deaf people.
Author/s:
  • San Segundo Hernández, Rubén
  • Montero Martínez, Juan Manuel
  • Macías Guarasa, Javier
  • Córdoba Herralde, Ricardo de
  • Ferreiros López, Javier
  • Pardo Muñoz, José Manuel
Item Type: Article
Título de Revista/Publicación: Journal of Visual Languages and Computing
Date: October 2008
ISSN: 1045-926X
Volume: 19
Subjects:
Freetext Keywords: Animation, animated agents, lifelike characters, Spanish Sign Language, speech to gesture translation, gesture complexity
Faculty: E.T.S.I. Telecomunicación (UPM)
Department: Ingeniería Electrónica
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

[img]
Preview
PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (1MB) | Preview

Abstract

This article describes an architecture for translating speech into Spanish Sign Language (SSL). The architecture proposed is made up of four modules: speech recognizer, semantic analysis, gesture sequence generation and gesture playing. For the speech recognizer and the semantic analysis modules, we use software developed by IBM and CSLR (Center for Spoken Language Research at University of Colorado), respectively. Gesture sequence generation and gesture animation are the modules on which we have focused our main effort. Gesture sequence generation uses semantic concepts (obtained from the semantic analysis) associating them with several SSL gestures. This association is carried out based on a number of generation rules. For gesture animation, we have developed an animated agent (virtual representation of a human person) and a strategy for reducing the effort in gesture animation. This strategy consists of making the system automatically generate all agent positions necessary for the gesture animation. In this process, the system uses a few main agent positions (two or three per second) and some interpolation strategies, both issues previously generated by the service developer (the person who adapts the architecture proposed in this paper to a specific domain). Related to this module, we propose a distance between agent positions and a measure of gesture complexity. This measure can be used to analyze the gesture perception versus its complexity. With the architecture proposed, we are not trying to build a domain independent translator but a system able to translate speech utterances into gesture sequences in a restricted domain: railway, flights or weather information.

More information

Item ID: 2140
DC Identifier: http://oa.upm.es/2140/
OAI Identifier: oai:oa.upm.es:2140
DOI: 10.1016/j.jvlc.2007.06.002
Official URL: http://www.sciencedirect.com/science/journal/1045926X
Deposited by: Memoria Investigacion
Deposited on: 02 Feb 2010 11:01
Last Modified: 29 Sep 2014 18:23
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM