Speech to sign language translation system for Spanish

San Segundo Hernández, Rubén ORCID: https://orcid.org/0000-0001-9659-5464, Barra Chicote, Roberto, Córdoba Herralde, Ricardo de ORCID: https://orcid.org/0000-0002-7136-9636, D'Haro Enríquez, Luis Fernando ORCID: https://orcid.org/0000-0002-3411-7384, Fernández Martínez, Fernando ORCID: https://orcid.org/0000-0003-3877-0089, Ferreiros López, Javier ORCID: https://orcid.org/0000-0001-8834-3080, Lucas Cuesta, Juan Manuel, Macías Guarasa, Javier, Montero Martínez, Juan Manuel ORCID: https://orcid.org/0000-0002-7908-5400 and Pardo Muñoz, José Manuel ORCID: https://orcid.org/0000-0002-1009-590X (2008). Speech to sign language translation system for Spanish. "Speech Communication", v. 50 (n. 11); pp. 1009-1020. ISSN 0167-6393. https://doi.org/10.1016/j.specom.2008.02.001.


Title: Speech to sign language translation system for Spanish
Item Type: Article
Título de Revista/Publicación: Speech Communication
Date: November 2008
ISSN: 0167-6393
Volume: 50
Freetext Keywords: Spanish Sign Language (LSE), Spoken language translation, Sign animation.
Faculty: E.T.S.I. Telecomunicación (UPM)
Department: Ingeniería Electrónica
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

[thumbnail of INVE_MEM_2008_52866.pdf]
PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (3MB) | Preview


This paper describes the development of and the first experiments in a Spanish to sign language translation system in a real domain. The developed system focuses on the sentences spoken by an official when assisting people applying for, or renewing their Identity Card. The system translates official explanations into Spanish Sign Language (LSE: Lengua de Signos Espan¿ola) for Deaf people. The translation system is made up of a speech recognizer (for decoding the spoken utterance into a word sequence), a natural language translator (for converting a word sequence into a sequence of signs belonging to the sign language), and a 3D avatar animation module (for playing back the hand movements). Two proposals for natural language translation have been evaluated: a rule-based translation module (that computes sign confidence measures from the word confidence measures obtained in the speech recognition module) and a statistical translation module (in this case, parallel corpora were used for training the statistical model). The best configuration reported 31.6% SER (Sign Error Rate) and 0.5780 BLEU (BiLingual Evaluation Understudy). The paper also describes the eSIGN 3D avatar animation module (considering the sign confidence), and the limitations found when implementing a strategy for reducing the delay between the spoken utterance and the sign sequence animation.

More information

Item ID: 2139
DC Identifier: https://oa.upm.es/2139/
OAI Identifier: oai:oa.upm.es:2139
DOI: 10.1016/j.specom.2008.02.001
Official URL: http://www.sciencedirect.com/science/journal/01676...
Deposited by: Memoria Investigacion
Deposited on: 02 Feb 2010 11:49
Last Modified: 22 Mar 2023 16:28
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM