A saliency-based attention LSTM model for cognitive load classification from speech

Gallardo Antolín, Ascensión and Montero Martínez, Juan Manuel (2019). A saliency-based attention LSTM model for cognitive load classification from speech. In: "20th Annual Conference of the International Speech Communication Association (ISCA 2019)", 15/09/2919 - 19/09/2019, Graz, Austria. pp. 216-220. https://doi.org/10.21437/Interspeech.2019-1603.

Description

Title: A saliency-based attention LSTM model for cognitive load classification from speech
Author/s:
  • Gallardo Antolín, Ascensión
  • Montero Martínez, Juan Manuel
Item Type: Presentation at Congress or Conference (Article)
Event Title: 20th Annual Conference of the International Speech Communication Association (ISCA 2019)
Event Dates: 15/09/2919 - 19/09/2019
Event Location: Graz, Austria
Title of Book: 20th Annual Conference of the International Speech Communication Association (ISCA 2019)
Date: 2019
Subjects:
Freetext Keywords: Cognitive load; speech; LSTM; weigthed pooling; auditory saliency; attention model
Faculty: E.T.S.I. Telecomunicación (UPM)
Department: Ingeniería Electrónica
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

[img]
Preview
PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (348kB) | Preview

Abstract

Cognitive Load (CL) refers to the amount of mental demand that a given task imposes on an individual’s cognitive system and it can affect his/her productivity in very high load situations. In this paper, we propose an automatic system capable of classifying the CL level of a speaker by analyzing his/her voice. Our research on this topic goes into two main directions. In the first one, we focus on the use of Long Short-Term Memory (LSTM) networks with different weighted pooling strategies for CL level classification. In the second contribution, for overcoming the need of a large amount of training data, we propose a novel attention mechanism that uses the Kalinli’s auditory saliency model. Experiments show that our proposal outperforms significantly both, a baseline system based on Support Vector Machines (SVM) and a LSTM-based system with logistic regression attention model.

Funding Projects

TypeCodeAcronymLeaderTitle
Government of SpainTEC2017-84395-PUnspecifiedUnspecifiedUnspecified
Government of SpainTEC2017-84593-C2-1-RUnspecifiedUnspecifiedInferencia de la respuesta afectiva de los espectadores de un video

More information

Item ID: 65335
DC Identifier: https://oa.upm.es/65335/
OAI Identifier: oai:oa.upm.es:65335
DOI: 10.21437/Interspeech.2019-1603
Official URL: https://www.isca-speech.org/archive/Interspeech_2019/pdfs/1603.pdf
Deposited by: Memoria Investigacion
Deposited on: 12 Apr 2021 15:22
Last Modified: 12 Apr 2021 15:22
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM