A controlled experiment on time pressure and confirmation bias in functional software testing

Salman, Iflaah and Turhan, Burak and Vegas Hernández, Sira (2019). A controlled experiment on time pressure and confirmation bias in functional software testing. "Empirical Software Engineering", v. 24 (n. 4); pp. 1727-1761. ISSN 1382-3256. https://doi.org/10.1007/s10664-018-9668-8.

Description

Title: A controlled experiment on time pressure and confirmation bias in functional software testing
Author/s:
  • Salman, Iflaah
  • Turhan, Burak
  • Vegas Hernández, Sira
Item Type: Article
Título de Revista/Publicación: Empirical Software Engineering
Date: 2019
ISSN: 1382-3256
Volume: 24
Subjects:
Freetext Keywords: Test quality; Software testing; Cognitive biases; Confirmation bias; Time pressure; Experiment; Human factors
Faculty: E.T.S. de Ingenieros Informáticos (UPM)
Department: Lenguajes y Sistemas Informáticos e Ingeniería del Software
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

[img]
Preview
PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (2MB) | Preview

Abstract

Context. Confirmation bias is a person’s tendency to look for evidence that strengthens his/her prior beliefs rather than refutes them. Manifestation of confirmation bias in software testing may have adverse effects on software quality. Psychology research suggests that time pressure could trigger confirmation bias. Objective. In the software industry, this phenomenon may deteriorate software quality. In this study, we investigate whether testers manifest confirmation bias and how it is affected by time pressure in functional software testing. Method. We performed a controlled experiment with 42 graduate students to assess manifestation of confirmation bias in terms of the conformity of their designed test cases to the provided requirements specification. We employed a one factor with two treatments between-subjects experimental design. Results. We observed, overall, participants designed significantly more confirmatory test cases as compared to disconfirmatory ones, which is in line with previous research. However, we did not observe time pressure as an antecedent to an increased rate of confirmatory testing behaviour. Conclusion. People tend to design confirmatory test cases regardless of time pressure. For practice, we find it necessary that testers develop self-awareness of confirmation bias and counter its potential adverse effects with a disconfirmatory attitude. We recommend further replications to investigate the effect of time pressure as a potential contributor to the manifestation of confirmation bias.

More information

Item ID: 63981
DC Identifier: http://oa.upm.es/63981/
OAI Identifier: oai:oa.upm.es:63981
DOI: 10.1007/s10664-018-9668-8
Official URL: https://link.springer.com/article/10.1007%2Fs10664-018-9668-8
Deposited by: Memoria Investigacion
Deposited on: 22 Oct 2020 09:54
Last Modified: 22 Oct 2020 09:54
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM