On (Mis) perceptions of testing effectiveness: an empirical study

Vegas Hernández, Sira and Riofrío Ojeda, Patricia and Marcos Martínez, Esperanza and Juristo Juzgado, Natalia (2020). On (Mis) perceptions of testing effectiveness: an empirical study. "Empirical Software Engineering", v. 25 (n. 2); pp. 2844-2896. ISSN 1382-3256. https://doi.org/10.1007/s10664-020-09805-y.


Title: On (Mis) perceptions of testing effectiveness: an empirical study
  • Vegas Hernández, Sira
  • Riofrío Ojeda, Patricia
  • Marcos Martínez, Esperanza
  • Juristo Juzgado, Natalia
Item Type: Article
Título de Revista/Publicación: Empirical Software Engineering
Date: 2020
ISSN: 1382-3256
Volume: 25
Freetext Keywords: Developers perceptions; Testing technique effectiveness; Software testing
Faculty: E.T.S. de Ingenieros Informáticos (UPM)
Department: Lenguajes y Sistemas Informáticos e Ingeniería del Software
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (2MB) | Preview


A recurring problem in software development is incorrect decision making on the techniques, methods and tools to be used. Mostly, these decisions are based on developers' perceptions about them. A factor influencing people's perceptions is past experience, but it is not the only one. In this research, we aim to discover how well the perceptions of the defect detection effectiveness of different techniques match their real effectiveness in the absence of prior experience. To do this, we conduct an empirical study plus a replication. During the original study, we conduct a controlled experiment with students applying two testing techniques and a code review technique. At the end of the experiment, they take a survey to find out which technique they perceive to be most effective. The results show that participants' perceptions are wrong and that this mismatch is costly in terms of quality. In order to gain further insight into the results, we replicate the controlled experiment and extend the survey to include questions about participants' opinions on the techniques and programs. The results of the replicated study confirm the findings of the original study and suggest that participants' perceptions might be based not on their opinions about complexity or preferences for techniques but on how well they think that they have applied the techniques.

Funding Projects

Government of SpainPGC2018-097265-B-I00UnspecifiedUniversidad Politécnica de MadridMejora de la calidad de los experimentos en ingeniería de software para el incremento de la fiabilidad en los resultados
Government of SpainTIN2017-88557-RUnspecifiedUniversidad Rey Juan CarlosMarco metodológico centrado en el usuario para el diseño de servicios en la nueva sociedad digital
Madrid Regional GovernmentS2018/TCS-4314FORTE-CMUniversidad Autónoma de MadridFORmal models and technologies for emerging applications

More information

Item ID: 57758
DC Identifier: https://oa.upm.es/57758/
OAI Identifier: oai:oa.upm.es:57758
DOI: 10.1007/s10664-020-09805-y
Official URL: https://link.springer.com/article/10.1007%2Fs10664-020-09805-y
Deposited by: Memoria Investigacion
Deposited on: 20 Oct 2020 06:22
Last Modified: 30 Nov 2022 09:00
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM