Effectiveness for detecting faults within and outside the scope of testing techniques: an independent replication

Apa, Cecilia and Dieste Tubio, Oscar and Espinosa Gallardo, Edison Gonzalo and Fonseca Carrera, Efrain Rodrigo (2013). Effectiveness for detecting faults within and outside the scope of testing techniques: an independent replication. "Empirical Software Engineering", v. 19 (n. 2); pp. 378-417. ISSN 1382-3256. https://doi.org/10.1007/s10664-013-9267-7.

Description

Title: Effectiveness for detecting faults within and outside the scope of testing techniques: an independent replication
Author/s:
  • Apa, Cecilia
  • Dieste Tubio, Oscar
  • Espinosa Gallardo, Edison Gonzalo
  • Fonseca Carrera, Efrain Rodrigo
Item Type: Article
Título de Revista/Publicación: Empirical Software Engineering
Date: 2013
ISSN: 1382-3256
Volume: 19
Subjects:
Freetext Keywords: Replication · Experiment · Unit testing ·Reporting guidelines
Faculty: Facultad de Informática (UPM)
Department: Lenguajes y Sistemas Informáticos e Ingeniería del Software
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

[img]
Preview
PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (1MB)

Abstract

The verification and validation activity plays a fundamental role in improving software quality. Determining which the most effective techniques for carrying out this activity are has been an aspiration of experimental software engineering researchers for years. This paper reports a controlled experiment evaluating the effectiveness of two unit testing techniques (the functional testing technique known as equivalence partitioning (EP) and the control-flow structural testing technique known as branch testing (BT)). This experiment is a literal replication of Juristo et al. (2013).Both experiments serve the purpose of determining whether the effectiveness of BT and EP varies depending on whether or not the faults are visible for the technique (InScope or OutScope, respectively). We have used the materials, design and procedures of the original experiment, but in order to adapt the experiment to the context we have: (1) reduced the number of studied techniques from 3 to 2; (2) assigned subjects to experimental groups by means of stratified randomization to balance the influence of programming experience; (3) localized the experimental materials and (4) adapted the training duration. We ran the replication at the Escuela Politécnica del Ejército Sede Latacunga (ESPEL) as part of a software verification & validation course. The experimental subjects were 23 master?s degree students. EP is more effective than BT at detecting InScope faults. The session/program andgroup variables are found to have significant effects. BT is more effective than EP at detecting OutScope faults. The session/program and group variables have no effect in this case. The results of the replication and the original experiment are similar with respect to testing techniques. There are some inconsistencies with respect to the group factor. They can be explained by small sample effects. The results for the session/program factor are inconsistent for InScope faults.We believe that these differences are due to a combination of the fatigue effect and a technique x program interaction. Although we were able to reproduce the main effects, the changes to the design of the original experiment make it impossible to identify the causes of the discrepancies for sure. We believe that further replications closely resembling the original experiment should be conducted to improve our understanding of the phenomena under study.

More information

Item ID: 26305
DC Identifier: http://oa.upm.es/26305/
OAI Identifier: oai:oa.upm.es:26305
DOI: 10.1007/s10664-013-9267-7
Official URL: http://link.springer.com/article/10.1007%2Fs10664-013-9267-7
Deposited by: Memoria Investigacion
Deposited on: 21 May 2014 09:24
Last Modified: 17 Jan 2017 08:11
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM