A process for managing interaction between experimenters to get useful similar replications

Juristo Juzgado, Natalia; Vegas Hernandez, Sira; Solari Buela, Martín; Abrahao, Silvia y Ramos, Isabel (2013). A process for managing interaction between experimenters to get useful similar replications. "Information and Software Technology", v. 55 (n. 2); pp. 215-225. ISSN 0950-5849. https://doi.org/10.1016/j.infsof.2012.07.016.

Descripción

Título: A process for managing interaction between experimenters to get useful similar replications
Autor/es:
  • Juristo Juzgado, Natalia
  • Vegas Hernandez, Sira
  • Solari Buela, Martín
  • Abrahao, Silvia
  • Ramos, Isabel
Tipo de Documento: Artículo
Título de Revista/Publicación: Information and Software Technology
Fecha: 2013
Volumen: 55
Materias:
Palabras Clave Informales: Empirical studies - Experimentation - Replication - Combination of experiment results
Escuela: Facultad de Informática (UPM) [antigua denominación]
Departamento: Lenguajes y Sistemas Informáticos e Ingeniería del Software
Licencias Creative Commons: Reconocimiento - Sin obra derivada - No comercial

Texto completo

[img]
Vista Previa
PDF (Document Portable Format) - Se necesita un visor de ficheros PDF, como GSview, Xpdf o Adobe Acrobat Reader
Descargar (1MB) | Vista Previa

Resumen

Context: A replication is the repetition of an experiment. Several efforts have been made to adopt replication as a common practice in software engineering. There are different types of replications, depending on their purpose. Similar replications keep the experimental conditions as alike as possible to the original ones. External similar replications, where the replicating experimenters are not the same people as the original experimenters, have been a stumbling block. Several attempts at combining the results of replications have resulted in failure. Software engineering does not appear to be well suited to such replications, because it works with complex experimentally immature contexts. Software engineering settings have a large number of variables, and the role that many of them play is unknown. A successful (or useful) similar replication helps to better understand the phenomenon under study by verifying results and/or identifying contextual variables that could influence (or not) the results, through the combination of experimental results. Objective: To be able to get successful similar replications, there needs to be interaction between original and replicating experimenters. In this paper, we propose an interaction process for achieving successful similar replications. Method: This process consists of: an adaptation meeting, where experimenters tailor the experiment to the new setting; querying, to settle occasional inquiries while the experiment is being run; and a combination meeting, where experimenters meet to discuss the combination of replication outcomes with previous results. To check its effectiveness, the process has been tested on three different replications of the same experiment. Results: The proposed interaction process has helped to identify new contextual variables that could potentially influence (or not) the experimental results in the three replications run. Additionally, the interaction process has helped to uncover certain problems and deviations that occurred during some of the replications that we would have not been aware of otherwise. Conclusions: There are signs that suggest that it is possible to get successful similar replications in soft- ware engineering experimentation, when there is appropriate interaction among experimenters.

Más información

ID de Registro: 26024
Identificador DC: http://oa.upm.es/26024/
Identificador OAI: oai:oa.upm.es:26024
Identificador DOI: 10.1016/j.infsof.2012.07.016
URL Oficial: http://www.sciencedirect.com/science/article/pii/S0950584912001425
Depositado por: Memoria Investigacion
Depositado el: 05 Jun 2014 09:31
Ultima Modificación: 04 Dic 2017 10:59
  • Open Access
  • Open Access
  • Sherpa-Romeo
    Compruebe si la revista anglosajona en la que ha publicado un artículo permite también su publicación en abierto.
  • Dulcinea
    Compruebe si la revista española en la que ha publicado un artículo permite también su publicación en abierto.
  • Recolecta
  • e-ciencia
  • Observatorio I+D+i UPM
  • OpenCourseWare UPM