The Risk of Using the Q Heterogeneity Estimator for Software Engineering Experiments

Dieste Tubio, Oscar; Fernández, Enrique; García Martínez, Ramón y Juristo Juzgado, Natalia (2011). The Risk of Using the Q Heterogeneity Estimator for Software Engineering Experiments. En: "International Symposium on Empirical Software Engineering and Measurement, ESEM 2011", 19/09/2011 - 23/09/2011, Banff, Albert, Canada. ISBN 978-1-4577-2203-5. pp. 68-76.

Descripción

Título: The Risk of Using the Q Heterogeneity Estimator for Software Engineering Experiments
Autor/es:
  • Dieste Tubio, Oscar
  • Fernández, Enrique
  • García Martínez, Ramón
  • Juristo Juzgado, Natalia
Tipo de Documento: Ponencia en Congreso o Jornada (Artículo)
Título del Evento: International Symposium on Empirical Software Engineering and Measurement, ESEM 2011
Fechas del Evento: 19/09/2011 - 23/09/2011
Lugar del Evento: Banff, Albert, Canada
Título del Libro: Proceedings of International Symposium on Empirical Software Engineering and Measurement, ESEM 2011
Fecha: 2011
ISBN: 978-1-4577-2203-5
Materias:
Palabras Clave Informales: Meta-analysis , effect size , heterogeneity , reliability , statistical power , weighted mean difference (WMD)
Escuela: Facultad de Informática (UPM) [antigua denominación]
Departamento: Lenguajes y Sistemas Informáticos e Ingeniería del Software
Licencias Creative Commons: Reconocimiento - Sin obra derivada - No comercial

Texto completo

[img]
Vista Previa
PDF (Document Portable Format) - Se necesita un visor de ficheros PDF, como GSview, Xpdf o Adobe Acrobat Reader
Descargar (604kB) | Vista Previa

Resumen

All meta-analyses should include a heterogeneity analysis. Even so, it is not easy to decide whether a set of studies are homogeneous or heterogeneous because of the low statistical power of the statistics used (usually the Q test). Objective: Determine a set of rules enabling SE researchers to find out, based on the characteristics of the experiments to be aggregated, whether or not it is feasible to accurately detect heterogeneity. Method: Evaluate the statistical power of heterogeneity detection methods using a Monte Carlo simulation process. Results: The Q test is not powerful when the meta-analysis contains up to a total of about 200 experimental subjects and the effect size difference is less than 1. Conclusions: The Q test cannot be used as a decision-making criterion for meta-analysis in small sample settings like SE. Random effects models should be used instead of fixed effects models. Caution should be exercised when applying Q test-mediated decomposition into subgroups.

Más información

ID de Registro: 11569
Identificador DC: http://oa.upm.es/11569/
Identificador OAI: oai:oa.upm.es:11569
URL Oficial: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=6092555&contentType=Conference+Publications&sortType%3Dasc_p_Sequence%26filter%3DAND%28p_IS_Number%3A6092538%29
Depositado por: Memoria Investigacion
Depositado el: 19 Jul 2012 10:53
Ultima Modificación: 20 Abr 2016 19:36
  • Open Access
  • Open Access
  • Sherpa-Romeo
    Compruebe si la revista anglosajona en la que ha publicado un artículo permite también su publicación en abierto.
  • Dulcinea
    Compruebe si la revista española en la que ha publicado un artículo permite también su publicación en abierto.
  • Recolecta
  • e-ciencia
  • Observatorio I+D+i UPM
  • OpenCourseWare UPM