2021-04-20T22:27:42Z
http://oa.upm.es/cgi/oai2
oai:oa.upm.es:11569
2016-04-20T19:36:11Z
7374617475733D707562
7375626A656374733D74656C65636F6D756E69636163696F6E6573
7375626A656374733D696E666F726D6174696361
747970653D636F6E666572656E63655F6974656D
The Risk of Using the Q Heterogeneity Estimator for Software Engineering Experiments
Dieste Tubio, Oscar
Fernández, Enrique
García Martínez, Ramón
Juristo Juzgado, Natalia
Telecommunications
Computer Science
All meta-analyses should include a heterogeneity analysis. Even so, it is not easy to decide whether a set of studies are homogeneous or heterogeneous because of the low statistical power of the statistics used (usually the Q test). Objective: Determine a set of rules enabling SE researchers to find out, based on the characteristics of the experiments to be aggregated, whether or not it is feasible to accurately detect heterogeneity. Method: Evaluate the statistical power of heterogeneity detection methods using a Monte Carlo simulation process. Results: The Q test is not powerful when the meta-analysis contains up to a total of about 200 experimental subjects and the effect size difference is less than 1. Conclusions: The Q test cannot be used as a decision-making criterion for meta-analysis in small sample settings like SE. Random effects models should be used instead of fixed effects models. Caution should be exercised when applying Q test-mediated decomposition into subgroups.
Facultad de Informática (UPM)
http://creativecommons.org/licenses/by-nc-nd/3.0/es/
2011
info:eu-repo/semantics/conferenceObject
Presentation at Congress or Conference
Proceedings of International Symposium on Empirical Software Engineering and Measurement, ESEM 2011 | International Symposium on Empirical Software Engineering and Measurement, ESEM 2011 | 19/09/2011 - 23/09/2011 | Banff, Albert, Canada
PeerReviewed
application/pdf
eng
http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=6092555&contentType=Conference+Publications&sortType%3Dasc_p_Sequence%26filter%3DAND%28p_IS_Number%3A6092538%29
info:eu-repo/semantics/openAccess
http://oa.upm.es/11569/