Evidence of the presence of bias in subjective metrics: analysis within a family of experiments

Aranda López King, Alejandrina and Juristo Juzgado, Natalia and Dieste Tubio, Oscar (2014). Evidence of the presence of bias in subjective metrics: analysis within a family of experiments. In: "18th International Conference on Evaluation and Assessment in Software Engineering (EASE 2014)", 13-14 May 2014, Londres, Reino Unido. ISBN 978-1-4503-2476-2. pp. 1-4.

Description

Title: Evidence of the presence of bias in subjective metrics: analysis within a family of experiments
Author/s:
  • Aranda López King, Alejandrina
  • Juristo Juzgado, Natalia
  • Dieste Tubio, Oscar
Item Type: Presentation at Congress or Conference (Other)
Event Title: 18th International Conference on Evaluation and Assessment in Software Engineering (EASE 2014)
Event Dates: 13-14 May 2014
Event Location: Londres, Reino Unido
Title of Book: EASE '14: proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering
Date: 2014
ISBN: 978-1-4503-2476-2
Subjects:
Freetext Keywords: Reliability; Validity; Objective and subjective measurements; Experiment; Quasi-experiment; Correlational study
Faculty: E.T.S. de Ingenieros Informáticos (UPM)
Department: Lenguajes y Sistemas Informáticos e Ingeniería del Software
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

[img]
Preview
PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (392kB) | Preview

Abstract

Context: Measurement is crucial and important to empirical software engineering. Although reliability and validity are two important properties warranting consideration in measurement processes, they may be influenced by random or systematic error (bias) depending on which metric is used. Aim: Check whether, the simple subjective metrics used in empirical software engineering studies are prone to bias. Method: Comparison of the reliability of a family of empirical studies on requirements elicitation that explore the same phenomenon using different design types and objective and subjective metrics. Results: The objectively measured variables (experience and knowledge) tend to achieve more reliable results, whereas subjective metrics using Likert scales (expertise and familiarity) tend to be influenced by systematic error or bias. Conclusions: Studies that predominantly use variables measured subjectively, like opinion polls or expert opinion acquisition.

Funding Projects

TypeCodeAcronymLeaderTitle
Government of SpainTIN 2011-23216UnspecifiedUnspecifiedUnspecified

More information

Item ID: 37491
DC Identifier: http://oa.upm.es/37491/
OAI Identifier: oai:oa.upm.es:37491
Official URL: http://dl.acm.org/citation.cfm?id=2601291
Deposited by: Memoria Investigacion
Deposited on: 29 Sep 2015 11:02
Last Modified: 14 Nov 2017 10:07
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM