Calculating classifier calibration performance with a custom modification of Weka

Zlotnik Enaliev, Alexander; Gallardo Antolín, Ascensión y Montero Martínez, Juan Manuel (2014). Calculating classifier calibration performance with a custom modification of Weka. En: "Proceedings of the 4th International Conference on Integrated Information", 05/09/2014 - 08/09/2014, Madrid, Spain. pp. 128-133.

Descripción

Título: Calculating classifier calibration performance with a custom modification of Weka
Autor/es:
  • Zlotnik Enaliev, Alexander
  • Gallardo Antolín, Ascensión
  • Montero Martínez, Juan Manuel
Tipo de Documento: Ponencia en Congreso o Jornada (Artículo)
Título del Evento: Proceedings of the 4th International Conference on Integrated Information
Fechas del Evento: 05/09/2014 - 08/09/2014
Lugar del Evento: Madrid, Spain
Título del Libro: International Conference on Integrated Information (IC-ININFO 2014)
Fecha: 2014
Volumen: 1644
Materias:
Escuela: E.T.S.I. Telecomunicación (UPM)
Departamento: Electrónica Física
Licencias Creative Commons: Reconocimiento - Sin obra derivada - No comercial

Texto completo

[img]
Vista Previa
PDF (Document Portable Format) - Se necesita un visor de ficheros PDF, como GSview, Xpdf o Adobe Acrobat Reader
Descargar (174kB) | Vista Previa

Resumen

Calibration is often overlooked in machine-learning problem-solving approaches, even in situations where an accurate estimation of predicted probabilities, and not only a discrimination between classes, is critical for decision-making. One of the reasons is the lack of readily available open-source software packages which can easily calculate calibration metrics. In order to provide one such tool, we have developed a custom modification of the Weka data mining software, which implements the calculation of Hosmer-Lemeshow groups of risk and the Pearson chi-square statistic comparison between estimated and observed frequencies for binary problems. We provide calibration performance estimations with Logistic regression (LR), BayesNet, Naïve Bayes, artificial neural network (ANN), support vector machine (SVM), knearest neighbors (KNN), decision trees and Repeated Incremental Pruning to Produce Error Reduction (RIPPER) models with six different datasets. Our experiments show that SVMs with RBF kernels exhibit the best results in terms of calibration, while decision trees, RIPPER and KNN are highly unlikely to produce well-calibrated models.

Proyectos asociados

TipoCódigoAcrónimoResponsableTítulo
Gobierno de EspañaDPI2010-21247-C02-02Sin especificarSin especificarSin especificar

Más información

ID de Registro: 37518
Identificador DC: http://oa.upm.es/37518/
Identificador OAI: oai:oa.upm.es:37518
Depositado por: Memoria Investigacion
Depositado el: 22 May 2017 16:55
Ultima Modificación: 22 May 2017 16:55
  • Open Access
  • Open Access
  • Sherpa-Romeo
    Compruebe si la revista anglosajona en la que ha publicado un artículo permite también su publicación en abierto.
  • Dulcinea
    Compruebe si la revista española en la que ha publicado un artículo permite también su publicación en abierto.
  • Recolecta
  • e-ciencia
  • Observatorio I+D+i UPM
  • OpenCourseWare UPM