Forward Stagewise Naive Bayes

Vidaurre Henche, Diego and Bielza, Concha and Larrañaga Múgica, Pedro María (2012). Forward Stagewise Naive Bayes. "Progress in Artificial Intelligence", v. 1 (n. 1); pp. 57-69. ISSN 978-3-642-04685-8.


Title: Forward Stagewise Naive Bayes
  • Vidaurre Henche, Diego
  • Bielza, Concha
  • Larrañaga Múgica, Pedro María
Item Type: Article
Título de Revista/Publicación: Progress in Artificial Intelligence
Date: April 2012
ISSN: 978-3-642-04685-8
Volume: 1
Faculty: Facultad de Informática (UPM)
Department: Inteligencia Artificial
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (1MB) | Preview


The naïve Bayes approach is a simple but often satisfactory method for supervised classification. In this paper, we focus on the naïve Bayes model and propose the application of regularization techniques to learn a naïve Bayes classifier. The main contribution of the paper is a stagewise version of the selective naïve Bayes, which can be considered a regularized version of the naïve Bayes model. We call it forward stagewise naïve Bayes. For comparison’s sake, we also introduce an explicitly regularized formulation of the naïve Bayes model, where conditional independence (absence of arcs) is promoted via an L 1/L 2-group penalty on the parameters that define the conditional probability distributions. Although already published in the literature, this idea has only been applied for continuous predictors. We extend this formulation to discrete predictors and propose a modification that yields an adaptive penalization. We show that, whereas the L 1/L 2 group penalty formulation only discards irrelevant predictors, the forward stagewise naïve Bayes can discard both irrelevant and redundant predictors, which are known to be harmful for the naïve Bayes classifier. Both approaches, however, usually improve the classical naïve Bayes model’s accuracy.

More information

Item ID: 10996
DC Identifier:
OAI Identifier:
DOI: 10.1007/s13748-011-0001-7
Official URL:
Deposited by: Memoria Investigacion
Deposited on: 05 Jun 2012 09:23
Last Modified: 20 Apr 2016 19:10
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM