Linear Bayes policy for learning in contextual-bandits

Vargas Perez, Ana Maria and Martín H., José Antonio (2013). Linear Bayes policy for learning in contextual-bandits. "Expert Systems with Applications", v. 40 (n. 18); pp. 7400-7406. ISSN 0957-4174.


Title: Linear Bayes policy for learning in contextual-bandits
  • Vargas Perez, Ana Maria
  • Martín H., José Antonio
Item Type: Article
Título de Revista/Publicación: Expert Systems with Applications
Date: December 2013
ISSN: 0957-4174
Volume: 40
Freetext Keywords: Contextual bandits; Online advertising; Recommender systems; One-to-one Marketing; Empirical Bayes
Faculty: E.T.S.I. Industriales (UPM)
Department: Ingeniería de Organización, Administración de Empresas y Estadística
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (7MB) | Preview


Machine and Statistical Learning techniques are used in almost all online advertisement systems. The problem of discovering which content is more demanded (e.g. receive more clicks) can be modeled as a multi-armed bandit problem. Contextual bandits (i.e., bandits with covariates, side information or associative reinforcement learning) associate, to each specific content, several features that define the “context” in which it appears (e.g. user, web page, time, region). This problem can be studied in the stochastic/statistical setting by means of the conditional probability paradigm using the Bayes’ theorem. However, for very large contextual information and/or real-time constraints, the exact calculation of the Bayes’ rule is computationally infeasible. In this article, we present a method that is able to handle large contextual information for learning in contextual-bandits problems. This method was tested in the Challenge on Yahoo! dataset at ICML2012’s Workshop “new Challenges for Exploration & Exploitation 3”, obtaining the second place. Its basic exploration policy is deterministic in the sense that for the same input data (as a time-series) the same results are obtained. We address the deterministic exploration vs. exploitation issue, explaining the way in which the proposed method deterministically finds an effective dynamic trade-off based solely in the input-data, in contrast to other methods that use a random number generator.

More information

Item ID: 26245
DC Identifier:
OAI Identifier:
DOI: 10.1016/j.eswa.2013.07.041
Official URL:
Deposited by: Memoria Investigacion
Deposited on: 20 Jan 2015 15:29
Last Modified: 01 Jan 2016 23:56
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM