A unified framework for linear function approximation of value functions in stochastic control

Sánchez Fernández, Matilde and Valcarcel Macua, Sergio and Zazo Bello, Santiago (2013). A unified framework for linear function approximation of value functions in stochastic control. In: "21st European Signal Processing Conference (EUSIPCO)", 09/09/2013 - 13/09/2013, Marrakech, Morocco. pp. 1-5.

Description

Title: A unified framework for linear function approximation of value functions in stochastic control
Author/s:
  • Sánchez Fernández, Matilde
  • Valcarcel Macua, Sergio
  • Zazo Bello, Santiago
Item Type: Presentation at Congress or Conference (Article)
Event Title: 21st European Signal Processing Conference (EUSIPCO)
Event Dates: 09/09/2013 - 13/09/2013
Event Location: Marrakech, Morocco
Title of Book: 21st European Signal Processing Conference (EUSIPCO)
Date: 2013
Subjects:
Freetext Keywords: Approximate dynamic programming, Linear value function approximation, Mean squared Bellman Error, Mean squared projected Bellman Error, Reinforcement Learning
Faculty: E.T.S.I. Telecomunicación (UPM)
Department: Señales, Sistemas y Radiocomunicaciones
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

[img]
Preview
PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (1MB)

Abstract

This paper contributes with a unified formulation that merges previ- ous analysis on the prediction of the performance ( value function ) of certain sequence of actions ( policy ) when an agent operates a Markov decision process with large state-space. When the states are represented by features and the value function is linearly approxi- mated, our analysis reveals a new relationship between two common cost functions used to obtain the optimal approximation. In addition, this analysis allows us to propose an efficient adaptive algorithm that provides an unbiased linear estimate. The performance of the pro- posed algorithm is illustrated by simulation, showing competitive results when compared with the state-of-the-art solutions.

More information

Item ID: 28942
DC Identifier: http://oa.upm.es/28942/
OAI Identifier: oai:oa.upm.es:28942
Official URL: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6811729&tag=1
Deposited by: Memoria Investigacion
Deposited on: 30 Jun 2014 16:04
Last Modified: 22 Sep 2014 11:43
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM