Department of Mathematics
 Search | Help | Login | pdf version | printable version

Math @ Duke



Publications [#243783] of Mauro Maggioni

Papers Published

  1. Bouvrie, J; Maggioni, M, Efficient solution of Markov decision problems with multiscale representations, 2012 50th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2012 (December, 2012), pp. 474-481 [doi]
    (last updated on 2018/12/16)

    Many problems in sequential decision making and stochastic control naturally enjoy strong multiscale structure: sub-tasks are often assembled together to accomplish complex goals. However, systematically inferring and leveraging hierarchical structure has remained a longstanding challenge. We describe a fast multiscale procedure for repeatedly compressing or homogenizing Markov decision processes (MDPs), wherein a hierarchy of sub-problems at different scales is automatically determined. Coarsened MDPs are themselves independent, deterministic MDPs, and may be solved using any method. The multiscale representation delivered by the algorithm decouples sub-tasks from each other and improves conditioning. These advantages lead to potentially significant computational savings when solving a problem, as well as immediate transfer learning opportunities across related tasks. © 2012 IEEE.
ph: 919.660.2800
fax: 919.660.2821

Mathematics Department
Duke University, Box 90320
Durham, NC 27708-0320