|
| Publications [#243783] of Mauro Maggioni
Papers Published
- Bouvrie, J; Maggioni, M, Efficient solution of Markov decision problems with multiscale representations,
2012 50th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2012
(December, 2012),
pp. 474-481, IEEE [doi]
(last updated on 2019/04/10)
Abstract:
Many problems in sequential decision making and stochastic control naturally enjoy strong multiscale structure: sub-tasks are often assembled together to accomplish complex goals. However, systematically inferring and leveraging hierarchical structure has remained a longstanding challenge. We describe a fast multiscale procedure for repeatedly compressing or homogenizing Markov decision processes (MDPs), wherein a hierarchy of sub-problems at different scales is automatically determined. Coarsened MDPs are themselves independent, deterministic MDPs, and may be solved using any method. The multiscale representation delivered by the algorithm decouples sub-tasks from each other and improves conditioning. These advantages lead to potentially significant computational savings when solving a problem, as well as immediate transfer learning opportunities across related tasks. © 2012 IEEE.
|
