Physics

Markov model

Decision theory

Mathematics

Markov chain

Operating system

Algorithm

Statistics

Markov decision process

Mathematical optimization

Machine learning

Artificial intelligence

Markov process

Partially observable Markov decision process

Computer science

Quantum mechanics

Process (computing)

Observable

State (computer science)

Management Science

This paper surveys models and algorithms dealing with partially observable Markov decision processes. A partially observable Markov decision process (POMDP) is a generalization of a Markov decision process which permits uncertainty regarding the state of a Markov process and allows for state information acquisition. A general framework for finite state and action POMDP's is presented. Next, there is a brief discussion of the development of POMDP's and their relationship with other decision processes. A wide range of models in such areas as quality control, machine maintenance, internal auditing, learning, and optimal stopping are discussed within the POMDP-framework. Lastly, algorithms for computing optimal solutions to POMDP's are presented.

State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms