Markov Decision Processes

Formulation and solution of sequential decision problems under uncertainty as a foundation for artificial intelligence, operations research, and economics. Finite-horizon, discounted, and average reward objectives. Optimization via value iteration, policy iteration, linear programming, and reinforcement learning algorithms. Semi-Markov decision processes. Multi-armed bandits and the Gittin's index theorem. Use of partial state information. Homework assignments involve a combination of analytic and computational exercises.