ECE 6950: Partially Observed Markov Decision Processes (POMDPs): Filtering to Controlled Sensing
Credits: 3 (Grade Option: Letter)
The class is taught at Cornell Tech in NYC on Tuesday./Thursday 10:10 am to 11:25 am and broadcasted live to 312 Rhodes Hall in Ithaca
Instructor: Vikram Krishnamurthy.
Pre-requisites Background in undergraduate (non measure theoretic) random processes and optimization.
Description
This PhD level course deals with state estimation and stochastic control of partially observed Markov chains. It is suitable for graduate students in ECE, CS and Operations research. The material is from my recent book Partially Observed Markov Decision Processes: Filtering to Controlled Sensing published in Cambridge Univ Press in 2016. POMDPs have numerous examples in controlled sensing, wireless communications, machine learning, control systems, social learning and sequential detection. This course will focus on fundamentals: formulation, algorithms and analysis.
Class Notes.
Please click on the pdf files below to download.
Part 1: Stochastic Models and Simulation
Part 2: Filtering
Part 3: Maximum likelihood estimation
Part 4: Fully Observed MDP
Part 5: Partially Observed MDP
Internet Supplement on POMDPs.
Assessment: The course will have 4 assignments plus a take home final exam.