ECE 6950:  Partially Observed Markov Decision Processes (POMDPs): Filtering to Controlled Sensing

Credits: 3 (Grade Option: Letter)
The class is taught at Cornell Tech in NYC on Tuesday./Thursday 10:10 am to 11:25 am and broadcasted live to 312 Rhodes Hall in Ithaca

Instructor: Vikram Krishnamurthy.

Pre-requisites  Background in undergraduate (non measure theoretic) random processes and optimization.

This PhD level course deals with state estimation and stochastic control of partially observed Markov chains. It is suitable for graduate students in ECE, CS and Operations research. The material is from my recent book Partially Observed Markov Decision Processes: Filtering to Controlled Sensing published in Cambridge Univ Press in 2016. POMDPs have numerous examples in controlled sensing, wireless communications, machine learning, control systems, social learning and sequential detection. This course will focus on fundamentals: formulation, algorithms and  analysis.

Class Notes.    

Please click on the pdf files below to download.

Part 0: Introduction

Part 1: Stochastic Models and Simulation

Part 2: Filtering

Part 3: Maximum likelihood estimation

Part 4: Fully Observed MDP

Part 5: Partially Observed MDP

Internet Supplement on POMDPs.


Assessment:  The course will have 4 assignments plus a take home final exam.