Active Sensing POMDPs

My new book published by Cambridge University Press in 2016.

How to build smart reconfigurable sensors that dynamically adapt their behavior over time? This is a partially observed stochastic control problem. Examples include cognitive radars, sensor scheduling and cognitive radio.

Our research in POMDPs focus mainly on structural results – that is, how to characterize the optimal policy using powerful ideas in supermodularity and stochastic dominance – without brute force computations.

 

Book and Related Papers

  1. V. Krishnamurthy, Convex Stochastic Dominance in Bayesian Localization, Filtering and Controlled Sensing POMDPs, IEEE Transactions Information Theory, 2019. (15 pages)
  2. V. Krishnamurthy, A. Aprem, S, Bhatt, Multiple Stopping Time POMDPs: Structural Results & Application in Interactive Advertising in Social Media, Automatica, 2018.
  3. (Book) V. Krishnamurthy, Partially Observed Markov Decision Processes – From Filtering to Controlled Sensing, Cambridge Univ Press 2016. Click on book image on right to access book and internet supplement.
  4. V. Krishnamurthy, S. Bhatt, Sequential Detection of Market Shocks with Risk-Averse CVaR social sensors, IEEE Journal Selected Topics Signal Processing, 2016
  5. V. Krishnamurthy, E Leoff, J. Sass, Filterbased stochastic volatility in continuous-time hidden Markov models, Econometrics and Statistics, Nov 2016.
  6. V. Krishnamurthy, U. Pareek, Myopic Bounds for Optimal Policies of POMDPs: An extension of Lovejoy’s Structural Results, Operations Research, 2014.
  7. V. Krishnamurthy, Quickest Detection POMDPs with Social Learning , IEEE Trans Information Theory, Aug 2012
  8. V. Krishnamurthy, How to schedule measurements of a noisy Markov chain for decision making, IEEE Trans Information Theory, July 2013.
  9. V. Krishnamurthy, Bayesian Sequential Detection with Phase-Distributed Change Time and Nonlinear Penalty – A Lattice Programming Approach, IEEE Transactions Information Theory, October 2011.
  10. V. Krishnamurthy, B. Wahlberg, POMDP Multiarmed Bandits–Structural Results, Mathematics of Operations Research, May 2009.
  11. V. Krishnamurthy, R. Bitmead, M. Gevers, E. Miehling, Sequential Detection with Mutual Information Stopping Cost: Application in GMTI Radar, IEEE Transactions Signal Processing, Vol.60, No.2, pp.700–714, Feb 2012.
  12. V. Krishnamurthy, D. Djonin,  Optimal Threshold Policies for Multivariate POMDPs in  Radar  Resource Management, IEEE Trans Signal Processing, Vol.57, No.10, pp.3954–3969, 2009.
  13. V. Krishnamurthy, D. Djonin, Structured  Threshold Policies for Dynamic Sensor Scheduling–A POMDP Approach, IEEE Trans Signal Processing, Vol.55, No.10, pp.4938–4957, Oct.2007.
  14. R. Evans, V. Krishnamurthy and G. Nair, Networked Sensor Management and Data Rate Control for Tracking Maneuvering Targets, IEEE Transactions on Signal Processing, Vol.53, No.6, pp.1979–1991, June 2005.
  15. V. Krishnamurthy, Algorithms for Optimal Scheduling and Management of Hidden Markov Model Sensors, IEEE Transactions Signal Processing, Vol.50, No.6, pp.1382–1397, June 2002.
  16. L. Johnston and V. Krishnamurthy, Opportunistic File Transfer over a Fading Channel – A POMDP  Search Theory Formulation with Optimal Threshold Policies,IEEE Transactions Wireless Communications, Vol.5, No.2, pp. 394–405, Feb. 2006.
  17. S. Singh and V. Krishnamurthy, The optimal search for  a Markovian target when the search path is constrained: the infinite horizon case, IEEE Transactions Automatic Control, Vol.48, No.3, pp.487–492, March 2003.