Decentralized Stochastic Control And Learning

Modern engineering systems are increasingly complex and comprise of many interdependent dynamic parts. This can enable the development of a novel framework to process large amounts of data and deliver real-time control actions that maximize associated benefits. However, as we move to increasingly complex systems, we need to develop new decentralized control approaches to optimize the impact on system behavior of the interaction between its entities.

Centralized stochastic control has been ubiquitous approach to control complex systems with stochastic uncertainties. Centralized stochastic control refers to multi-stage optimization problems of systems with external disturbances and noisy observations by a single decision maker. A key assumption in deriving solutions of centralized stochastic control problems is that the decision maker perfectly recalls all past control actions and observations.

While centralized systems have been extensively studied, their core assumption does not hold for larger systems comprising of multiple agents, e.g., connected automated vehicles, drone swarms, and smart grids. In such systems, all agents simultaneously take a decision at every stage based only on their memory, which comprises of the history of their own observations and local information received through communication with other agents. Finding optimal control strategies for such decentralized systems has been shown to be computationally very hard. Thus, the focus of our research is to develop approaches that can improve the feasibility of deriving optimal control strategies for decentralized systems comprising of multiple agents.

Relevant Publications:

  1. Dave, A., Venkatesh, N., and Malikopoulos, A.A., “Approximate Information States for Worst-Case Control and Learning in Uncertain Systems,”  IEEE Trans. Autom. Control, Vol. 70,  1, pp. 127–142, 2025 (pdf).
  2. Malikopoulos, A.A., “On Separation Between Learning and Control in Partially Observed Markov Decision Processes,” in Smarter Cyber-Physical Systems: Enabling Methodologies and Applications, Y. Wan, K. G. Vamvoudakis, Y. Chen, F. L. Lewis (Eds.), CRC Press, 2024.
  3. Malikopoulos, A.A. “Combining Learning and Control in Linear Systems,” European Journal of Control, Vol. 80, Part A, 101043, 2024 (pdf).
  4. Venkatesh, N., Dave, A.,  Faros, I. and Malikopoulos, A.A., “Stochastic Control with Distributionally Robust Constraints for Cyber-Physical Systems Vulnerable to Attacks,” European Journal of Control, Vol. 80, Part A, 101044, 2024 (pdf).
  5. Dave, A., Bang, H., and Malikopoulos, A.A., “A Framework for Effective AI Recommendations in Cyber-Physical-Human Systems,” Proceedings of 63rd IEEE Conference on Decision and Control, 2024 — see IEEE Control Systems Letters (L-CSS), Vol. 8, pp. 1379–1384, 2024 (pdf).
  6. Venkatesh, N., Dave, A.,  Faros, I. and Malikopoulos, A.A., “Stochastic Control with Distributionally Robust Constraints for Cyber-Physical Systems Vulnerable to Attacks,” European Journal of Control, Vol. 80, Part A, 101044, 2024 (pdf).
  7. Beaver, L. E., and Malikopoulos, A.A., “Optimal Control of Differentially Flat Systems is Surprisingly Easy,” Automatica, 159, 111404, 2024 (pdf).
  8. Dave, A., Venkatesh, N., Faros, I., and Malikopoulos, A.A., “Worst-Case Control and Learning Using Partial Observations Over an Infinite Time Horizon,” Proceedings of 62nd IEEE Conference on Decision and Control, pp. 6014-6019, 2023 (pdf).
  9. Faros, I., Dave, A., and Malikopoulos, A.A., “A Q-learning Approach for Adherence-Aware Recommendations,” Proceedings of 2024 American Control Conference, 2024 (to appear) —  see IEEE Control Systems Letters (L-CSS), 7, pp. 3645 – 3650, 2023 (pdf).
  10. Dave, A., Venkatesh, N., and Malikopoulos, A.A., “On Robust Control of Partially Observed Uncertain Systems with Additive Costs,” Proceedings of 2023 American Control Conference, pp. 4639-4644, 2023 (pdf).
  11. Malikopoulos, A.A., “Separation of Learning and Control for Cyber-Physical Systems,” Automatica, 151, 110912, 2023 (pdf).
  12. Dave, A., Venkatesh, N., and Malikopoulos, A.A., “Approximate Information States for Worst-case Control of Uncertain Systems,” Proceedings of 61st IEEE Conference on Decision and Control, pp. 4945-4950, 2022 (pdf).
  13. Malikopoulos, A.A., “On Team Decision Problems with Nonclassical Information Structures,” IEEE Trans. Autom. Control, Vol. 68, 7, pp. 3915–3930, 2023 (pdf).
  14. Dave, A., Venkatesh, N., and Malikopoulos, A.A., “Decentralized Control of Two Agents with Nested Accessible Information,” Proceedings of 2022 American Control Conference, pp. 3423-3430, 2022 (pdf).
  15. Dave, A., Venkatesh, N., and Malikopoulos, A.A., “On Decentralized Minimax Control with Nested Subsystems,” Proceedings of 2022 American Control Conference, pp. 3437-3444, 2022 (pdf).
  16. Dave, A., and Malikopoulos, A.A., “A Dynamic Program for a Team of Two Agents with Nested Information,” Proceedings of the 60th IEEE Conference on Decision and Control, pp. 3768-3773, 2021 (pdf).
  17. Dave, A., and Malikopoulos, A.A., “Structural Results for Decentralized Stochastic Control with a Word-of-Mouth Communication” Proceedings of 2020 American Control Conference, pp. 2796–2801, 2020 (pdf).
  18. Dave, A., and Malikopoulos, A. A., “Decentralized Stochastic Control in Partially Nested Information Structures,” Proceedings of the 8th IFAC Workshop on Distributed Estimation and Control in Networked Systems, 2019 (pdf).