Prof. Andreas Malikopoulos presented a lecture on “Combining Learning and Control in Cyber-Physical Systems” as part of the A Warren Distinguished Lecture series at the University of Minnesota on October 27, 2023.
Cyber-physical systems (CPS), in most instances, represent systems of subsystems with an informationally decentralized structure. To derive optimal control strategies for such systems, we typically assume an ideal model. Such model-based control approaches cannot effectively facilitate optimal solutions with performance guarantees due to the discrepancy between the model and the actual cyber-physical system. On the other hand, in most CPS there is a large volume of data of a dynamic nature, which is added to the system gradually in real time and not altogether in advance. Thus, traditional supervised learning approaches cannot always facilitate robust solutions using data derived offline. By contrast, applying reinforcement learning approaches directly to the actual CPS might impose significant implications on the safety and robust operation of the system. In this talk, Malikopoulos discusses the challenges of supervised learning and model-based control approaches in several CPS applications, including self-learning powertrain control, power management control of hybrid electric vehicles, and optimal coordination of connected and automated vehicles. Then, Malikopoulos presents a theoretical framework founded at the intersection of control theory and learning that circumvents these challenges in deriving optimal strategies for CPS. In this framework, Malikopoulos and his research team aim to identify a sufficient information state for the CPS that takes values in a time-invariant space and use this information state to derive separated control strategies. Separated control strategies are related to the concept of separation between the estimation of the information state and control of the system. By establishing separated control strategies, they can derive offline the optimal control strategy of the system with respect to the information state, which might not be precisely known due to model uncertainties or the complexity of the system, and then use learning methods to learn the information state online while data are added gradually to the system in real time. This approach could effectively facilitate optimal solutions with performance guarantees in a wide range of CPS applications, such as emerging mobility systems, networked control systems, smart power grids, cooperative cyber-physical networks, cooperation of robots, and the Internet of Things.
More details can be found here.