NSF SLES

Learning-enabled systems have been rapidly increasing in size, acquiring new capabilities. These systems are typically deployed in complex operating environments, so their safety is extremely important. Ensuring safety requires that systems are robust to extreme events while we can monitor them for anomalous and unsafe behavior.  While traditional machine learning systems are evaluated pointwise with respect to a fixed test set, such static coverage provides only limited assurance when exposed to unprecedented conditions in complex operating environments. One key question that still remains unanswered is “how can we design and deploy learning-enabled systems that can be robust to extreme events while monitoring them for anomalous and unsafe behavior by synthesizing model-free and model-based techniques?” 

The overarching goal of the proposed research is to establish a framework that leads to the design and implementation of learning-enabled systems in which safety is ensured with high levels of confidence. The framework will leverage tools from control theory, multi-agent autonomy, and formal methods for rigorously probabilistic reasoning to yield safe learning-enabled systems. The expected outcome of this project will yield safe model-free, mode-based, and interacting model-free and model-based learning-enabled systems with sound design principles that practitioners could leverage to achieve safety specifications. The proposed research could effectively facilitate safe learning-enabled systems even within complex environments while monitoring them for anomalous and unsafe behavior. It will yield learning-enabled systems that could be deployed in complex operating environments while ensuring that the systems will be robust to extreme events and monitoring them for anomalous and unsafe behavior.  The fundamental knowledge created in the proposed research will be the basis upon which future-safe autonomous systems with embodied intelligence can be built.