Complexity, uncertainty and the Safety of ML

Simon Burton (1) , Benjamin Herd (1)

1 Fraunhofer IKS - Fraunhofer Institute for Cognitive Systems

There is currently much debate regarding whether or not applications based on Machine Learning (ML) can be made demonstrably safe. We assert that our ability to argue the safety of ML-based functions depends on the complexity of the task and environment of the function, the observations (training and test data) used to develop the function and the complexity of the ML models. Our inability to adequately address this complexity inevitably leads to uncertainties in the specification of the safety requirements, the performance of the ML models and our assurance argument itself. By understanding each of these dimensions as a continuum, can we better judge what level of safety can be achieved for a particular ML-based function?

Keywords: Artificial intelligence machine learning safety assurance uncertainty complexity; Artificial intelligence; machine learning; safety assurance; uncertainty; complexity

Domains: Computer Science [cs]

Simon Burton, Benjamin Herd. Complexity, uncertainty and the Safety of ML. SAFECOMP 2023, Position Paper, Sep 2023, Toulouse, France. ⟨hal-04191756⟩