Prof. John McDermid
John McDermid became Professor of Software Engineering at the University of York in 1987. His research covers a broad range of issues in systems, software and safety engineering. He became Director of the Lloyd’s Register Foundation funded Assuring Autonomy International Programme in January 2018, focusing on safety of robotics and autonomous systems, including the role of Artificial Intelligence. He has acted as an advisor to government and industry, including FiveAI, the UK MoD and Rolls-Royce, and he will take up a post as non-executive director of the HSE in October 2019. He has been actively involved in standards development, including work on safety and software standards for civilian and defence applications. He is author or editor of six books and has published around 400 papers. He became a Fellow of the Royal Academy of Engineering in 2002 and was awarded an OBE in 2010.
INVITED TALK: Safety of Artificial Intelligence: A Collaborative Model
Achieving and assuring the safety of systems that use artificial intelligence (AI), especially machine learning (ML), pose some specific challenges that require unique solutions. However, that does not mean that good safety and software engineering practices are no longer relevant. This paper shows how the issues associated with AI and ML can be tackled by integrating with established safety and software engineering practices. It sets out a three-layer model, going from top to bottom: system safety/functional safety; ``AI/ML safety''; and safety-critical software engineering. This model gives both a basis for achieving and assuring safety and a structure for collaboration between safety engineers and AI/ML specialists. The model is illustrated with a healthcare example which uses deep reinforcement learning for treating sepsis patients. It is argued that this model is general and that it should underpin future standards and guidelines for safety of this class of system which employ ML, particularly because the model can facilitate collaboration between the different communities.