Mercolli, LorenzoLorenzoMercolli0000-0002-3170-7311Rominger, Axel OliverAxel OliverRominger0000-0002-1954-736XShi, KuangyuKuangyuShi2024-10-262024-10-262024-05https://boris-portal.unibe.ch/handle/20.500.12422/174971The use of artificial intelligence systems in clinical routine is still hampered by the necessity of a medical device certification and/or by the difficulty of implementing these systems in a clinic's quality management system. In this context, the key questions for a user are how to ensure robust model predictions and how to appraise the quality of a model's results on a regular basis. In this paper we discuss some conceptual foundation for a clinical implementation of a machine learning system and argue that both vendors and users should take certain responsibilities, as is already common practice for high-risk medical equipment. We propose the methodology from AAPM Task Group 100 report No. 283 as a conceptual framework for developing risk-driven a quality management program for a clinical process that encompasses a machine learning system. This is illustrated with an example of a clinical workflow. Our analysis shows how the risk evaluation in this framework can accommodate artificial intelligence based systems independently of their robustness evaluation or the user's in-house expertise. In particular, we highlight how the degree of interpretability of a machine learning system can be systematically accounted for within the risk evaluation and in the development of a quality management system.enArtificial intelligence Machine learning Quality management Risk analysis600 - Technology::610 - Medicine & healtharticle10.48350/1935403841335510.1016/j.zemedi.2024.02.001