The anesthesiologist's guide to critically assessing machine learning research: a narrative review

Abstract

Artificial Intelligence (AI), especially Machine Learning (ML), has developed systems capable of performing tasks that require human intelligence. In anesthesiology and other medical fields, AI applications can improve the precision and efficiency of daily clinical practice, and can also facilitate a personalized approach to patient care, which can lead to improved outcomes and quality of care. ML has been successfully applied in various settings of daily anesthesiology practice, such as predicting acute kidney injury, optimizing anesthetic doses, and managing postoperative nausea and vomiting. The critical evaluation of ML models in healthcare is crucial to assess their validity, safety, and clinical applicability. Evaluation metrics allow an objective statistical assessment of model performance. Tools such as Shapley Values (SHAP) help interpret how individual variables contribute to model predictions. Transparency in reporting is key in maintaining trust in these technologies and to ensure their use follows ethical principles, aiming to reduce safety concerns while also benefiting patients. Understanding evaluation metrics is essential, as they provide detailed information on model performance and their ability to discriminate between individual class rates. This article offers a comprehensive framework in assessing the validity, applicability, and limitations of models, guiding responsible and effective integration of ML technologies into clinical practice. A balance between innovation, patient safety and ethical considerations must be pursued.

Keywords: Anesthesiology; Artificial intelligence; Clinical decision support system; Machine learning; Patient-specific modeling.

Fig. 1

Authors’ recommended pathway for reading AI research

Fig. 2

DNN SHAP values [45]