Consensus Between Radiologists, Specialists in Internal Medicine, and AI Software on Chest X-Rays in a Hospital-at-Home Service: Prospective Observational Study

Abstract

Background: Home hospitalization is a care modality growing in popularity worldwide. Telemedicine-driven hospital-at-home (HAH) services could replace traditional hospital departments for selected patients. Chest x-rays typically serve as a key diagnostic tool in such cases.

Objective: The implementation, analysis, and clinical assimilation of chest x-rays into an HAH service has not been described yet. Our objective is to introduce this essential information to the realm of HAH services for the first time worldwide.

Methods: The study involved a prospective follow-up, description, and analysis of the HAH patient population who underwent chest x-rays at home. A comparative analysis was performed to evaluate the level of agreement among three interpretation modalities: a radiologist, a specialist in internal medicine, and a designated artificial intelligence (AI) algorithm.

Results: Between February 2021 and May 2023, 300 chest radiographs were performed at the homes of 260 patients, with the median age being 78 (IQR 65-87) years. The most frequent underlying morbidity was cardiovascular disease (n=185, 71.2%). Of the x-rays, 286 (95.3%) were interpreted by a specialist in internal medicine, 29 (9.7%) by a specialized radiologist, and 95 (31.7%) by the AI software. The overall raw agreement level among these three modalities exceeded 90%. The consensus level evaluated using the Cohen κ coefficient showed substantial agreement (κ=0.65) and moderate agreement (κ=0.49) between the specialist in internal medicine and the radiologist, and between the specialist in internal medicine and the AI software, respectively.

Conclusions: Chest x-rays play a crucial role in the HAH setting. Rapid and reliable interpretation of these x-rays is essential for determining whether a patient requires transfer back to in-hospital surveillance. Our comparative results showed that interpretation by an experienced specialist in internal medicine demonstrates a significant level of consensus with that of the radiologists. However, AI algorithm-based interpretation needs to be further developed and revalidated prior to clinical applications.

Keywords: AI; artificial intelligence; chest; chest x-ray; clinical data; comparative analysis; home hospitalization; hospital-at-home; implementation; kappa; radiologist; telemedicine; x-ray.