Computer-assisted detection of pulmonary nodules: performance evaluation of an expert knowledge-based detection system in consensus reading with experienced and inexperienced chest radiologists

Abstract

To evaluate the performance of experienced versus inexperienced radiologists in comparison and in consensus with an interactive computer-aided detection (CAD) system for detection of pulmonary nodules. Eighteen consecutive patients (mean age: 62.2 years; range 29-83 years) prospectively underwent routine 16-row multislice computed tomography (MSCT). Four blinded radiologists (experienced: readers 1, 2; inexperienced: readers 3, 4) assessed image data against CAD for pulmonary nodules. Thereafter, consensus readings of readers 1+3, reader 1+CAD and reader 3+CAD were performed. Data were compared against an independent gold standard. Statistical tests used to calculate interobserver agreement, reader performance and nodule size were Kappa, ROC and Mann-Whitney U. CAD and experienced readers outperformed inexperienced readers (Az=0.72, 0.71, 0.73, 0.49 and 0.50 for CAD, readers 1-4, respectively; P<0.05). Performance of reader 1+CAD was superior to single reader and reader 1+3 performances (Az=0.93, 0.72 for reader 1+CAD and reader 1+3 consensus, respectively, P<0.05). Reader 3+CAD did not perform superiorly to experienced readers or CAD (Az=0.79 for reader 3+CAD; P>0.05). Consensus of reader 1+CAD significantly outperformed all other readings, demonstrating a benefit in using CAD as an inexperienced reader replacement. It is questionable whether inexperienced readers can be regarded as adequate for interpretation of pulmonary nodules in consensus with CAD, replacing an experienced radiologist.