Routine receiver operating characteristic analysis in mammography as a measure of radiologists' performance

Abstract

A system for calculating receiver operating characteristic (ROC) curves from routine audit data is described. Both diagnostic opinion and pathology outcome data from the symptomatic mammography department were recorded in an audit database, from which ROC curves were calculated. A comparison of overall radiologists' performance was made and the appropriate performance indices discussed. Similar data were collected from the breast screening centre where the same radiologists read mammograms. The radiologists' performance in the symptomatic department was compared with that in the breast screening centre and was found to be better in the screening centre. The difference could not be wholly attributed to the different age distribution of women and highlighted the different nature of the diagnostic task and the differing sampled populations. ROC curves were drawn for different clinical signs to determine any particular area of difficulty the radiologist may experience. ROC analysis as part of routine audit can be used for maintaining and improving the quality of an individual's performance, and targeting learning on areas of particular weakness.