Applying a natural language processing tool to electronic health records to assess performance on colonoscopy quality measures

Abstract

Background: Gastroenterology specialty societies have advocated that providers routinely assess their performance on colonoscopy quality measures. Such routine measurement has been hampered by the costs and time required to manually review colonoscopy and pathology reports. Natural language processing (NLP) is a field of computer science in which programs are trained to extract relevant information from text reports in an automated fashion.

Objective: To demonstrate the efficiency and potential of NLP-based colonoscopy quality measurement.

Design: In a cross-sectional study design, we used a previously validated NLP program to analyze colonoscopy reports and associated pathology notes. The resulting data were used to generate provider performance on colonoscopy quality measures.

Setting: Nine hospitals in the University of Pittsburgh Medical Center health care system.

Patients: Study sample consisted of the 24,157 colonoscopy reports and associated pathology reports from 2008 to 2009.

Main outcome measurements: Provider performance on 7 quality measures.

Results: Performance on the colonoscopy quality measures was generally poor, and there was a wide range of performance. For example, across hospitals, the adequacy of preparation was noted overall in only 45.7% of procedures (range 14.6%-86.1% across 9 hospitals), cecal landmarks were documented in 62.7% of procedures (range 11.6%-90.0%), and the adenoma detection rate was 25.2% (range 14.9%-33.9%).

Limitations: Our quality assessment was limited to a single health care system in western Pennsylvania.

Conclusions: Our study illustrates how NLP can mine free-text data in electronic records to measure and report on the quality of care. Even within a single academic hospital system, there is considerable variation in the performance on colonoscopy quality measures, demonstrating the need for better methods to regularly and efficiently assess quality.

Figure 1. Performance on Quality Indicators for 9 Individual Hospitals and 77 Physicians with ≥30 Reports^*

^*Heavy vertical line indicates median performance and the surrounding box represents the boundaries of the 25^th and 75^th percentile of performance. The extended horizontal line (“whisker”) extends one and a half times the interquartile range beyond the box. The individual circles mark the performance of providers whose performance is outside the boundaries of the box (i.e. the 25^th and 75^th percentile).

Figure 2. Performance on Quality Indicators Based on How Colonoscopy Report Was Created^*

^*Difference between two types of methods of generating reports statistically significant at p<0.001 level for 6 of the 7 quality measures. For difference on withdrawal time, p-value 0.02