Approaches for classifying the indications for colonoscopy using detailed clinical data

Abstract

Background: Accurate indication classification is critical for obtaining unbiased estimates of colonoscopy effectiveness and quality improvement efforts, but there is a dearth of published systematic classification approaches. The objective of this study was to evaluate the effects of data-source and adjudication on indication classification and on estimates of the effectiveness of screening colonoscopy on late-stage colorectal cancer diagnosis risk.

Methods: This was an observational study in members of four U.S. health plans. Eligible persons (n = 1039) were age 55-85 and had been enrolled for 5 years or longer in their health plans during 2006-2008. Patients were selected based on late-stage colorectal cancer diagnosis in a case-control design; each case patient was matched to 1-2 controls by study site, age, sex, and health plan enrollment duration. Reasons for colonoscopies received in the 10-year period before the reference date were collected from three medical records sources (progress notes; referral notes; procedure reports) and categorized using an algorithm, with committee adjudication of some tests. We evaluated indication classification concordance before and after adjudication and used logistic regressions with the Wald Chi-square test to compare estimates of the effects of screening colonoscopy on late-stage colorectal cancer diagnosis risk for each of our data sources to the adjudicated indication.

Results: Classification agreement between each data-source and adjudication was 78.8-94.0% (weighted kappa = 0.53-0.72); the highest agreement (weighted kappa = 0.86-0.88) was when information from all data sources was considered together. The choice of data-source influenced the association between screening colonoscopy and late-stage colorectal cancer diagnosis; estimates based on progress notes were closest to those based on the adjudicated indication (% difference in regression coefficients = 2.4%, p-value = 0.98), as compared to estimates from only referral notes (% difference in coefficients = 34.9%, p-value = 0.12) or procedure reports (% difference in coefficients = 27.4%, p-value = 0.23).

Conclusion: There was no single gold-standard source of information in medical records. The estimates of colonoscopy effectiveness from progress notes alone were the closest to estimates using adjudicated indications. Thus, the details in the medical records are necessary for accurate indication classification.

Figure 1

Decision algorithm for colonoscopy indication classification.

Figure 2

Flow diagram of the derivation of indication variables for colonoscopy. *Up to three coded reasons were recorded from each data source during the chart audit. †One indication variable was derived for each data source. ‡This is a single indication assigned to each test combining all coded data collected on each test during chart audit using the computer algorithm shown in Figure 1. It combined data from referral note, progress note and procedure report. §‘N’ is the number of patients. The numbers in parentheses are the tests received by the N patients. ¶A test was selected for review if more than one indication could be assigned or was unknown in all data sources, or relevant free-text data. #Tests on these patients were not selected for review and/or adjudication (see text).

Figure 3

Percentage distribution of colonoscopy indication by medical records data sources and targeted adjudication, at the test-level and analytic or patient-level. *The numbers are the percentages in each classification group for colonoscopies in Figure 3A or patients in Figure 3B. There were 647 colonoscopies observed in 524 patients. The distribution of indication in Figure 3B, correspond to the analytic variable. Each of the colored sections of the stacked bars represents the classification of the indication as shown in the legend. The “all sources combined” indication is assigned with data from all sources using the classification algorithm.

Figure 4

Agreement on colonoscopy indication classification across three medical records data sources: test-level and patient-level analysis. The percentages are the observed agreement and the proportions are the weighted kappa (ĸ) statistic. *The numbers in the circles are the patient-level analyses results.