Validation of rule-based algorithms to determine colorectal, breast, and cervical cancer screening status using electronic health record data from an urban healthcare system in New York City

Abstract

Although cancer screening has greatly reduced colorectal cancer, breast cancer, and cervical cancer morbidity and mortality over the last few decades, adherence to cancer screening guidelines remains inconsistent, particularly among certain demographic groups. This study aims to validate a rule-based algorithm to determine adherence to cancer screening. A novel screening algorithm was applied to electronic health record (EHR) from an urban healthcare system in New York City to automatically determine adherence to national cancer screening guidelines for patients deemed eligible for screening. First, a subset of patients was randomly selected from the EHR and their data were exported in a de-identified manner for manual review of screening adherence by two teams of human reviewers. Interrater reliability for manual review was calculated using Cohen's Kappa and found to be high in all instances. The sensitivity and specificity of the algorithm was calculated by comparing the algorithm to the final manual dataset. When assessing cancer screening adherence, the algorithm performed with a high sensitivity (79%, 70%, 80%) and specificity (92%, 99%, 97%) for colorectal cancer, breast cancer, and cervical cancer screenings, respectively. This study validates an algorithm that can effectively determine patient adherence to colorectal cancer, breast cancer, and cervical cancer screening guidelines. This design improves upon previous methods of algorithm validation by using computerized extraction of essential components of patients' EHRs and by using de-identified data for manual review. Use of the described algorithm could allow for more precise and efficient allocation of public health resources to improve cancer screening rates.

Keywords: Algorithm; BMI, Body Mass Index; CDSS, Computerized Clinical Decision Support Systems; CPT, Current Procedural Terminology; Cancer screening; EHR, Electronic Health Record; EHR/electronic health record; ICD, International Classification of Disease; IRR, Inter-rater reliability; NYULH, New York University Langone Health; SQL, Structured Query Language.

Fig. 1

Algorithm validation workflow. Diagram demonstrating the parallel workflows of the manual review teams and algorithm.