Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2019 Mar 5;92(10):e1016-e1028.
doi: 10.1212/WNL.0000000000007043. Epub 2019 Feb 15.

Validation of an algorithm for identifying MS cases in administrative health claims datasets

William J Culpepper  1 , Ruth Ann Marrie  2 , Annette Langer-Gould  2 , Mitchell T Wallin  2 , Jonathan D Campbell  2 , Lorene M Nelson  2 , Wendy E Kaye  2 , Laurie Wagner  2 , Helen Tremlett  2 , Lie H Chen  2 , Stella Leung  2 , Charity Evans  2 , Shenzhen Yao  2 , Nicholas G LaRocca  2 ; United States Multiple Sclerosis Prevalence Workgroup (MSPWG)
Collaborators, Affiliations
Multicenter Study

Validation of an algorithm for identifying MS cases in administrative health claims datasets

William J Culpepper et al. Neurology. .

Abstract

Objective: To develop a valid algorithm for identifying multiple sclerosis (MS) cases in administrative health claims (AHC) datasets.

Methods: We used 4 AHC datasets from the Veterans Administration (VA), Kaiser Permanente Southern California (KPSC), Manitoba (Canada), and Saskatchewan (Canada). In the VA, KPSC, and Manitoba, we tested the performance of candidate algorithms based on inpatient, outpatient, and disease-modifying therapy (DMT) claims compared to medical records review using sensitivity, specificity, positive and negative predictive values, and interrater reliability (Youden J statistic) both overall and stratified by sex and age. In Saskatchewan, we tested the algorithms in a cohort randomly selected from the general population.

Results: The preferred algorithm required ≥3 MS-related claims from any combination of inpatient, outpatient, or DMT claims within a 1-year time period; a 2-year time period provided little gain in performance. Algorithms including DMT claims performed better than those that did not. Sensitivity (86.6%-96.0%), specificity (66.7%-99.0%), positive predictive value (95.4%-99.0%), and interrater reliability (Youden J = 0.60-0.92) were generally stable across datasets and across strata. Some variation in performance in the stratified analyses was observed but largely reflected changes in the composition of the strata. In Saskatchewan, the preferred algorithm had a sensitivity of 96%, specificity of 99%, positive predictive value of 99%, and negative predictive value of 96%.

Conclusions: The performance of each algorithm was remarkably consistent across datasets. The preferred algorithm required ≥3 MS-related claims from any combination of inpatient, outpatient, or DMT use within 1 year. We recommend this algorithm as the standard AHC case definition for MS.

PubMed Disclaimer

Figures

Figure 1
Figure 1. Performance of algorithm MS_E-1 [(IP + OP + DMT) ≥ 3] stratified by sex across the VA, KPSC, and MB datasets
(A) Men and women, (B) men only, and (C) women only.Data are presented as a proportion that can range between 0 and 1. DMT = disease-modifying therapy; IP = inpatient; KPSC = Kaiser Permanente Southern California; MB = Manitoba; MS = multiple sclerosis; NPV = negative predictive value; OP = outpatient; PPV = positive predictive value; VA = Department of Veterans Affairs.
Figure 2
Figure 2. Performance of algorithm MS_E-1 [(IP + OP + DMT) ≥ 3] stratified by age group across the VA, KPSC, and MB datasets
(A) Sensitivity, (B) specificity, (C) positive predictive value, (D) negative predictive value, (E) accuracy, and (F) Youden J statistic.Data from Manitoba (MB) for the 55- to 64- and 64- to 74-year age groups are suppressed due to small cell sizes. Data are presented as a proportion that can range between 0 and 1. DMT = disease-modifying therapy; IP = inpatient; KPSC = Kaiser Permanente Southern California; MS = multiple sclerosis; NPV = negative predictive value; OP = outpatient; PPV = positive predictive value; VA = Department of Veterans Affairs.
Figure 3
Figure 3. Comparison of prevalence based on a 3- vs 10-year ascertainment period as of 2010 in the (A) VA and (B) MB datasets
CI = confidence interval; MB = Manitoba; VA = Department of Veterans Affairs.
Figure 4
Figure 4. Comparison of prevalence based on a 3- vs 9-year ascertainment period as of 2015 in the IMS (validation) dataset
CI = confidence interval; IMS = Intercontinental Marketing Services.
See this image and copyright information in PMC

References

    1. Evans C, Beland SG, Kulaga S, et al. Incidence and prevalence of multiple sclerosis in the Americas: a systematic review. Neuroepidemiology 2013;40:195–210. - PubMed
    1. Kingwell E, Marriott JJ, Jetté N, et al. Incidence and prevalence of multiple sclerosis in Europe: a systematic review. BMC Neurol 2013;13:128. - PMC - PubMed
    1. Makhani N, Morrow SA, Fisk J, et al. MS incidence and prevalence in Africa, Asia, Australia and New Zealand: a systematic review. Mult Scler Relat Disord 2014;3:48–60. - PubMed
    1. Tricco AC, Pham B, Rawson NSB. Manitoba and Saskatchewan administrative health care utilization databases are used differently to answer epidemiologic research questions. J Clin Epidemiol 2008;61:192–197.e112. - PubMed
    1. Culpepper WJ, Ehrmantraut M, Wallin MT, Flannery K, Bradham DD. Veterans Health Administration Multiple Sclerosis Surveillance Registry: the problem of case-finding from administrative databases. J Rehabil Res Dev 2006;43:17. - PubMed

Publication types