Empirical performance of a self-controlled cohort method: lessons for developing a risk identification and analysis system
- PMID: 24166227
- DOI: 10.1007/s40264-013-0101-3
Empirical performance of a self-controlled cohort method: lessons for developing a risk identification and analysis system
Abstract
Background: Observational healthcare data offer the potential to enable identification of risks of medical products, but appropriate methodology has not yet been defined. The self-controlled cohort method, which compares the post-exposure outcome rate with the pre-exposure rate among an exposed cohort, has been proposed as a potential approach for risk identification but its performance has not been fully assessed.
Objectives: To evaluate the performance of the self-controlled cohort method as a tool for risk identification in observational healthcare data.
Research design: The method was applied to 399 drug-outcome scenarios (165 positive controls and 234 negative controls across 4 health outcomes of interest) in 5 real observational databases (4 administrative claims and 1 electronic health record) and in 6 simulated datasets with no effect and injected relative risks of 1.25, 1.5, 2, 4, and 10, respectively.
Measures: Method performance was evaluated through area under ROC curve (AUC), bias, and coverage probability.
Results: The self-controlled cohort design achieved strong predictive accuracy across the outcomes and databases under study, with the top-performing settings exceeding AUC >0.76 in all scenarios. However, the estimates generated were observed to be highly biased with low coverage probability.
Conclusions: If the objective for a risk identification system is one of discrimination, the self-controlled cohort method shows promise as a potential tool for risk identification. However, if a system is intended to generate effect estimates to quantify the magnitude of potential risks, the self-controlled cohort method may not be suitable, and requires substantial calibration to be properly interpreted under nominal properties.
Similar articles
-
A comparison of the empirical performance of methods for a risk identification system.Drug Saf. 2013 Oct;36 Suppl 1:S143-58. doi: 10.1007/s40264-013-0108-9. Drug Saf. 2013. PMID: 24166231
-
Empirical performance of a new user cohort method: lessons for developing a risk identification and analysis system.Drug Saf. 2013 Oct;36 Suppl 1:S59-72. doi: 10.1007/s40264-013-0099-6. Drug Saf. 2013. PMID: 24166224
-
Empirical performance of the calibrated self-controlled cohort analysis within temporal pattern discovery: lessons for developing a risk identification and analysis system.Drug Saf. 2013 Oct;36 Suppl 1:S107-21. doi: 10.1007/s40264-013-0095-x. Drug Saf. 2013. PMID: 24166228
-
Desideratum for evidence based epidemiology.Drug Saf. 2013 Oct;36 Suppl 1:S5-14. doi: 10.1007/s40264-013-0102-2. Drug Saf. 2013. PMID: 24166219 Review.
-
Trigger Tool-Based Automated Adverse Event Detection in Electronic Health Records: Systematic Review.J Med Internet Res. 2018 May 30;20(5):e198. doi: 10.2196/jmir.9901. J Med Internet Res. 2018. PMID: 29848467 Free PMC article.
Cited by
-
Bridging islands of information to establish an integrated knowledge base of drugs and health outcomes of interest.Drug Saf. 2014 Aug;37(8):557-67. doi: 10.1007/s40264-014-0189-0. Drug Saf. 2014. PMID: 24985530 Free PMC article.
-
Electronic Health Data for Postmarket Surveillance: A Vision Not Realized.Drug Saf. 2015 Jul;38(7):601-10. doi: 10.1007/s40264-015-0305-9. Drug Saf. 2015. PMID: 26025018 Review.
-
Pharmacovigilance via Baseline Regularization with Large-Scale Longitudinal Observational Data.KDD. 2017 Aug;2017:1537-1546. doi: 10.1145/3097983.3097998. KDD. 2017. PMID: 29755826 Free PMC article.
-
Characterizing non-heroin opioid overdoses using electronic health records.JAMIA Open. 2019 Nov 26;3(1):77-86. doi: 10.1093/jamiaopen/ooz063. eCollection 2020 Apr. JAMIA Open. 2019. PMID: 32607490 Free PMC article.
-
A comparison of the empirical performance of methods for a risk identification system.Drug Saf. 2013 Oct;36 Suppl 1:S143-58. doi: 10.1007/s40264-013-0108-9. Drug Saf. 2013. PMID: 24166231
References
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources
Medical