Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Dec 1;26(12):1645-1650.
doi: 10.1093/jamia/ocz145.

Prognostic models will be victims of their own success, unless…

Matthew C Lenert  1 , Michael E Matheny  2   3   4   5 , Colin G Walsh  2   4   6
Affiliations

Prognostic models will be victims of their own success, unless…

Matthew C Lenert et al. J Am Med Inform Assoc. .

Abstract

Predictive analytics have begun to change the workflows of healthcare by giving insight into our future health. Deploying prognostic models into clinical workflows should change behavior and motivate interventions that affect outcomes. As users respond to model predictions, downstream characteristics of the data, including the distribution of the outcome, may change. The ever-changing nature of healthcare necessitates maintenance of prognostic models to ensure their longevity. The more effective a model and intervention(s) are at improving outcomes, the faster a model will appear to degrade. Improving outcomes can disrupt the association between the model's predictors and the outcome. Model refitting may not always be the most effective response to these challenges. These problems will need to be mitigated by systematically incorporating interventions into prognostic models and by maintaining robust performance surveillance of models in clinical use. Holistically modeling the outcome and intervention(s) can lead to resilience to future compromises in performance.

Keywords: learning health system; model updating; predictive modeling.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Model life cycle.
Figure 2.
Figure 2.
Model interactions with outcome.
Figure 3.
Figure 3.
Consequences of model success.
See this image and copyright information in PMC

Comment in

References

    1. Xiao C, Choi E, Sun J.. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Inform Assoc 2018; 2510: 1419–28. doi:10.1093/jamia/ocy068. - PMC - PubMed
    1. Stead WW. Clinical implications and challenges of artificial intelligence and deep learning. JAMA 2018; 32011: 1107–8. doi:10.1037/h0030806. - PubMed
    1. Moons KGM, Altman DG, Vergouwe Y, Royston P.. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 2009; 3382: b606.. - PubMed
    1. Horvitz EF. Data to predictions and decisions: enabling evidence-based healthcare. 2010. https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/Evid... Accessed February 19, 2019.
    1. Miotto R, Wang F, Wang S, Jiang X, Dudley JT.. Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform. - PMC - PubMed

Publication types