Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2023 Feb 15;9(7):eabn3999.
doi: 10.1126/sciadv.abn3999. Epub 2023 Feb 15.

Toward a taxonomy of trust for probabilistic machine learning

Tamara Broderick  1 , Andrew Gelman  2   3 , Rachael Meager  4 , Anna L Smith  5 , Tian Zheng  2
Affiliations
Review

Toward a taxonomy of trust for probabilistic machine learning

Tamara Broderick et al. Sci Adv. .

Abstract

Probabilistic machine learning increasingly informs critical decisions in medicine, economics, politics, and beyond. To aid the development of trust in these decisions, we develop a taxonomy delineating where trust in an analysis can break down: (i) in the translation of real-world goals to goals on a particular set of training data, (ii) in the translation of abstract goals on the training data to a concrete mathematical problem, (iii) in the use of an algorithm to solve the stated mathematical problem, and (iv) in the use of a particular code implementation of the chosen algorithm. We detail how trust can fail at each step and illustrate our taxonomy with two case studies. Finally, we describe a wide variety of methods that can be used to increase trust at each step of our taxonomy. The use of our taxonomy highlights not only steps where existing research work on trust tends to concentrate and but also steps where building trust is particularly challenging.

PubMed Disclaimer

Figures

Fig. 1.
Fig. 1.. Diagram illustrating steps where trust can break down in a data analysis workflow.
See the section “Where trust can break down” for a detailed description of the (nonitalicized) steps. See the section “Building trust” for a discussion of the italicized connectors, which include many of the characteristics of Schwartz et al. (83) for cultivating trust in AI systems.
Fig. 2.
Fig. 2.. Our election forecast for the Economist on the day of its release in June 2020.
Before taking this forecast public, we went through a series of checks of the data, model, and fitting procedure. We made further changes to the model as the campaign went on.
See this image and copyright information in PMC

References

    1. S. Flaxman, S. Mishra, A. Gandy, H. J. T. Unwin, T. A. Mellan, H. Coupland, C. Whittaker, H. Zhu, T. Berah, J. W. Eaton, M. Monod; Imperial College COVID- Response Team, A. C. Ghani, C. A. Donnelly, S. Riley, M. A. C. Vollmer, N. M. Ferguson, L. C. Okell, S. Bhatt, Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Nature 584, 257–261 (2020). - PubMed
    1. N. van Doremalen, T. Bushmaker, D. H. Morris, M. G. Holbrook, A. Gamble, B. N. Williamson, A. Tamin, J. L. Harcourt, N. J. Thornburg, S. I. Gerber, J. O. Lloyd-Smith, E. de Wit, V. J. Munster, Aerosol and surface stability of SARS-CoV-2 as compared with SARS-CoV-1. N. Engl. J. Med. 382, 1564–1567 (2020). - PMC - PubMed
    1. R. J. Fischer, D. H. Morris, N. van Doremalen, S. Sarchette, M. Jeremiah Matson, T. Bushmaker, C. K. Yinda, S. N. Seifert, A. Gamble, B. N. Williamson, S. D. Judson, E. de Wit, J. O. Lloyd-Smith, V. J. Munster, Effectiveness of N95 respirator decontamination and reuse against SARS-CoV-2 virus. Emerg. Infect. Dis. 26, 2253–2255 (2020). - PMC - PubMed
    1. M. Heidemanns, A. Gelman, G. E. Morris, An updated dynamic Bayesian forecasting model for the US presidential election. Harv Data Sci Rev 22020). 10.1162/99608f92.fc62f1e1.
    1. A. Gelman, J. Hullman, C. Wlezien, G. E. Morris, Information, incentives, and goals in election forecasts. Judgm. Decis. Mak. 15, 863–880 (2020).