Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Sep 1;3(9):e2012734.
doi: 10.1001/jamanetworkopen.2020.12734.

Validation of a Machine Learning Model to Predict Childhood Lead Poisoning

Eric Potash  1 , Rayid Ghani  2 , Joe Walsh  1 , Emile Jorgensen  3 , Cortland Lohff  4 , Nik Prachand  3 , Raed Mansour  3
Affiliations

Validation of a Machine Learning Model to Predict Childhood Lead Poisoning

Eric Potash et al. JAMA Netw Open. .

Abstract

Importance: Childhood lead poisoning causes irreversible neurobehavioral deficits, but current practice is secondary prevention.

Objective: To validate a machine learning (random forest) prediction model of elevated blood lead levels (EBLLs) by comparison with a parsimonious logistic regression.

Design, setting, and participants: This prognostic study for temporal validation of multivariable prediction models used data from the Women, Infants, and Children (WIC) program of the Chicago Department of Public Health. Participants included a development cohort of children born from January 1, 2007, to December 31, 2012, and a validation WIC cohort born from January 1 to December 31, 2013. Blood lead levels were measured until December 31, 2018. Data were analyzed from January 1 to October 31, 2019.

Exposures: Blood lead level test results; lead investigation findings; housing characteristics, permits, and violations; and demographic variables.

Main outcomes and measures: Incident EBLL (≥6 μg/dL). Models were assessed using the area under the receiver operating characteristic curve (AUC) and confusion matrix metrics (positive predictive value, sensitivity, and specificity) at various thresholds.

Results: Among 6812 children in the WIC validation cohort, 3451 (50.7%) were female, 3057 (44.9%) were Hispanic, 2804 (41.2%) were non-Hispanic Black, 458 (6.7%) were non-Hispanic White, and 442 (6.5%) were Asian (mean [SD] age, 5.5 [0.3] years). The median year of housing construction was 1919 (interquartile range, 1903-1948). Random forest AUC was 0.69 compared with 0.64 for logistic regression (difference, 0.05; 95% CI, 0.02-0.08). When predicting the 5% of children at highest risk to have EBLLs, random forest and logistic regression models had positive predictive values of 15.5% and 7.8%, respectively (difference, 7.7%; 95% CI, 3.7%-11.3%), sensitivity of 16.2% and 8.1%, respectively (difference, 8.1%; 95% CI, 3.9%-11.7%), and specificity of 95.5% and 95.1% (difference, 0.4%; 95% CI, 0.0%-0.7%).

Conclusions and relevance: The machine learning model outperformed regression in predicting childhood lead poisoning, especially in identifying children at highest risk. Such a model could be used to target the allocation of lead poisoning prevention resources to these children.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Potash reported receiving grants from Robert Wood Johnson Foundation during the conduct of the study. Dr Mansour reported grants from Robert Wood Johnson Foundation during the conduct of the study. No other disclosures were reported.

Figures

Figure 1.
Figure 1.. Temporal Validation Flowchart
BLL indicates blood lead level; EBLL, elevated BLL; and WIC, Women, Infants, and Children.
Figure 2.
Figure 2.. Receiver Operating Characteristic Curves for Random Forest and Logistic Regression Models
Difference in the areas under the receiver operating characteristics curve was 0.05 (95% CI, 0.02-0.08).
See this image and copyright information in PMC

References

    1. Tsoi M-F, Cheung C-L, Cheung TT, Cheung BMY. Continual decrease in blood lead level in Americans: United States National Health Nutrition and Examination Survey 1999-2014. Am J Med. 2016;129(11):1213-1218. doi:10.1016/j.amjmed.2016.05.042 - DOI - PubMed
    1. National Toxicology Program NTP monograph on health effects of low-level lead. NTP Monogr. 2012;(1):e2012734. - PubMed
    1. Dietrich KN, Ware JH, Salganik M, et al. ; Treatment of Lead-Exposed Children Clinical Trial Group . Effect of chelation therapy on the neuropsychological and behavioral development of lead-exposed children after school entry. Pediatrics. 2004;114(1):19-26. doi:10.1542/peds.114.1.19 - DOI - PubMed
    1. Centers for Disease Control and Prevention Low level lead exposure harms children: a renewed call of primary prevention: report of the Advisory Committee on Childhood Lead Poisoning Prevention. Published January 4, 2012. Accessed October 31, 2019. https://www.cdc.gov/nceh/lead/acclpp/final_document_030712.pdf
    1. Council on Environmental Health Prevention of childhood lead toxicity. Pediatrics. 2016;138(1):e20161493. doi:10.1542/peds.2016-1493 - DOI - PubMed

Publication types