Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jul;49(7):785-795.
doi: 10.1007/s00134-023-07137-1. Epub 2023 Jun 24.

Machine learning to predict poor school performance in paediatric survivors of intensive care: a population-based cohort study

Patricia Gilholm  1 , Kristen Gibbons  1 , Sarah Brüningk  2   3 , Juliane Klatt  2   3 , Rhema Vaithianathan  4 , Debbie Long  1   5 , Johnny Millar  6   7   8 , Wojtek Tomaszewski  4 , Luregn J Schlapbach  9   10 ; Australian and New Zealand Intensive Care Society (ANZICS) Centre for Outcomes & Resource Evaluation (CORE) and ANZICS Paediatric Study Group (ANZICS PSG)
Collaborators, Affiliations

Machine learning to predict poor school performance in paediatric survivors of intensive care: a population-based cohort study

Patricia Gilholm et al. Intensive Care Med. 2023 Jul.

Abstract

Purpose: Whilst survival in paediatric critical care has improved, clinicians lack tools capable of predicting long-term outcomes. We developed a machine learning model to predict poor school outcomes in children surviving intensive care unit (ICU).

Methods: Population-based study of children < 16 years requiring ICU admission in Queensland, Australia, between 1997 and 2019. Failure to meet the National Minimum Standard (NMS) in the National Assessment Program-Literacy and Numeracy (NAPLAN) assessment during primary and secondary school was the primary outcome. Routine ICU information was used to train machine learning classifiers. Models were trained, validated and tested using stratified nested cross-validation.

Results: 13,957 childhood ICU survivors with 37,200 corresponding NAPLAN tests after a median follow-up duration of 6 years were included. 14.7%, 17%, 15.6% and 16.6% failed to meet NMS in school grades 3, 5, 7 and 9. The model demonstrated an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.8 (standard deviation SD, 0.01), with 51% specificity to reach 85% sensitivity [relative Area Under the Precision Recall Curve (rel-AUPRC) 3.42, SD 0.06]. Socio-economic status, illness severity, and neurological, congenital, and genetic disorders contributed most to the predictions. In children with no comorbidities admitted between 2009 and 2019, the model achieved a AUROC of 0.77 (SD 0.03) and a rel-AUPRC of 3.31 (SD 0.42).

Conclusions: A machine learning model using data available at time of ICU discharge predicted failure to meet minimum educational requirements at school age. Implementation of this prediction tool could assist in prioritizing patients for follow-up and targeting of rehabilitative measures.

Keywords: Child; Intensive care; Machine learning; Neurodevelopment; School.

PubMed Disclaimer

Conflict of interest statement

None declared. The funding bodies had no role in study design, conduct, analyses and reporting.

Figures

Fig. 1
Fig. 1
Participant flow diagram demonstrating the selection of the analyzed sample for each of the four school grades and numbers included the final combined model. ANZPICR Australia and New Zealand Paediatric Intensive Care Registry, NAPLAN National Assessment Program-Literacy and Numeracy, ICU intensive care unit; *For the combined model, only one ICU admission and one NAPLAN testing year outcome were used per child. Specifically, the NAPLAN test following the last recorded and eligible ICU admission per child was used for the combined model. As most children are admitted to ICU before they are eight years old, the NAPLAN test included most commonly in the final model was the Grade 3 test.
Fig. 2
Fig. 2
Mean area under the receiver operating characteristic curve (AUROC) (A), mean area under the precision recall curve (AUPRC) (B), and mean AUROCs for specific patient subgroups (C) of the machine learning model predicting failure to meet National Minimal Standard in school. Curves represent findings across cross-validation folds for the combined model. Shading indicates ± two standard deviation error (A, B), and ± one standard deviation error for patient subgroups (C). The horizontal lines (C) indicate the mean AUROC ± one standard deviation of the entire cohort
Fig. 3
Fig. 3
Mean normalized Shapley additive explanations (SHAP) importance scores for the top 20 predictors for the combined model. The colours reference the direction of each variable for prediction of failing to meet the national minimum standard (NMS) on the Reading and Numeracy domains. Red indicates a higher value or the presence of the variable contributing most to failing to meet the NMS (e.g. “total ICU length of stay”, implying that a higher number of ICU days are associated with a higher risk of failing to meet the NMS). Blue indicates a lower value or absence of that variable contributing most to failing to meet the NMS (e.g. “systolic blood pressure”, implying that lower systolic blood pressure values are associated with a higher risk of failing to meet the NMS). The principal component (PC) variables shown are labelled with the diagnostic ICU code that has the highest component loading. Please refer to Supplementary Table 1 for the list of all codes with high loadings on each PC
See this image and copyright information in PMC

References

    1. Moynihan KM, Alexander PMA, Schlapbach LJ, Millar J, Jacobe S, Ravindranathan H, Croston EJ, Staffa SJ, Burns JP, Gelbart B, Australian, New Zealand Intensive Care Society Pediatric Study G, the ACfO, Resource E Epidemiology of childhood death in Australian and New Zealand intensive care units. Intensive Care Med. 2019;45:1262–1271. - PubMed
    1. Schlapbach LJ, Straney L, Alexander J, MacLaren G, Festa M, Schibler A, Slater A, Group APS Mortality related to invasive infections, sepsis, and septic shock in critically ill children in Australia and New Zealand, 2002–13: a multicentre retrospective cohort study. Lancet Infect Dis. 2015;15:46–54. - PubMed
    1. Schlapbach LJ, Gibbons KS, Horton SB, Johnson K, Long DA, Buckley DHF, Erickson S, Festa M, d'Udekem Y, Alphonso N, Winlaw DS, Delzoppo C, van Loon K, Jones M, Young PJ, Butt W, Schibler A, Nitric Study Group tA, New Zealand Intensive Care Society Clinical Trials G, the APSG Effect of nitric oxide via cardiopulmonary bypass on ventilator-free days in young children undergoing congenital heart disease surgery: the NITRIC Randomized Clinical Trial. JAMA. 2022;328:38–47. - PMC - PubMed
    1. Carlton EF, Pinto N, Smith M, Fink EL, Watson RS, Hoehn KS, Marupudi N, Dervan LA, Lackey M, Ringwood M, Maddux AB, Network tP-PIotP, Health tEKSNIoC, Network HDCPCCR Overall health following pediatric critical illness: a scoping review of instruments and methodology. Pediatr Crit Care Med. 2021;22:1061–1071. - PMC - PubMed
    1. Fink EL, Jarvis JM, Maddux AB, Pinto N, Galyean P, Olson LM, Zickmund S, Ringwood M, Sorenson S, Dean JM, Carcillo JA, Berg RA, Zuppa A, Pollack MM, Meert KL, Hall MW, Sapru A, McQuillen PS, Mourani PM, Watson RS, Pediatric Acute Lung I, Sepsis Investigators Long-term Outcomes Subgroup Investigators a, Eunice Kennedy Shriver National Institute of Child H, Human Development Collaborative Pediatric Critical Care Research N Development of a core outcome set for pediatric critical care outcomes research. Contemp Clin Trials. 2020;91:105968. - PMC - PubMed