Explainable time-to-progression predictions in multiple sclerosis
- PMID: 39965473
- DOI: 10.1016/j.cmpb.2025.108624
Explainable time-to-progression predictions in multiple sclerosis
Abstract
Background: Prognostic machine learning research in multiple sclerosis has been mainly focusing on black-box models predicting whether a patients' disability will progress in a fixed number of years. However, as this is a binary yes/no question, it cannot take individual disease severity into account. Therefore, in this work we propose to model the time to disease progression instead. Additionally, we use explainable machine learning techniques to make the model outputs more interpretable.
Methods: A preprocessed subset of 29,201 patients of the international data registry MSBase was used. Disability was assessed in terms of the Expanded Disability Status Scale (EDSS). We predict the time to significant and confirmed disability progression using random survival forests, a machine learning model for survival analysis. Performance is evaluated on a time-dependent area under the receiver operating characteristic and the precision-recall curves. Importantly, predictions are then explained using SHAP and Bellatrex, two explainability toolboxes, and lead to both global (population-wide) as well as local (patient visit-specific) insights.
Results: On the task of predicting progression in 2 years, the random survival forest achieves state-of-the-art performance, comparable to previous work employing a random forest. However, here the random survival forest has the added advantage of being able to predict progression over a longer time horizon, with AUROC >60% for the first 10 years after baseline. Explainability techniques further validated the model by extracting clinically valid insights from the predictions made by the model. For example, a clear decline in the per-visit probability of progression is observed in more recent years since 2012, likely reflecting globally increasing use of more effective MS therapies.
Conclusion: The binary classification models found in the literature can be extended to a time-to-event setting without loss of performance, thus allowing a more comprehensive prediction of patient prognosis. Furthermore, explainability techniques proved to be key to reach a better understanding of the model and increase validation of its behaviour.
Keywords: Disability progression; Explainable artificial intelligence; Longitudinal data; Multiple sclerosis; Survival analysis.
Copyright © 2025 Elsevier B.V. All rights reserved.
Conflict of interest statement
Declaration of competing interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Tomas Kalincik served on scientific advisory boards for MS International Federation and World Health Organisation, BMS, Roche, Janssen, Sanofi Genzyme, Novartis, Merck and Biogen, steering committee for Brain Atrophy Initiative by Sanofi Genzyme, received conference travel support and/or speaker honoraria from WebMD Global, Eisai, Novartis, Biogen, Roche, Sanofi-Genzyme, Teva, BioCSL and Merck and received research or educational event support from Biogen, Novartis, Genzyme, Roche, Celgene and Merck. Eva Kubala Havrdova received honoraria/research support from Biogen, Merck Serono, Novars, Roche, and Teva; has been member of advisory boards for Actelion, Biogen, Celgene, Merck Serono, Novars, and Sanofi Genzyme;received honoraria/research support from Biogen, Merck Serono, Novars, Roche, and Teva; has been member of advisory boards for Actelion, Biogen, Celgene, Merck Serono, Novars, and Sanofi Genzyme; and has been supported by the Czech Ministry of Education – project Cooperatio LF1, research area Neuroscience, and the project National Institute for Neurological Research (Programme EXCELES, ID project No LX22NPO5107) – funded by the European Union-Next Generation EU. Alessandra Lugaresi has received personal compensation for consulting, serving on a scientific advisory board, speaking or other activities from Alexion, Biogen, Bristol Myers Squibb, Horizon, Janssen, Merck Serono, Novartis, and Sanofi/Genzyme, and Her institutions have received research grants from Novartis and Sanofi/Genzyme. Bianca Weinstock-Guttman served as a consultant for Biogen, EMD Serono, Novartis, Genentech, Celgene/Bristol Meyers Squibb, Sanofi Genzyme, Bayer, Janssen, Labcorp, Horizon and SANA. Dr. Weinstock-Guttman also has received grant/research support from Novartis, Biogen, Horizon/Amgen. She serves in the editorial board for Children, CNS Drugs, MS International, Journal of Neurology Frontiers Epidemiology. Saloua Mrabet has received a MENACTRIMS clinical fellowship grant (2020). Patrice Lalive received honoraria for speaking and or travel expense from Biogen, Merck, Novartis, Roche; consulting fees from Biogen, GeNeuro, Merck, Novartis, Roche; research support from Biogen, Merck, Novartis. None were related to this work. Allan G Kermode received speaker honoraria and scientific advisory board fees from Bayer, BioCSL, Biogen, Genzyme, Innate Immunotherapeutics, Merck, Novartis, Sanofi, Sanofi-Aventis, and Teva. Stephen Reddel has received funds over the last 5 years including but not limited to travel support, honoraria, trial payments, research and clinical support to the institution from Alexion, Biogen, Merck, Novartis, Roche, Sandoz, Sanofi. Additional interests and potential conflicts of interest include: Australian Technical Advisory Group on Immunisation Varicella Zoster working party (unpaid). Francesco Patti received personal compensation for serving on advisory board by Almirall, Alexion, Biogen, Bristol, Janssen, Merck, Novartis and Roche. He further received research grant by Alexion, Almirall, Biogen, Bristol, Merck, Novartis and Roche and by FISM, Reload Association (Onlus), Italian Health Minister, and University of Catania. Valentina Tomassini has received consultation and speaker fees, travel grants and research support from: Biogen, Sanofi Genzyme, Merck, Novartis, Roche, Alexion, Viatris, Janssen, Bristol Myers Squibb, Almirall. Izanne Roos has served on scientific advisory boards, received conference travel support and/or speaker honoraria from Roche, Novartis, Merck and Biogen. Izanne Roos is supported by a MS Australia and the Trish Multiple Sclerosis Research Foundation. Raed Alroughani received honoraria as a speaker and for serving on scientific advisory boards from Bayer, Biogen, GSK, Merck, Novartis, Roche and Sanofi-Genzyme. Samia J. Khoury received compensation for scientific advisory board activity from Merck and Roche, and received compensation for serving on the IDMC for Biogen. Vincent van Pesch received travel grants from Merck Healthcare KGaA (Darmstadt, Germany), Biogen, Sanofi, Bristol Meyer Squibb, Almirall and Roche. His institution has received research grants and consultancy fees from Roche, Biogen, Sanofi, Merck Healthcare KGaA (Darmstadt, Germany), Bristol Meyer Squibb, Janssen, Almirall, Novartis Pharma, and Alexion. Maria José Sá received consulting fees, speaker honoraria, and/or travel expenses for scientific meetings from Alexion, Bayer Healthcare, Biogen, Bristol Myers Squibb, Celgene, Janssen, Merck-Serono, Novartis, Roche, Sanofi and Teva. Julie Prevost accepted travel compensation from Novartis, Biogen, Genzyme, Teva, and speaking honoraria from Biogen, Novartis, Genzyme and Teva. Daniele Spitaleri received honoraria as a consultant on scientific advisory boards by Bayer-Schering, Novartis and Sanofi-Aventis and compensation for travel from Novartis, Biogen, Sanofi Aventis, Teva and Merck. Pamela McCombe received speakers fees and travel grants from Novartis, Biogen, T’évalua, Sanofi. Claudio Solaro served on scientific advisory boards for Merck, Genzyme, Almirall, and Biogen; received honoraria and travel grants from Sanofi Aventis, Novartis, Biogen, Merck, Genzyme and Teva. Anneke van der Walt served on advisory boards and receives unrestricted research grants from Novartis, Biogen, Merck and Roche She has received speaker’s honoraria and travel support from Novartis, Roche, and Merck. She receives grant support from the National Health and Medical Research Council of Australia and MS Research Australia. Helmut Butzkueven received institutional (Monash University) funding from Biogen, Roche, Merck, Alexion and Novartis; has carried out contracted research for Novartis, Merck, Roche and Biogen; has taken part in speakers’ bureaus for Biogen, Novartis, Roche and Merck; has received personal compensation from Oxford Health Policy Forum for the Brain Health Steering Committee. Guy Laureys received travel and/or consultancy compensation from Sanofi-Genzyme, Roche, Teva, Merck, Novartis, Celgene, Biogen. José Luis Sánchez-Menoyo accepted travel compensation from Novartis, Merck and Biogen, speaking honoraria from Biogen, Novartis, Sanofi, Merck, Almirall, Bayer and Teva and has participated in clinical trials by Biogen, Merck and Roche. Koen de Gans served on scientific advisory boards for Roche, Janssen, Sanofi-Genzyme, Novartis and Merck, received conference fee and travel support from Novartis, Biogen, Sanofi-Genzyme, Teva, Abbvie and Merck and received educational event support from Novartis. Abdullah Al-Asmi received personal compensation for serving as a Scientific Advisory or speaker/moderator for Novartis, Biogen, Roche, Sanofi-Genzyme, and Merck. Norma Deri received funding from Bayer, Merck, Biogen, Genzyme and Novartis. Tunde Csepany received speaker honoraria/ conference travel support from Biogen, Merck, Novartis, Roche, Sanofi-Aventis and Teva. William M Carroll received travel assistance and honoraria for participation in industry sponsored meetings from and provided advice to Bayer Schering Pharma, Biogen-Idec, Novartis, Roche, Genzyme, Sanofi-Aventis, CSL, Teva, Merck and Cellgene. Csilla Rozsa received speaker honoraria from Bayer Schering, Novartis and Biogen, congress and travel expense compensations from Biogen, Teva, Merck and Bayer Schering. Bhim Singhal received consultancy honoraria and compensation for travel from Biogen and Merck. Todd A. Hardy received speaker honoraria/ conference travel support or served on advisory boards for Bayer Schering, Biogen, Merck, Novartis, Roche, Sanofi-Genzyme, Bristol Myers Squibb and Teva. Sudarshini Ramanathan has received research funding from the National Health and Medical Research Council (NHMRC, Australia), the Petre Foundation, the Brain Foundation, the Royal Australasian College of Physicians, and the University of Sydney. She is supported by an NHMRC Investigator Grant (GNT2008339). She serves as a consultant on the International Steering Committee for a clinical trial led by UCB (NCT05063162). She is on the advisory board for educational activities led by Limbic Neurology. She has been an invited speaker for educational/research sessions coordinated by Biogen, Alexion, Novartis, Excemed and Limbic Neurology. She is on the medical advisory board (non-remunerated positions) of The MOG Project and the Sumaira Foundation. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
MeSH terms
LinkOut - more resources
Full Text Sources
Medical
