Artificial intelligence outperforms pulmonologists in the interpretation of pulmonary function tests

Marko Topalovic¹, Nilakash Das¹, Pierre-Régis Burgel², Marc Daenen³, Eric Derom⁴, Christel Haenebalcke⁵, Rob Janssen⁶, Huib A M Kerstjens⁷, Giuseppe Liistro⁸, Renaud Louis⁹, Vincent Ninane¹⁰, Christophe Pison¹¹, Marc Schlesser¹², Piet Vercauter¹³, Claus F Vogelmeier¹⁴, Emiel Wouters¹⁵, Jokke Wynants^{16

17}, Wim Janssens¹; Pulmonary Function Study Investigators; Pulmonary Function Study Investigators:

Collaborators, Affiliations

Collaborators

R De Pauw, C Depuydt, C Haenebalcke, S Muyldermans, V Ringoet, D Stevens, S Bayat, J Benet, E Catho, J Claustre, A Fedi, M A Ferjani, R Guzun, M Isnard, S Nicolas, T Pierret, C Pison, S Rouches, B Wuyam, J L Corhay, J Guiot, K Ghysen, L Renaud, A Sibille, H De La Barriere, C Charpentier, S Corhut, K A Hamdan, M Schlesser, G Wirtz, E Alabadan, G Birsen, P R Burgel, A Chohra, C Hamard, B Lemarié, M N Lothe, C Martin, A C Sainte-Marie, L Sebane, Y Berk, B de Brouwer, R Janssen, J Kerkhoff, A Spaanderman, M Stegers, A Termeer, I van Grimbergen, A van Veen, L van Ruitenbeek, L Vermeer, R Zaal, M Zijlker, J Aumann, K Cuppens, D Degraeve, K Demuynck, B Dieriks, K Pat, L Spaas, R Van Puijenbroek, K Weytjens, J Wynants, V Adam, B J Berendes, E Hardeman, P Jordens, E Munghen, K Tournoy, P Vercauter, T Alame, M Bruyneel, M Gabrovska, I Muylle, V Ninane, D Rozen, P Rummens, S Van Den Broecke, A Froidure, S Gohy, G Liistro, T Pieters, C Pilette, F Pirson, H Kerstjens, M Van den Berge, N Ten Hacken, M Duiverman, D Koster, B Vosse, L Conemans, M Maus, M Bischoff, M Rutten, D Agterhuis, R Sprooten, B Beutel, A Jerrentrup, A Klemmer, C Viniol, C Vogelmeier, H Bode, C Dooms, D Gullentops, W Janssens, K Nackaerts, D Rutens, E Wauters, W Wuyts, E Derom, S Dobbelaere, S Loof, G Serry, B Putman, L Van Acker, Y Vandeweygaerde, M Criel, M Daenen, R Gubbelmans, S Klerkx, E Michiels, M Thomeer, A Vanhauwaert

Affiliations

¹ Respiratory Medicine, University Hospital Leuven, Chronic Diseases, Metabolism and Ageing, KU Leuven, Leuven, Belgium.
² Cochin Hospital, AP-HP, Université Paris Descartes, Sorbonne Paris Cité, Paris, France.
³ Dept of Respiratory Medicine, Hospital Oost-Limburg, Genk, Belgium.
⁴ Dept of Respiratory Medicine, Ghent University Hospital, Ghent, Belgium.
⁵ Dept of Respiratory Medicine, AZ Sint-Jan Hospital, Bruges, Belgium.
⁶ Dept of Pulmonary Medicine, Canisius Wilhelmina Hospital, Nijmegen, The Netherlands.
⁷ Dept of Pulmonary Medicine and Tuberculosis, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands.
⁸ Dept of Pneumology, Cliniques Universitaires St-Luc, Université Catholique de Louvain, Brussels, Belgium.
⁹ Dept of Respiratory Medicine, University Hospital, Liege, Belgium.
¹⁰ Dept of Respiratory Medicine, Saint-Pierre Hospital, Université Libre de Bruxelles, Brussels, Belgium.
¹¹ Service Hospitalier Universitaire de Pneumologie et Physiologie, CHU Grenoble Alpes, Université Grenoble Alpes, Grenoble, France.
¹² Dept of Pulmonary Medicine, Centre Hospitalier de Luxembourg, Luxembourg, Luxembourg.
¹³ Dept of Respiratory Medicine, Onze-Lieve-Vrouw Hospital, Aalst, Belgium.
¹⁴ Dept of Medicine, Pulmonary and Critical Care Medicine, University Medical Center Giessen and Marburg, Member of the German Center for Lung Research (DZL), Marburg, Germany.
¹⁵ Dept of Respiratory Medicine, Maastricht University Medical Center, Maastricht, The Netherlands.
¹⁶ Dept of Pneumology, Jessa Hospital, Hasselt, Belgium.
¹⁷ For a full list of Pulmonary Function Study Investigators, please refer to the Acknowledgements section.

PMID: 30765505
DOI: 10.1183/13993003.01660-2018

Comparative Study

Artificial intelligence outperforms pulmonologists in the interpretation of pulmonary function tests

Marko Topalovic et al. Eur Respir J. 2019.

. 2019 Apr 11;53(4):1801660.

doi: 10.1183/13993003.01660-2018. Print 2019 Apr.

Authors

Collaborators

R De Pauw, C Depuydt, C Haenebalcke, S Muyldermans, V Ringoet, D Stevens, S Bayat, J Benet, E Catho, J Claustre, A Fedi, M A Ferjani, R Guzun, M Isnard, S Nicolas, T Pierret, C Pison, S Rouches, B Wuyam, J L Corhay, J Guiot, K Ghysen, L Renaud, A Sibille, H De La Barriere, C Charpentier, S Corhut, K A Hamdan, M Schlesser, G Wirtz, E Alabadan, G Birsen, P R Burgel, A Chohra, C Hamard, B Lemarié, M N Lothe, C Martin, A C Sainte-Marie, L Sebane, Y Berk, B de Brouwer, R Janssen, J Kerkhoff, A Spaanderman, M Stegers, A Termeer, I van Grimbergen, A van Veen, L van Ruitenbeek, L Vermeer, R Zaal, M Zijlker, J Aumann, K Cuppens, D Degraeve, K Demuynck, B Dieriks, K Pat, L Spaas, R Van Puijenbroek, K Weytjens, J Wynants, V Adam, B J Berendes, E Hardeman, P Jordens, E Munghen, K Tournoy, P Vercauter, T Alame, M Bruyneel, M Gabrovska, I Muylle, V Ninane, D Rozen, P Rummens, S Van Den Broecke, A Froidure, S Gohy, G Liistro, T Pieters, C Pilette, F Pirson, H Kerstjens, M Van den Berge, N Ten Hacken, M Duiverman, D Koster, B Vosse, L Conemans, M Maus, M Bischoff, M Rutten, D Agterhuis, R Sprooten, B Beutel, A Jerrentrup, A Klemmer, C Viniol, C Vogelmeier, H Bode, C Dooms, D Gullentops, W Janssens, K Nackaerts, D Rutens, E Wauters, W Wuyts, E Derom, S Dobbelaere, S Loof, G Serry, B Putman, L Van Acker, Y Vandeweygaerde, M Criel, M Daenen, R Gubbelmans, S Klerkx, E Michiels, M Thomeer, A Vanhauwaert

Affiliations

¹ Respiratory Medicine, University Hospital Leuven, Chronic Diseases, Metabolism and Ageing, KU Leuven, Leuven, Belgium.
² Cochin Hospital, AP-HP, Université Paris Descartes, Sorbonne Paris Cité, Paris, France.
³ Dept of Respiratory Medicine, Hospital Oost-Limburg, Genk, Belgium.
⁴ Dept of Respiratory Medicine, Ghent University Hospital, Ghent, Belgium.
⁵ Dept of Respiratory Medicine, AZ Sint-Jan Hospital, Bruges, Belgium.
⁶ Dept of Pulmonary Medicine, Canisius Wilhelmina Hospital, Nijmegen, The Netherlands.
⁷ Dept of Pulmonary Medicine and Tuberculosis, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands.
⁸ Dept of Pneumology, Cliniques Universitaires St-Luc, Université Catholique de Louvain, Brussels, Belgium.
⁹ Dept of Respiratory Medicine, University Hospital, Liege, Belgium.
¹⁰ Dept of Respiratory Medicine, Saint-Pierre Hospital, Université Libre de Bruxelles, Brussels, Belgium.
¹¹ Service Hospitalier Universitaire de Pneumologie et Physiologie, CHU Grenoble Alpes, Université Grenoble Alpes, Grenoble, France.
¹² Dept of Pulmonary Medicine, Centre Hospitalier de Luxembourg, Luxembourg, Luxembourg.
¹³ Dept of Respiratory Medicine, Onze-Lieve-Vrouw Hospital, Aalst, Belgium.
¹⁴ Dept of Medicine, Pulmonary and Critical Care Medicine, University Medical Center Giessen and Marburg, Member of the German Center for Lung Research (DZL), Marburg, Germany.
¹⁵ Dept of Respiratory Medicine, Maastricht University Medical Center, Maastricht, The Netherlands.
¹⁶ Dept of Pneumology, Jessa Hospital, Hasselt, Belgium.
¹⁷ For a full list of Pulmonary Function Study Investigators, please refer to the Acknowledgements section.

PMID: 30765505
DOI: 10.1183/13993003.01660-2018

Abstract

The interpretation of pulmonary function tests (PFTs) to diagnose respiratory diseases is built on expert opinion that relies on the recognition of patterns and the clinical context for detection of specific diseases. In this study, we aimed to explore the accuracy and interrater variability of pulmonologists when interpreting PFTs compared with artificial intelligence (AI)-based software that was developed and validated in more than 1500 historical patient cases.120 pulmonologists from 16 European hospitals evaluated 50 cases with PFT and clinical information, resulting in 6000 independent interpretations. The AI software examined the same data. American Thoracic Society/European Respiratory Society guidelines were used as the gold standard for PFT pattern interpretation. The gold standard for diagnosis was derived from clinical history, PFT and all additional tests.The pattern recognition of PFTs by pulmonologists (senior 73%, junior 27%) matched the guidelines in 74.4±5.9% of the cases (range 56-88%). The interrater variability of κ=0.67 pointed to a common agreement. Pulmonologists made correct diagnoses in 44.6±8.7% of the cases (range 24-62%) with a large interrater variability (κ=0.35). The AI-based software perfectly matched the PFT pattern interpretations (100%) and assigned a correct diagnosis in 82% of all cases (p<0.0001 for both measures).The interpretation of PFTs by pulmonologists leads to marked variations and errors. AI-based software provides more accurate interpretations and may serve as a powerful decision support tool to improve clinical practice.

PubMed Disclaimer

Conflict of interest statement

Conflict of interest: M. Topalovic has nothing to disclose. Conflict of interest: N. Das has nothing to disclose. Conflict of interest: P-R. Burgel reports personal fees from AstraZeneca, Boehringer Ingelheim, Chiesi, Novartis, Teva and Vertex, outside the submitted work. Conflict of interest: M. Daenen has nothing to disclose. Conflict of interest: E. Derom has nothing to disclose. Conflict of interest: C. Haenebalcke reports personal fees from Novartis, Chiesi, GSK and AstraZeneca, outside the submitted work. Conflict of interest: R. Janssen has nothing to disclose. Conflict of interest: H.A.M. Kerstjens has nothing to disclose. Conflict of interest: G. Liistro has nothing to disclose. Conflict of interest: R. Louis reports grants and personal fees from GSK and Novartis, personal fees from AstraZeneca, and grants from Chiesi, outside the submitted work. Conflict of interest: V. Ninane has nothing to disclose. Conflict of interest: C. Pison has nothing to disclose. Conflict of interest: M. Schlesser has nothing to disclose. Conflict of interest: P. Vercauter has nothing to disclose. Conflict of interest: C.F. Vogelmeier reports personal fees from Almirall, Cipla, Berlin-Chemie/Menarini, CSL Behring and Teva, grants and personal fees from AstraZeneca, Boehringer Ingelheim, Chiesi, GSK, Grifols, Mundipharma, Novartis and Takeda, grants from German Federal Ministry of Education and Research (BMBF) Competence Network Asthma and COPD (ASCONET), Bayer Schering Pharma AG, MSD and Pfizer, outside the submitted work. Conflict of interest: E. Wouters reports personal fees for board membership from Nycomed and Boehringer, grants from AstraZeneca and GSK, and personal fees for lectures from AstraZeneca, GSK, Novartis and Chiesi, outside the submitted work. Conflict of interest: J. Wynants has nothing to disclose. Conflict of interest: W. Janssens has nothing to disclose.

Comment in

Artificial intelligence for pulmonary function test interpretation.
Gonem S, Siddiqui S. Gonem S, et al. Eur Respir J. 2019 Jun 5;53(6):1900638. doi: 10.1183/13993003.00638-2019. Print 2019 Jun. Eur Respir J. 2019. PMID: 31167884 No abstract available.
Artificial intelligence for pulmonary function test interpretation.
Topalovic M, Das N, Janssens W; authors. Topalovic M, et al. Eur Respir J. 2019 Jun 5;53(6):1900782. doi: 10.1183/13993003.00782-2019. Print 2019 Jun. Eur Respir J. 2019. PMID: 31167887 No abstract available.
No need for pulmonologists to interpret pulmonary function tests.
Delclaux C. Delclaux C. Eur Respir J. 2019 Jul 18;54(1):1900829. doi: 10.1183/13993003.00829-2019. Print 2019 Jul. Eur Respir J. 2019. PMID: 31320479 No abstract available.
Künstliche Intelligenz schlägt Lungenärzte.
Gillissen A. Gillissen A. MMW Fortschr Med. 2019 Oct;161(17):36. doi: 10.1007/s15006-019-0955-0. MMW Fortschr Med. 2019. PMID: 31587207 Review. German. No abstract available.

Publication types

Actions
Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Artificial intelligence outperforms pulmonologists in the interpretation of pulmonary function tests

Collaborators

Affiliations

Artificial intelligence outperforms pulmonologists in the interpretation of pulmonary function tests

Authors

Collaborators

Affiliations

Abstract

Conflict of interest statement

Comment in

Publication types

MeSH terms