Machine Learning of Plasma Proteomics Classifies Diagnosis of Interstitial Lung Disease

Abstract

Rationale: Distinguishing connective tissue disease-associated interstitial lung disease (CTD-ILD) from idiopathic pulmonary fibrosis (IPF) can be clinically challenging. Objectives: To identify proteins that separate and classify patients with CTD-ILD and those with IPF. Methods: Four registries with 1,247 patients with IPF and 352 patients with CTD-ILD were included in analyses. Plasma samples were subjected to high-throughput proteomics assays. Protein features were prioritized using recursive feature elimination to construct a proteomic classifier. Multiple machine learning models, including support vector machine, LASSO (least absolute shrinkage and selection operator) regression, random forest, and imbalanced Random Forest, were trained and tested in independent cohorts. The validated models were used to classify each case iteratively in external datasets. Measurements and Main Results: A classifier with 37 proteins (proteomic classifier 37 [PC37]) was enriched in the biological process of bronchiole development and smooth muscle proliferation and immune responses. Four machine learning models used PC37 with sex and age score to generate continuous classification values. Receiver operating characteristic curve analyses of these scores demonstrated consistent areas under the curve of 0.85-0.90 in the test cohort and 0.94-0.96 in the single-sample dataset. Binary classification demonstrated 78.6-80.4% sensitivity and 76-84.4% specificity in the test cohort and 93.5-96.1% sensitivity and 69.5-77.6% specificity in the single-sample classification dataset. Composite analysis of all machine learning models confirmed 78.2% (194 of 248) accuracy in the test cohort and 82.9% (208 of 251) in the single-sample classification dataset. Conclusions: Multiple machine learning models trained with large cohort proteomic datasets consistently distinguished CTD-ILD from IPF. Many of the identified proteins are involved in immune pathways. We further developed a novel approach for single-sample classification, which could facilitate honing the differential diagnosis of ILD in challenging cases and improve clinical decision making.

Keywords: connective tissue disease with ILD; differential diagnosis; idiopathic pulmonary fibrosis; machine learning model; plasma proteomics.

Figure 1.

Diagram for proteomics data processing and machine learning procedure. (A) PFF Patient Registry training cohort. Samples were filtered on the basis of clinical diagnosis and technical variation. The proteomic classifier was prioritized using RFE. (B) Machine learning classification for UVA/UChicago test cohort (left panel) and single sample in RECITAL/UC-Davis datasets (right panel). AUC = area under the curve; CTD-ILD = connective tissue disease–associated interstitial lung disease; IPF = idiopathic pulmonary fibrosis; LASSO = least absolute shrinkage and selection operator; PCA = principal component analysis; PFF = Pulmonary Fibrosis Foundation; ROC = receiver operating characteristic; SVM = support vector machine; UC-Davis = University of California, Davis; UChicago = University of Chicago; UVA = University of Virginia.

Figure 2.

Differential protein levels and pathways between connective tissue disease–associated interstitial lung disease (CTD-ILD) and idiopathic pulmonary fibrosis (IPF) in the Pulmonary Fibrosis Foundation (PFF) training cohort. (A) Volcano plot of t test results. Colored dots represent the top significant proteins higher in IPF (right) or CTD-ILD (left), respectively. Log fold changes (FCs; x-axis) and log inverse of Benjamini-Hochberg–adjusted P values (y-axis) of t test results were computed from the average of 100 times random subsampling of the PFF cohort as described in the Methods section. (B) Gene Set Enrichment Analysis (GSEA) of pathways that are activated or suppressed in IPF compared to CTD-ILD.

Figure 3.

Prioritization and characterization of the proteomic classifier. (A) Recursive feature elimination (RFE) procedure was performed to prioritize the optimized protein features in 100 times random subsampling of the Pulmonary Fibrosis Foundation cohort (Table E5). In this example subsampling for RFE, the optimal size was 138 proteins with 91% accuracy of repeated cross-validation. (B) Partial effects of the selected protein features on IPF probabilities in the training cohort. x-Axis, relative protein quantification unit on a log2 scale (NPX) values of Olink assay. (See the other 31 of 37 features in Figure E4.) (C) Variable importance (VIMP) plot of proteomic classifier 37 (PC37) with sex and age scores in the training cohort. VIMP with confidence intervals (CIs) in the training cohort. Bar represents 95% CI. Red and blue bars represent lower CI boundaries above or below 0, respectively. IPF = idiopathic pulmonary fibrosis.

Figure 4.

Performance of machine learning (ML) models for disease classification. ML classification supervised by proteomic classifier 37 (PC37) with sex and age scores. (A and B) Classification of UVA/UChicago test cohort. (C and D) Classification of RECITAL/University of California, Davis (UC-Davis), samples. (A and C) Binary classification by four ML models. (B and D) Receiver operating characteristic curve analysis of the continuous classification values. (E) Proportion of patients with connective tissue disease–associated interstitial lung disease (CTD-ILD) or patients with idiopathic pulmonary fibrosis (IPF) with different composite diagnosis scores (CDSs). CDS = 0 or 1, CTD-ILD; CDS = 2, unclassified patients; CDS = 3 or 4, IPF. (F) Decision curve analysis of RECITAL/UC-Davis samples to compare ML models with sex and age. All models and composite classification surpassed sex from 0 to 100% probability and age when probability was >50%. AUC = area under the curve; LASSO = least absolute shrinkage and selection operator; RF = random forest; SVM = support vector machine; UChicago = University of Chicago; UVA = University of Virginia.