A standardized immune phenotyping and automated data analysis platform for multicenter biomarker studies

Abstract

The analysis and validation of flow cytometry-based biomarkers in clinical studies are limited by the lack of standardized protocols that are reproducible across multiple centers and suitable for use with either unfractionated blood or cryopreserved PBMCs. Here we report the development of a platform that standardizes a set of flow cytometry panels across multiple centers, with high reproducibility in blood or PBMCs from either healthy subjects or patients 100 days after hematopoietic stem cell transplantation. Inter-center comparisons of replicate samples showed low variation, with interindividual variation exceeding inter-center variation for most populations (coefficients of variability <20% and interclass correlation coefficients >0.75). Exceptions included low-abundance populations defined by markers with indistinct expression boundaries (e.g., plasmablasts, monocyte subsets) or populations defined by markers sensitive to cryopreservation, such as CD62L and CD45RA. Automated gating pipelines were developed and validated on an independent data set, revealing high Spearman's correlations (rs >0.9) with manual analyses. This workflow, which includes pre-formatted antibody cocktails, standardized protocols for acquisition, and validated automated analysis pipelines, can be readily implemented in multicenter clinical trials. This approach facilitates the collection of robust immune phenotyping data and comparison of data from independent studies.

Keywords: Adaptive immunity; Immunology; Innate immunity; Stem cell transplantation; Transplantation.

Figure 1. Inter-center comparison of populations quantified in cryopreserved PBMCs from healthy subjects with the Basic, TCR, B cell, or DC panels.

Replicate aliquots of cryopreserved PBMCs from 5 subjects were analyzed at 5 different sites using the indicated DuraClone panels. Raw data (LMD files) were analyzed centrally, and the reproducibility of population proportions was determined by statistical analysis. All calculations were based on the cell proportion relative to the parent gate; the identity of parent gates for all variables is listed in Supplemental Table 1. Shown are CV and ICC values for each population. CV and ICC values indicative of poor reproducibly (i.e., CV >20% and/or ICC <0.75) are shaded in gray. CS memory, class-switched memory; DNT cells, double-negative T cells; mDCs, myeloid DCs; MZB, marginal zone B cells; pDCs, plasmacytoid DCs; RO, CD45RO.

Figure 2. Inter-center comparison of populations quantified in cryopreserved PBMCs from healthy subjects with the T-ACT and T-MEM-REG panels.

Replicate aliquots of cryopreserved PBMCs from 5 subjects were analyzed at 5 different sites using the indicated DuraClone panels. Raw data were analyzed centrally, and the reproducibility of population proportions was determined by statistical analysis. All calculations are based on the cell proportion relative to the parent gate; the identity of parent gates for all variables is listed in Supplemental Table 1. Shown are CV and ICC values for each population. CV and ICC values indicative of poor reproducibly (i.e., CV >20% and/or ICC <0.75) are shaded in gray. 45RA, CD45RA.

Figure 3. Effects of cryopreservation on standardization.

Peripheral blood from healthy subjects was either analyzed immediately or processed into PBMCs and cryopreserved for later analysis with the same DuraClone panels. Raw data were analyzed manually at one center. Data are from 3 subjects analyzed at 3 different sites (n = 9); unfractionated blood was only analyzed at the collection site. The identity of the parent gates is shown in Supplemental Table 1. Representative populations as measured in blood versus PBMCs with (A) CV <20% or (B) CV >20% as shown in Figures 1 and 2. *P < 0.05, **P < 0.01, multiple t test with FDR adjustment according to Benjamini, Hochberg, and Yekutiel. (C) Representative data from 1 individual for naive/memory CD4⁺ T cell proportions detected in blood or in replicate samples of cryopreserved PBMCs analyzed at 3 different centers (sites 1–3). (D) The proportion of CD45RA⁺ Tregs (of total Tregs) in 3 different individuals was measured in unfractionated blood or PBMCs. **P < 0.01, 2-way repeated-measures ANOVA with Šidák’s multiple comparison test. (E) Representative data from Tregs quantified in blood or replicate samples of cryopreserved PBMCs analyzed at 3 different centers. CS mem B, class-switched memory B cells; mono, monocytes.

Figure 4. Inter-center comparison of cryopreserved PBMC data from post-HSCT subjects.

Replicate aliquots of cryopreserved PBMCs from 5 subjects were analyzed at 3 different sites. Raw data were analyzed centrally, and the reproducibility of population proportions was determined by statistical analysis. All calculations were based on the cell proportion relative to the input gate; the identity of parent gates for all variables is listed in Supplemental Table 1. Shown are CV and ICC values for each population. CV and ICC values indicative of poor reproducibility (i.e., CV >20% and/or ICC <0.75) are shaded in gray. 45RA, CD45RA.

Figure 5. The effect of population size on CVs.

CVs were plotted against the log of median population sizes from all inter-site comparisons. All population sizes are expressed as percent of CD45⁺ PBMCs. Solid and dotted lines indicate CVs of 10% and 20%, respectively. Gray box indicates populations that are less than 2.5% of PBMCs. Red symbols indicate monocyte subtypes. Populations shown to be affected by cryopreservation (defined by CD45RA or CD62L) were excluded.

Figure 6. Detecting differences between 2 cohorts using unmanipulated blood versus cryopreserved PBMCs.

Populations from all panels that significantly differed between healthy controls and patients 100 ± 20 days after HSCT in either (A) whole blood or (B) cryopreserved PBMCs. Note: Post-HSCT, but not healthy control, PBMCs were incubated 24 hours before cryopreservation. Proportions of parent gates for each population are shown as box-and-whisker plots; midline is the median, box is the interquartile range, and whiskers show minimum and maximum values. Means are indicated by a thick black band. See Supplemental Table 1 for a list of parent gates. Only samples that were evaluable in both unmanipulated blood and PBMCs were compared; n = 9 healthy and n = 8 post-HSCT subjects (except n = 4 for B cell–derived populations). *P < 0.05, **P < 0.01, ***P < 0.001; unpaired t test with Holm-Šidák corrections.

Figure 7. Generation of automated data analysis pipelines for the DuraClone IM Basic panel.

The proportions of cell populations indicated in Supplemental Table 1 for the Basic panel were determined in unfractionated blood from healthy (n = 9) and HSCT (n = 11) subjects using automated gating, 2 different manual analyzers (Manual 1, Manual 2), or a reference manual analyzer. Data from Manual 1 or 2, or automated gating were compared with those from the reference manual to determine (A) Spearman’s correlation coefficients (r_s) or (B) F₁ scores (harmonization of precision and recall, maximum value 1.0) ordered by population size (highest on the left). Box-and-whisker plot: midline is the median, box is the IQR, and the whiskers extend to 1.5 times the IQR. (C) Representative automated or manually gated data from 1 healthy subject and correlation graphs for (C) CD14⁺ total monocytes, (D) CD14⁺CD16⁺ monocytes, and (E) CD56⁺⁺ NK cells. White circles indicate Manual 1, and gray squares automated, both plotted against the reference manual data.

Figure 8. Generation of automated data analysis pipelines for the DuraClone B cell panel.

The proportions of cell populations indicated in Supplemental Table 1 for the B cell panel were determined in unfractionated blood from healthy (n = 9) and HSCT (n = 5) subjects using automated gating, 2 different manual analyzers (Manual 1, Manual 2), or a reference manual analyzer. Data with event counts under threshold values for key gates were not included (see Methods, Flow cytometry data analyses). Data from Manual 1 or 2, or automated gating were compared with those from the reference manual to determine (A) Spearman’s correlation coefficients (R_S) or (B) F₁ scores. Box-and-whisker plot: midline is the median, box is the IQR, and the whiskers extend to 1.5 times the IQR. (C and D) Correlation graphs and representative manual and automated plots and gates for (C) plasmablasts and class-switched (CS) memory B cells, both derived from the CD27 versus CD38 plot, pre-gated on IgM^–IgD^– B cells; or (D) naive and IgD^–CD27^– B cells. Red boxes indicate data points and manual or automated gates from which they were derived for the samples with outlying proportions of plasmablasts or IgD^–CD27^– cells.

Figure 9. Validation of automated gating pipelines in an independent dataset.

Blood from healthy subjects enrolled in the ONE (n = 9) or CNTRP (n = 20 for Basic, n = 10–14 for B cells) study was analyzed with the Basic or B cell panels, and data were analyzed manually or using automated pipelines developed using the CNTRP data. All population proportions listed in Supplemental Table 1 for the Basic and B cell panels were measured, and results from automated versus manual analyses were compared. Shown are correlation graphs merging all data from either the CNTRP (training set) or ONE (validation set) study quantified by automated versus manual gating in relation to input gate. Each population proportion is a different color, and each data point represents an individual. Spearman’s correlation coefficient is shown at the upper left. See Supplemental Table 2 for individual r_s values.

Figure 10. Application of approach to alternate flow cytometer platform.

Blood from 5 healthy subjects was stained using off-the-shelf Basic and B cell DuraClone tubes and acquired on a Navios (3 lasers) or an LSR-Fortessa (4 lasers) cytometer. LMD or FSC3.0 files were analyzed manually or with automated pipelines. (A) Comparison of data from cytometers. Each variable from 5 donors is shown as an individual data point, with the LSR-Fortessa data linked to the same data point obtained on the Navios. (B and C) Comparison of results from manual or automated analysis on the indicated cytometer. Each population proportion is a different color, and each data point is an individual. Spearman’s correlation coefficient is shown at the upper left.