Predictive value of preclinical models for CAR-T cell therapy clinical trials: a systematic review and meta-analysis

Abstract

Background Experimental mouse models are indispensable for the preclinical development of cancer immunotherapies, whereby complex interactions in the tumor microenvironment can be somewhat replicated. Despite the availability of diverse models, their predictive capacity for clinical outcomes remains largely unknown, posing a hurdle in the translation from preclinical to clinical success. Methods This study systematically reviews and meta-analyzes clinical trials of chimeric antigen receptor (CAR)-T cell monotherapies with their corresponding preclinical studies. Adhering to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, a comprehensive search of PubMed and ClinicalTrials.gov was conducted, identifying 422 clinical trials and 3,157 preclinical studies. From these, 105 clinical trials and 180 preclinical studies, accounting for 44 and 131 distinct CAR constructs, respectively, were included. Results Patients' responses varied based on the target antigen, expectedly with higher efficacy and toxicity rates in hematological cancers. Preclinical data analysis revealed homogeneous and antigen-independent efficacy rates. Our analysis revealed that only 4% (n=12) of mouse studies used syngeneic models, highlighting their scarcity in research. Three logistic regression models were trained on CAR structures, tumor entities, and experimental settings to predict treatment outcomes. While the logistic regression model accurately predicted clinical outcomes based on clinical or preclinical features (Macro F1 and area under the curve (AUC)>0.8), it failed in predicting preclinical outcomes from preclinical features (Macro F1<0.5, AUC<0.6), indicating that preclinical studies may be influenced by experimental factors not accounted for in the model. Conclusion These findings underscore the need to better understand the experimental factors enhancing the predictive accuracy of mouse models in preclinical settings.

Keywords: Chimeric antigen receptor - CAR; Hematologic Malignancies; META-ANALYSIS; Solid tumor; T cell.

Figure 1. Overview of literature review and data collection process. Literature search performed until December 1, 2023, on Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 led to 105 clinical trials included in our study. Using the target antigens from the included clinical trials (dashed arrow), 303 relevant preclinical studies employing the same target antigen were identified and included in our analysis. CAR, chimeric antigen receptor.

Figure 2. Hematological tumors are associated with higher response rates and toxicity than solid tumors in clinical CAR-T cell trials. (A) Number of clinical trials analyzed for each included target antigen. (B) Total number of included patients for hematological and solid cancer clinical trials. (C) Detailed information regarding costimulatory domain of CAR construct, as well as source of single-chain variable fragment. (D) ORR separated by target antigen. (E) ORR of all clinical trials for hematological and solid tumor entities. (F) Percentage of patients experiencing ICANS in hematological and solid tumor entities. (G) Percentage of patients experiencing CRS in hematological and solid tumor entities. (H) Correlation between ORR and occurrence of side effects in terms of CRS in patients. (I) Fraction of patients experiencing graft-versus-host disease. Distribution of patients experiencing (J) thrombocytopenia, (K) neutropenia, (L) anemia, (M) lymphopenia or (N) leukopenia. CAR, chimeric antigen receptor; CRS, cytokine release syndrome; ICANS, immune effector cell-associated neurotoxicity syndrome; ORR, overall response rate.

Figure 3. Preclinical mouse studies of CAR-T cell therapy in hematological and solid cancers are primarily of immunodeficient nature. (A) Overall number of preclinical entries per CAR target for either hematological or solid tumors. (B) Overall number of preclinical entries per disease entity for either hematological or solid tumors. (C) Sum of all mice belonging to CAR treatment groups for either category of tumors. (D) Overall responses as for tumor clearance, partial response (tumor decrease and slower growth) or no benefit for all mice, non-immunocompetent mice (E) and immunocompetent mice (F) for either type of tumor. (G) Alluvial plot displaying the different proportions of preclinical studies according to the single-chain variable fragment and costimulatory domain of their CAR molecule for either type of tumor. (F) Alluvial plot highlighting, in yellow, the proportion of preclinical studies employing fully syngeneic mouse models for in vivo CAR-T cell investigation. AML, acute myeloid leukemia; B-ALL, B-cell acute lymphoblastic leukemia; B-NHL, B-cell non-Hodgkin lymphoma; BRCA, breast adenocarcinoma; CAR, chimeric antigen receptor; CNS, central nervous system tumors; CRC, colorectal carcinoma; ESO, esophageal cancer; GBC, gallbladder cancer; GBM, glioblastoma; GC, gastric cancer; GLM, glioma; HCC, hepatocellular carcinoma; HL, Hodgkin’s lymphoma; LC, lung cancer; MCT, mast cell tumor; MDBL, medulloblastoma; MESO, mesothelioma; MLNM, melanoma; MM, multiple myeloma; NB, neuroblastoma; NPC, nasopharyngeal cancer; OS, osteosarcoma; OVC, ovarian cancer; PAAD, pancreatic adenocarcinoma; PDX, patient-derived xenografts; PRAD, prostate adenocarcinoma; RB, retinoblastoma; RCC, renal cell carcinoma; SARCO, sarcoma; T-ALL, T-cell acute lymphoblastic leukemia; TC, testicular cancer; T-NHL, T-cell non-Hodgkin’s lymphoma; USCC, serous carcinoma of the uterine cervix.

Figure 4. Machine learning analysis of preclinical and clinical datasets identifies tumor type as the most predictive feature across both classification and regression tasks. (A) Schematic outline of the model training and testing strategies: Models A, B, and C are classification models, whereas Model D is a regression model. Models A, C, and D were trained and validated using fivefold cross-validation; specifically, Model A used preclinical data, while Models C and D used clinical data. Model B was trained and validated on preclinical data and subsequently tested on clinical data. (B–C) Performance metrics, including macro F1 score (B) and AUC (C), are reported for Models A, B, and C across the entire dataset (“All tumors”), hematological and solid tumor subsets. Results for the solid tumor subset are only presented for Model A due to the limited size and label imbalance of the clinical solid tumor subset (online supplemental methods). Horizontal, dashed lines indicate the performance of a model using random guessing as a baseline (online supplemental methods). Error bars represent the SD of the scores across the 5*10 validation folds (online supplemental methods) for Models A and C. (D) Feature importance scores from Models A, B and C considering hematologic and solid tumors (“All tumors”). Tumor type (solid vs hematologic) consistently emerges as the most predictive feature for both Models B and C. In Model B, this is followed by “TM domain”, whereas in Model C, “Preconditioning” ranks second. All feature importance scores are normalized and expressed on a relative scale. (E) Scatter plots show true versus predicted overall response rates (ORR) across all cross-validation folds for Model D, with a moderate correlation (mean R²=0.51). (F) Feature weights indicate that lower predicted ORRs are mainly linked to solid tumors, which have the most negative weight. AUC, area under the curve; CAR, chimeric antigen receptor; scFv, single-chain variable fragment; TM, transmembrane.