Deep learning predicts therapy-relevant genetics in acute myeloid leukemia from Pappenheim-stained bone marrow smears

Abstract

The detection of genetic aberrations is crucial for early therapy decisions in acute myeloid leukemia (AML) and recommended for all patients. Because genetic testing is expensive and time consuming, a need remains for cost-effective, fast, and broadly accessible tests to predict these aberrations in this aggressive malignancy. Here, we developed a novel fully automated end-to-end deep learning pipeline to predict genetic aberrations directly from single-cell images from scans of conventionally stained bone marrow smears already on the day of diagnosis. We used this pipeline to compile a multiterabyte data set of >2 000 000 single-cell images from diagnostic samples of 408 patients with AML. These images were then used to train convolutional neural networks for the prediction of various therapy-relevant genetic alterations. Moreover, we created a temporal test cohort data set of >444 000 single-cell images from further 71 patients with AML. We show that the models from our pipeline can significantly predict these subgroups with high areas under the curve of the receiver operating characteristic. Potential genotype-phenotype links were visualized with 2 different strategies. Our pipeline holds the potential to be used as a fast and inexpensive automated tool to screen patients with AML for therapy-relevant genetic aberrations directly from routine, conventionally stained bone marrow smears already on the day of diagnosis. It also creates a foundation to develop similar approaches for other bone marrow disorders in the future.

Graphical abstract

Figure 1.

Overview of our pipeline. (Left to right) Whole bone marrow smear scan images are processed using a CFM to automatically extract the most relevant single-cell crops, which are rudimentary classified into 4 cell classes. Subsequently, a GFEN is used to predict 5 genetic indicators that are important for first-line therapy decisions (CBFB::MYH11, MRC cytogenetics, FLT3^mut, NPM1^mut, and ELN 2017 favorable risk) based on appropriate single-cell images only. Additionally, visualization strategies were used to gain explainability with respect to the used deep learning models.

Figure 2.

Randomly sampled example image for the classes of the CFM. (A) Images for the 4 classes of the general CFM and (B) of the eCFM are shown.

Figure 3.

Cross-validated prediction of therapeutically relevant genetic groups using the GFEN on the discovery cohort. Deep learning models were trained to predict therapy relevant genetic subgroups from bone marrow smears. The patient-level performance of the GFEN is evaluated for every genetically defined group (ELN 2017 favorable risk, CBFB::MYH11 fusions, NPM1 mutations, FLT3-ITD and -tyrosine kinase domain (TKD) mutations, and MRC cytogenetics) with the AUROC and 2-sided P value for the prediction scores. Values for the validation runs of the fivefold cross-validation are given for the individual folds (cv0, cv1, cv2, cv3, and cv4) and all validation folds of the complete cohort combined (compiled). Error bars show 95% CI. The dashed line represents a P value <.05.

Figure 4.

Highest scoring single-cell images from occlusion sensitivity analysis for the different genetic categories. The images represent the most important images for the decision toward a label.

Figure 5.

Single-cell saliency maps for different genetic categories. The importance of individual pixel for the classification decision are calculated based on gradient computations in the GFEN model and are overlaid onto the original image (high gradient: yellow to green).