Advances in predicting omics profiles from imaging data

Abstract

While traditional imaging techniques, such as histopathology, are often part of clinical workflows, molecular profiling remains more difficult to conduct and is less cost-effective. Thus, the prediction of molecular 'omics' data directly from imaging has emerged as an appealing alternative. While existing reviews have mentioned image-based prediction of biomarkers within specific disease contexts, this review provides a comprehensive overview of current methods that leverage imaging to predict (i) DNA-based aberrations, (ii) bulk transcriptomic profiles, (iii) single-cell transcriptomics, and (iv) spatial transcriptomics across disease contexts and imaging modalities. To address the complexity of these predictive tasks, we find that many studies employ cutting-edge deep learning strategies for image processing, feature extraction, feature aggregation, and downstream molecular prediction. In this review, we highlight the diverse applications of both deep learning-based and modern statistical frameworks designed for image-based omics prediction. The insights gleaned from these inferred molecular data have broad clinical relevance and will continue to improve our understanding of the relationships between molecular and visual features, paving the way for new diagnostic and therapeutic applications.

Keywords: deep learning; genomics; histology imaging; transcriptomics.

Figure 1

Schematic of end-to-end workflow for leveraging diverse imaging modalities to predict molecular and genomic features. (left) Input imaging data ranges from tissue-level histology to sub-cellular Raman microscopy. (center) These inputs are mapped to various prediction tasks, primarily focusing on transcriptomic profiles and DNA-based alterations. (right) A suite of computational methods, dominated by deep learning architectures, are employed to predict meaningful biological signals from the raw imagery. The connecting lines between imaging types to prediction tasks, and between prediction tasks to computational methods, represent the presence of published algorithms linking them. Notably, histology imaging is linked to all molecular prediction tasks, and spatial RNA-seq prediction is conducted using all listed computational methods.

Figure 2

Taxonomy and interrelationships of methodological themes in image-based omics prediction. The hierarchical diagram organizes modeling approaches based on their conceptual lineage and data handling strategies, from feature-driven statistical learning to spatially aware deep learning. The connecting lines represent the presence of integrated frameworks that combine multiple approaches (e.g. CNN-based feature extraction combined with transformer-based spatial modeling).