Computational pathology for musculoskeletal conditions using machine learning: advances, trends, and challenges

Abstract

Histopathology is widely used to analyze clinical biopsy specimens and tissues from pre-clinical models of a variety of musculoskeletal conditions. Histological assessment relies on scoring systems that require expertise, time, and resources, which can lead to an analysis bottleneck. Recent advancements in digital imaging and image processing provide an opportunity to automate histological analyses by implementing advanced statistical models such as machine learning and deep learning, which would greatly benefit the musculoskeletal field. This review provides a high-level overview of machine learning applications, a general pipeline of tissue collection to model selection, and highlights the development of image analysis methods, including some machine learning applications, to solve musculoskeletal problems. We discuss the optimization steps for tissue processing, sectioning, staining, and imaging that are critical for the successful generalizability of an automated image analysis model. We also commenting on the considerations that should be taken into account during model selection and the considerable advances in the field of computer vision outside of histopathology, which can be leveraged for image analysis. Finally, we provide a historic perspective of the previously used histopathological image analysis applications for musculoskeletal diseases, and we contrast it with the advantages of implementing state-of-the-art computational pathology approaches. While some deep learning approaches have been used, there is a significant opportunity to expand the use of such approaches to solve musculoskeletal problems.

Keywords: Computational pathology; Convolutional neural network; Deep learning; Histopathology; Image analysis; Machine learning; Orthopedics; Rheumatology.

Fig. 1

Overview of image analysis and machine learning subdisciplines. Extracting features is a critical step in image analysis and can be generalized to two methods, knowledge-driven and data-driven (A). Once image features are extracted, then one of two main ML subdisciplines are often used, (1) supervised learning and (2) unsupervised learning (B). A non-comprehensive summary of typical tasks solved with each subdiscipline, and common methods used to solve each task are listed below each subdiscipline. Solid lines indicate methodologies that follow the principles of each subdiscipline. Dotted arrows indicate direct links between the methods of disciplines

Fig. 2

Examples of knowledge-driven feature extraction to identify nuclei. There are several mechanisms to extract information from an image. In this example, (A) the red channel of an image of H&E-stained cartilage with chondrocytes has been isolated and (B) thresholded to start the process of identifying the nuclei. (C) To identify the edges of the nuclei, convolutional kernels that have been designed to identify edges are applied (Sobel kernels) and the resulting images (feature maps) are added together. (D) Object detection algorithms, which can trace edges, can then be used to isolate the independent objects (nuclei) within the image. (E) Finally, color and shape features can be calculated to generate information about the nuclei that may help with pathologic analysis

Fig. 3

A generalized pipeline from tissue collection to model building. We have identified six main steps that are crucial in image analysis pipelines that can influence the results of an image analysis model: (1) biospecimen procurement, (2) processing, (3) sectioning, (4) staining, (5) imaging, (6) model selection