Deep Learning Segmentation, Visualization, and Automated 3D Assessment of Ciliary Body in 3D Ultrasound Biomicroscopy Images

Abstract

Purpose: This study aimed to develop a fully automated deep learning ciliary body segmentation and assessment approach in three-dimensional ultrasound biomicroscopy (3D-UBM) images.

Methods: Each 3D-UBM eye volume was aligned to the optic axis via multiplanar reformatting. Ciliary muscle and processes were manually annotated, and Deeplab-v3+ models with different loss functions were trained to segment the ciliary body (ciliary muscle and processes) in both en face and radial images.

Results: We trained and tested the models on 4320 radial and 3864 en face images from 12 cadaver eye volumes. Deep learning models trained on radial images with Dice loss achieved the highest mean F1-score (0.89) for ciliary body segmentation. For three-class segmentation (ciliary muscle, processes, and background), radial images with Dice loss achieved the highest mean F1-score (0.75 for the ciliary process and 0.82 for the ciliary muscle). Part of the ciliary muscle (10.9%) was misclassified as the ciliary process and vice versa, which occurred owing to the difficulty in differentiating the ciliary muscle-processes border, even by experts. Deep learning segmentation made further editing by experts at least seven times faster than a fully manual approach. In eight cadaver eyes, the average ciliary muscle, process, and body volumes were 56 ± 9, 43 ± 13, and 99 ± 18 mm3, respectively. The average surface area of the ciliary muscle, process, and body were 346 ± 45, 363 ± 83, and 709 ± 80 mm2, respectively. We performed transscleral cyclophotocoagulation in cadaver eyes to shrink the ciliary processes. Both manual and automated measurements from deep learning segmentation show a decrease in volume, surface area, and 360° cross-sectional area measurements.

Conclusions: The proposed deep learning segmentation of the ciliary body and 3D measurements showed transscleral cyclophotocoagulation-related changes in the ciliary body.

Translational relevance: Automated ciliary body assessment using 3D-UBM has the translational potential for ophthalmic treatment planning and monitoring.

Figure 1.

A 3D-UBM system and images of ciliary body tissues. The 2D-UBM probe is translated across the eye using a motorized stage and surgical microscope to acquire a 2D image (yz plane) stack along the slow-scan direction (x-axis). From the acquired rendered volume, sagittal image (yz), axial image (xz), and en face image (xy) of the ciliary muscle and ciliary processes can be visualized clearly.

Figure 2.

Alignment of the 3D-UBM volume to the optic axis. (a) Direction convention used in 3D-UBM; (b) before resampling, the z-axis is not parallel to the optic axis (green line); (c) misaligned en face view contains parts of the iris; (d) sagittal view after aligning volume to the optic axis; and (e) correctly aligned en face view of the ciliary body, which contains ciliary muscle and processes.

Figure 3.

Segmentation of ocular structures in the anterior segment of the eye. Green, anterior chamber; purple, iris; blue, ciliary muscle; red, ciliary processes. Ciliary muscle and ciliary processes together are called the ciliary body. Experts perform manual annotation by looking at the ciliary body in 3D. From manual annotation, 3D volumetric and area measurements of ciliary muscle and processes can be made.

Figure 4.

Deep learning convolution neural network (Deeplab-v3+) architecture for the semantic segmentation of the ciliary body. In the output-labeled image, the red area is the ciliary process, blue is the ciliary muscle, and black is the background. For two-class segmentation, the ciliary muscle and body are lumped together and noted as the ciliary body.

Figure 5.

Deep learning segmentation of the ciliary body. All four panels (a)–(d) show expert annotation of the ciliary body, prediction from deep learning segmentation, and the difference (top to bottom). Areas with yellow arrows show incorrect segmentation of the ciliary body. Areas with a green oval indicate possible ciliary body location that might have been missed by experts.

Figure 6.

Deep learning segmentation of the ciliary muscle and ciliary processes. Panels (a)–(d) show expert annotation of the ciliary muscle (green) and ciliary processes (purple), prediction from deep learning segmentation, and difference (top to bottom). Areas with yellow arrows show incorrect segmentation of tissues, and areas with green ovals indicate possible volumes that might have been missed by experts. Predicted labels are smoother possibly because interpolation and small discontinuities in ground truth are not maintained.

Figure 7.

Visualization of ciliary muscle and processes in 3D-UBM volume. Rendering of the segmentation of the ciliary body provides unique visualization and measurements, that is, the total ciliary muscle and processes volume and surface area, number of processes, average ciliary process volume, and area.

Figure 8.

Ciliary body size reduction after TS-CPC treatment. Shown are renderings using semiautomated ciliary muscle and processes segmentation before (a, c) and after (b, d) TS-CPC treatments on two cadaver eyes. Shown are both anterior to posterior views of the ciliary body (a, b) and posterior to anterior views (c, d). Both eyes show a visible decrease in ciliary processes.

Figure 9.

Comparison of manual and automated measurements of the ciliary body cross-sectional area from radial images. The 3D-UBM enables 360° 2D measurements of the ciliary body. For two cadaver eyes after CPC, 2D cross-sectional areas of the ciliary body were reduced in the location (12 o'clock to 6 o'clock) where CPC was performed.