Multimodal Segmentation of Optic Disc and Cup From SD-OCT and Color Fundus Photographs Using a Machine-Learning Graph-Based Approach

Abstract

In this work, a multimodal approach is proposed to use the complementary information from fundus photographs and spectral domain optical coherence tomography (SD-OCT) volumes in order to segment the optic disc and cup boundaries. The problem is formulated as an optimization problem where the optimal solution is obtained using a machine-learning theoretical graph-based method. In particular, first the fundus photograph is registered to the 2D projection of the SD-OCT volume. Three in-region cost functions are designed using a random forest classifier corresponding to three regions of cup, rim, and background. Next, the volumes are resampled to create radial scans in which the Bruch's Membrane Opening (BMO) endpoints are easier to detect. Similar to in-region cost function design, the disc-boundary cost function is designed using a random forest classifier for which the features are created by applying the Haar Stationary Wavelet Transform (SWT) to the radial projection image. A multisurface graph-based approach utilizes the in-region and disc-boundary cost images to segment the boundaries of optic disc and cup under feasibility constraints. The approach is evaluated on 25 multimodal image pairs from 25 subjects in a leave-one-out fashion (by subject). The performances of the graph-theoretic approach using three sets of cost functions are compared: 1) using unimodal (OCT only) in-region costs, 2) using multimodal in-region costs, and 3) using multimodal in-region and disc-boundary costs. Results show that the multimodal approaches outperform the unimodal approach in segmenting the optic disc and cup.

Figure 1

Cupping causes an increase in cup-to-disc ratio (CDR). (a) CDR = 0.15, (b) CDR = 0.46, (c) CDR = 0.84.

Figure 2

Bruch’s membrane opening (BMO) within an SD-OCT volume. (a) An SD-OCT B-scan with BMO points marked with two filled circles. RPE = retinal pigment epithelium. (b) 3D view of all BMO points for the entire SD-OCT volume. (c) SD-OCT projection image. (d) Projected view of BMO points on SD-OCT projection image.

Figure 3

Flowchart of overall method.

Figure 4

An example of a central B-scan of an SD-OCT volume and intraretinal layer segmentation. (a) The original OCT B-scan. (b) Segmented three layers: the first surface is the ILM (red), the second surface (yellow) is the IS/OS junction, and the third surface (blue) is the lower bound of RPE complex. The pink surface indicates the thin-plate spline fitted to the third surface. (c) Flattened OCT B-scan along with surfaces. (d) 3D view of the three surfaces.

Figure 5

Registration of fundus photograph to projection image of SD-OCT volume. (a) Original fundus photograph. (b) Corresponding projection image of SD-OCT volume. (c) Alignment of fundus photograph to OCT projection image. (d) Registered fundus photograph.

Figure 6

Radial slice segmentation. (a) Example (non-central) scan from original SD-OCT volume demonstrating the how the BMO points may appear close together. (b) Example radial scan (with BMO points appearing as they would in a central scan from the original volume) with segmented surfaces. Interpolation is used to define the second and third surfaces in the neural canal region. (c) The radial projection image.

Figure 7

In-region cost function design flowchart.

Figure 8

OCT features. (a)–(f) Average intensity of subvolumes in z-direction. (g) Distance of first surface to the thin-plate spline fitted to the third surface. (h) The SD-OCT projection image.

Figure 9

Fundus pixel features. From left to right are the filtered image using Gaussian filter bank having sizes σ=4, 6, 8 respectively. (a)–(c) Red channel. (d)–(f) Green channel. (g)–(i) Blue channel. (j)–(l) Dark-bright channel. (m)–(o) Blue-yellow channel. (p)–(r) Red-green channel.

Figure 10

Spatial features. (a)–(c) Three a priori maps corresponding to cup, rim, and optic disc regions derived from PCA. (d) Distance of the x position with respect to the optic disc center. (e) Distance of the y position with respect to the optic disc center. (f) Radial distance with respect to the optic disc center.

Figure 11

Disc-boundary cost function design flowchart.

Figure 12

SWT decomposition. (a)–(d) SWT 6-level decomposition. In each image, upper left is the approximation, upper right is the horizontal, lower left is the vertical, and lower right is the diagonal SWT coefficient.

Figure 13

On-boundary cost function feature set. (a)–(d) Horizontal coefficients of level 1, 2, 3 and 4. (e) Vessel-free projection image. (f) Result of averaging derivative of Gaussian in the vertical direction. (g) Spatial feature imposing the shape of the optic disc boundary. (h) Spatial feature that has the anatomic information of BMO points.

Figure 14

A schematic representation of segmenting the optic disc and cup boundaries using a theoretical graph based approach. (a) Left is the original image and right is the resampled image in radial domain. (b) Example cost of two boundaries for the multiple boundary segmentation problem. The two boundaries divide the images into three regions.

Figure 15

An example of cost functions. (a) The in-region cost function for the background. (b) The in-region cost function for the rim. (c) The in-region cost function for the optic cup. (d) The optic-disc-boundary cost function. Note that there is no cup-boundary cost function.

Figure 16

An example segmentation result. The first row contains the (a) registered fundus photograph with (b) the reference standard boundaries, (c) the boundaries of the first method (using unimodal region costs), (c) the boundaries of the second method (using multimodal region costs), and (d) the boundaries of the third method (using multimodal region plus disc-boundary costs). The blue boundary corresponds to the optic disc boundary and the green boundary corresponds to the cup boundary. The second row contains the boundaries of the methods shown on the SD-OCT projection image. The third row contains a central B-scan of the SD-OCT with green indicating the rim region and red indicating the cup region from the different methods. The last row contains the region-based segmentation results (black = background; gray = rim; white = cup). It is especially noticeable on the inferior (I) and temporal (T) sides of optic disc boundary that the third method has the closest boundary to the reference standard. In addition, the unimodal approach has a relatively smaller optic cup than the multimodal approach in comparison with the reference standard.