Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2015 Sep;34(9):1854-66.
doi: 10.1109/TMI.2015.2412881. Epub 2015 Mar 13.

Multimodal Segmentation of Optic Disc and Cup From SD-OCT and Color Fundus Photographs Using a Machine-Learning Graph-Based Approach

Mohammad Saleh Miri, Michael D Abràmoff, Kyungmoo Lee, Meindert Niemeijer, Jui-Kai Wang, Young H Kwon, Mona K Garvin

Multimodal Segmentation of Optic Disc and Cup From SD-OCT and Color Fundus Photographs Using a Machine-Learning Graph-Based Approach

Mohammad Saleh Miri et al. IEEE Trans Med Imaging. 2015 Sep.

Abstract

In this work, a multimodal approach is proposed to use the complementary information from fundus photographs and spectral domain optical coherence tomography (SD-OCT) volumes in order to segment the optic disc and cup boundaries. The problem is formulated as an optimization problem where the optimal solution is obtained using a machine-learning theoretical graph-based method. In particular, first the fundus photograph is registered to the 2D projection of the SD-OCT volume. Three in-region cost functions are designed using a random forest classifier corresponding to three regions of cup, rim, and background. Next, the volumes are resampled to create radial scans in which the Bruch's Membrane Opening (BMO) endpoints are easier to detect. Similar to in-region cost function design, the disc-boundary cost function is designed using a random forest classifier for which the features are created by applying the Haar Stationary Wavelet Transform (SWT) to the radial projection image. A multisurface graph-based approach utilizes the in-region and disc-boundary cost images to segment the boundaries of optic disc and cup under feasibility constraints. The approach is evaluated on 25 multimodal image pairs from 25 subjects in a leave-one-out fashion (by subject). The performances of the graph-theoretic approach using three sets of cost functions are compared: 1) using unimodal (OCT only) in-region costs, 2) using multimodal in-region costs, and 3) using multimodal in-region and disc-boundary costs. Results show that the multimodal approaches outperform the unimodal approach in segmenting the optic disc and cup.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Cupping causes an increase in cup-to-disc ratio (CDR). (a) CDR = 0.15, (b) CDR = 0.46, (c) CDR = 0.84.
Figure 2
Figure 2
Bruch’s membrane opening (BMO) within an SD-OCT volume. (a) An SD-OCT B-scan with BMO points marked with two filled circles. RPE = retinal pigment epithelium. (b) 3D view of all BMO points for the entire SD-OCT volume. (c) SD-OCT projection image. (d) Projected view of BMO points on SD-OCT projection image.
Figure 3
Figure 3
Flowchart of overall method.
Figure 4
Figure 4
An example of a central B-scan of an SD-OCT volume and intraretinal layer segmentation. (a) The original OCT B-scan. (b) Segmented three layers: the first surface is the ILM (red), the second surface (yellow) is the IS/OS junction, and the third surface (blue) is the lower bound of RPE complex. The pink surface indicates the thin-plate spline fitted to the third surface. (c) Flattened OCT B-scan along with surfaces. (d) 3D view of the three surfaces.
Figure 5
Figure 5
Registration of fundus photograph to projection image of SD-OCT volume. (a) Original fundus photograph. (b) Corresponding projection image of SD-OCT volume. (c) Alignment of fundus photograph to OCT projection image. (d) Registered fundus photograph.
Figure 6
Figure 6
Radial slice segmentation. (a) Example (non-central) scan from original SD-OCT volume demonstrating the how the BMO points may appear close together. (b) Example radial scan (with BMO points appearing as they would in a central scan from the original volume) with segmented surfaces. Interpolation is used to define the second and third surfaces in the neural canal region. (c) The radial projection image.
Figure 7
Figure 7
In-region cost function design flowchart.
Figure 8
Figure 8
OCT features. (a)–(f) Average intensity of subvolumes in z-direction. (g) Distance of first surface to the thin-plate spline fitted to the third surface. (h) The SD-OCT projection image.
Figure 9
Figure 9
Fundus pixel features. From left to right are the filtered image using Gaussian filter bank having sizes σ=4, 6, 8 respectively. (a)–(c) Red channel. (d)–(f) Green channel. (g)–(i) Blue channel. (j)–(l) Dark-bright channel. (m)–(o) Blue-yellow channel. (p)–(r) Red-green channel.
Figure 10
Figure 10
Spatial features. (a)–(c) Three a priori maps corresponding to cup, rim, and optic disc regions derived from PCA. (d) Distance of the x position with respect to the optic disc center. (e) Distance of the y position with respect to the optic disc center. (f) Radial distance with respect to the optic disc center.
Figure 11
Figure 11
Disc-boundary cost function design flowchart.
Figure 12
Figure 12
SWT decomposition. (a)–(d) SWT 6-level decomposition. In each image, upper left is the approximation, upper right is the horizontal, lower left is the vertical, and lower right is the diagonal SWT coefficient.
Figure 13
Figure 13
On-boundary cost function feature set. (a)–(d) Horizontal coefficients of level 1, 2, 3 and 4. (e) Vessel-free projection image. (f) Result of averaging derivative of Gaussian in the vertical direction. (g) Spatial feature imposing the shape of the optic disc boundary. (h) Spatial feature that has the anatomic information of BMO points.
Figure 14
Figure 14
A schematic representation of segmenting the optic disc and cup boundaries using a theoretical graph based approach. (a) Left is the original image and right is the resampled image in radial domain. (b) Example cost of two boundaries for the multiple boundary segmentation problem. The two boundaries divide the images into three regions.
Figure 15
Figure 15
An example of cost functions. (a) The in-region cost function for the background. (b) The in-region cost function for the rim. (c) The in-region cost function for the optic cup. (d) The optic-disc-boundary cost function. Note that there is no cup-boundary cost function.
Figure 16
Figure 16
An example segmentation result. The first row contains the (a) registered fundus photograph with (b) the reference standard boundaries, (c) the boundaries of the first method (using unimodal region costs), (c) the boundaries of the second method (using multimodal region costs), and (d) the boundaries of the third method (using multimodal region plus disc-boundary costs). The blue boundary corresponds to the optic disc boundary and the green boundary corresponds to the cup boundary. The second row contains the boundaries of the methods shown on the SD-OCT projection image. The third row contains a central B-scan of the SD-OCT with green indicating the rim region and red indicating the cup region from the different methods. The last row contains the region-based segmentation results (black = background; gray = rim; white = cup). It is especially noticeable on the inferior (I) and temporal (T) sides of optic disc boundary that the third method has the closest boundary to the reference standard. In addition, the unimodal approach has a relatively smaller optic cup than the multimodal approach in comparison with the reference standard.
See this image and copyright information in PMC

References

    1. Abràmoff MD, Garvin MK, Sonka M. Retinal imaging and image analysis. IEEE Rev. Biomed. Eng. 2010;3:169–208. - PMC - PubMed
    1. Abràmoff MD, Alward WLM, Greenlee EC, Shuba L, Kim CY, Fingert JH, Kwon YH. Automated segmentation of the optic disc from stereo color photographs using physiologically plausible features. Invest. Ophthalmol. Vis. Sci. 2007 Apr.48(4):1665–1673. - PMC - PubMed
    1. Cheng J, Liu J, Xu Y, Yin F, Wong DWK, Tan N-M, Tao D, Cheng C-Y, Aung T, Wong TY. Superpixel classification based optic disc and optic cup segmentation for glaucoma screening. IEEE Trans. Med. Imag. 2013 Jun;32(6):1019–1032. - PubMed
    1. Xu J, Chutatape O, Sung E, Zheng C, Kuan PCT. Optic disk feature extraction via modified deformable model technique for glaucoma analysis. Patt. Rec. 2007 Jul;40(7):2063–2076.
    1. Yin F, Liu J, Ong SH, Sun Y, Wong DWK, Tan NM, Cheung C, Baskaran M, Aung T, Wong TY. Model-based optic nerve head segmentation on retinal fundus images; 33rd Ann. Int. Conf. IEEE EMBS; Aug. 2011; Boston, Massachusetts USA. pp. 2626–2629. - PubMed

Publication types