Plus disease in retinopathy of prematurity: diagnostic impact of field of view

Abstract

Purpose: To examine the impact of retinal field of view and magnification on interexpert reliability of plus disease diagnosis in retinopathy of prematurity.

Methods: Fifteen wide-angle images from infants with retinopathy of prematurity were cropped and adjusted in magnification to create 2 additional image categories: medium angle (40°-50°) and narrow angle (20°-30°). These 45 images were uploaded to a Web-based system and interpreted independently by 13 experts of retinopathy of prematurity using a 3-level (plus, preplus, neither) and 2-level (plus, not plus) classification. Absolute agreement and kappa statistics were calculated to compare interexpert reliability.

Results: In the 3-level classification, ≥ 70% experts agreed on the same diagnosis in 8 of the 15 wide-angle images (53%), but only in 3 of the 15 medium-angle (20%) and 3 of the 15 narrow-angle (20%) images. In the 2-level classification, ≥ 80% experts agreed on the same diagnosis in 11 of the 15 wide-angle images (73%), but only in 9 of the 15 medium-angle (60%) and 3 of the 15 narrow-angle (20%) images. Mean kappa of each expert compared with all other experts was 0.40 to 0.59 in 8 of 13 experts (62%) using wide-angle images, was 0 to 0.19 in 7 of 13 experts (54%) using medium-angle images, and was 0.20 to 0.39 in 9 of 13 experts (69%) using narrow-angle images.

Conclusion: Interexpert agreement in plus disease diagnosis in wide-angle images is higher than from medium-angle and narrow-angle images. Plus disease is defined using a narrow-angle standard published photograph, yet this study suggests that peripheral findings also contribute to diagnosis.

Figure 1. Diagnostic responses of expert participants based on field of view

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow-angle fields of view. Responses are displayed using (A) 3-level (plus, pre-plus, neither) and (B) 2-level (plus, not plus) categorizations. The number of experts making each diagnosis is shown by field of view (out of 195 total responses for each angle).

Figure 1. Diagnostic responses of expert participants based on field of view

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow-angle fields of view. Responses are displayed using (A) 3-level (plus, pre-plus, neither) and (B) 2-level (plus, not plus) categorizations. The number of experts making each diagnosis is shown by field of view (out of 195 total responses for each angle).

Figure 2. Absolute inter-expert agreement based on field of view

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow-angle fields of view (45 images total). Percentage of experts in agreement of an image diagnosis is shown by field of view and for an increasing number of images. The number of images having a particular percentage of expert agreement are displayed using (A) 3-level (plus, pre-plus, neither) and (B) 2-level (plus, not plus) categorizations.

Figure 2. Absolute inter-expert agreement based on field of view

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow-angle fields of view (45 images total). Percentage of experts in agreement of an image diagnosis is shown by field of view and for an increasing number of images. The number of images having a particular percentage of expert agreement are displayed using (A) 3-level (plus, pre-plus, neither) and (B) 2-level (plus, not plus) categorizations.

Figure 3. Inter-expert kappa statistic based on field of view

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow-angle fields of view (45 images total). The mean kappa of each expert compared to all other experts was calculated for each field of view. The strength of agreement by field of view is shown, as indicated by the number of kappas in certain numerical ranges: a higher kappa implies greater strength. (A) 3-level (plus, pre-plus, neither) and (B) 2-level (plus, not plus) categorization.

Figure 3. Inter-expert kappa statistic based on field of view

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow-angle fields of view (45 images total). The mean kappa of each expert compared to all other experts was calculated for each field of view. The strength of agreement by field of view is shown, as indicated by the number of kappas in certain numerical ranges: a higher kappa implies greater strength. (A) 3-level (plus, pre-plus, neither) and (B) 2-level (plus, not plus) categorization.

Figure 4. Intra-expert kappa statistic based on field of view

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow-angle fields of view (45 images total). Intra-expert kappa measures agreement of an expert’s ratings of the three images of the same retina, one image from each angle. Results are displayed using a 3-level (plus, pre-plus, neither) categorization.

Figure 5. Representative examples of intra-expert agreement

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow angle fields of view. (A), (B), and (C) display wide-angle, medium-angle, and narrow-angle views from one retina. (A) was diagnosed as pre-plus by 9/13 experts, (B) was diagnosed as pre-plus by 8/13 experts, and (C) was diagnosed as pre-plus by 1/13 experts and plus by 12/13 experts. (D), (E), and (F) display wide-angle, medium-angle, and narrow-angle views from a second retina. (D) was diagnosed as pre-plus by 10/13 experts, (E) was diagnosed as pre-plus by 10/13 experts, and (F) was diagnosed as pre-plus by 3/13 experts and plus by 10/13 experts.

Figure 5. Representative examples of intra-expert agreement

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow angle fields of view. (A), (B), and (C) display wide-angle, medium-angle, and narrow-angle views from one retina. (A) was diagnosed as pre-plus by 9/13 experts, (B) was diagnosed as pre-plus by 8/13 experts, and (C) was diagnosed as pre-plus by 1/13 experts and plus by 12/13 experts. (D), (E), and (F) display wide-angle, medium-angle, and narrow-angle views from a second retina. (D) was diagnosed as pre-plus by 10/13 experts, (E) was diagnosed as pre-plus by 10/13 experts, and (F) was diagnosed as pre-plus by 3/13 experts and plus by 10/13 experts.

Figure 5. Representative examples of intra-expert agreement

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow angle fields of view. (A), (B), and (C) display wide-angle, medium-angle, and narrow-angle views from one retina. (A) was diagnosed as pre-plus by 9/13 experts, (B) was diagnosed as pre-plus by 8/13 experts, and (C) was diagnosed as pre-plus by 1/13 experts and plus by 12/13 experts. (D), (E), and (F) display wide-angle, medium-angle, and narrow-angle views from a second retina. (D) was diagnosed as pre-plus by 10/13 experts, (E) was diagnosed as pre-plus by 10/13 experts, and (F) was diagnosed as pre-plus by 3/13 experts and plus by 10/13 experts.

Figure 5. Representative examples of intra-expert agreement

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow angle fields of view. (A), (B), and (C) display wide-angle, medium-angle, and narrow-angle views from one retina. (A) was diagnosed as pre-plus by 9/13 experts, (B) was diagnosed as pre-plus by 8/13 experts, and (C) was diagnosed as pre-plus by 1/13 experts and plus by 12/13 experts. (D), (E), and (F) display wide-angle, medium-angle, and narrow-angle views from a second retina. (D) was diagnosed as pre-plus by 10/13 experts, (E) was diagnosed as pre-plus by 10/13 experts, and (F) was diagnosed as pre-plus by 3/13 experts and plus by 10/13 experts.

Figure 5. Representative examples of intra-expert agreement

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow angle fields of view. (A), (B), and (C) display wide-angle, medium-angle, and narrow-angle views from one retina. (A) was diagnosed as pre-plus by 9/13 experts, (B) was diagnosed as pre-plus by 8/13 experts, and (C) was diagnosed as pre-plus by 1/13 experts and plus by 12/13 experts. (D), (E), and (F) display wide-angle, medium-angle, and narrow-angle views from a second retina. (D) was diagnosed as pre-plus by 10/13 experts, (E) was diagnosed as pre-plus by 10/13 experts, and (F) was diagnosed as pre-plus by 3/13 experts and plus by 10/13 experts.

Figure 5. Representative examples of intra-expert agreement

Thirteen experts were asked to diagnose presence of plus disease in 15 retinal images cropped to wide-angle, medium-angle, and narrow angle fields of view. (A), (B), and (C) display wide-angle, medium-angle, and narrow-angle views from one retina. (A) was diagnosed as pre-plus by 9/13 experts, (B) was diagnosed as pre-plus by 8/13 experts, and (C) was diagnosed as pre-plus by 1/13 experts and plus by 12/13 experts. (D), (E), and (F) display wide-angle, medium-angle, and narrow-angle views from a second retina. (D) was diagnosed as pre-plus by 10/13 experts, (E) was diagnosed as pre-plus by 10/13 experts, and (F) was diagnosed as pre-plus by 3/13 experts and plus by 10/13 experts.