A Novel Tool for Collaborative and Blinded Orthopedic Image Analysis

Abstract

Image analysis plays a central role in orthopedics and research but comes with many challenges, including anonymization, bias, and achieving efficient analyses using multiple independent observers. Appropriate software is still lacking. Tyche is a free online tool that displays images in a random order without showing any metadata. Additionally, when using Tyche, observers can store results in the same window, and the final results are immediately visible to the project manager. In this study, we compared results from Tyche with those from a validated tool. One hundred pelvic radiographs were analyzed separately by five orthopedic surgeons using both Tyche and the validated software. Common orthopedic measurement modalities and scores were determined. The methods were compared using intra-class correlations and Fleiss' kappa coefficients as well as Bland-Altman plots. Significant correlations ranging from r = 0.17 (Kallgren and Lawrence Score) to r = 0.99 (area measurements) were calculated for inter- and intraobserver agreements between the two tools for all measurements. The Bland-Altman plots indicated the non-inferiority of either tool. The images were analyzed significantly faster when Tyche was used. We conclude that Tyche is a valid tool for use in orthopedic image analysis. Tyche could be utilized for determining inter- and intraobserver agreements, in multicenter studies and for score validations.

Keywords: Tyche; blinded image analysis; collaborative image analysis.

Figure 1

Parameters that were measured using mediCAD^® and Tyche. (A) (length): the distance between the Köhler teardrops; (B) (angle): the collum–center–diaphysis angle; (C) (area): the left obturator foramen; (D) (score): the Kellgren and Lawrence score, with sample images for grades 0, 1, 2, 3, and 4, defined as follows: grade 0 (none) = a definite absence of changes in osteoarthritis on an X-ray image; grade 1 (doubtful) = doubtful joint space narrowing and possible osteophytic lipping; grade 2 (minimal) = definite osteophytes and possible joint space narrowing; grade 3 (moderate) = moderate multiple osteophytes, definite narrowing of the joint space, some sclerosis, and possible deformity of bone ends; grade 4 (severe) = large osteophytes, marked narrowing of the joint space, severe sclerosis, and definite deformity of bone ends.

Figure 2

Image analysis and immediate visibility of results, as seen when using Tyche in a web browser. (A) Shows the core functions of Tyche, which can be seen when analyzing images inside a web browser. On the left side, an anonymized and randomly selected image from the database is shown. A set of tools is provided on top. The right side shows the questions and answers defined by the project master that must be answered by the analyzers for every single image. (B) Shows the same image being analyzed by analyzers using mediCAD^®. When using mediCAD^®, the results were noted on a separate spreadsheet. No Excel or datasheet-based protocolling was available. (C) Shows the results that are visible to the project master immediately after an analyzer finishes performing the analysis using Tyche. Every row represents an image; the first column shows the file name, and the following columns show the results for the questions shown in the headline of each column.

Figure 3

Flowchart showing the study design. In total, 100 hip radiographs were analyzed by 5 orthopedic surgeons, once using mediCAD^® and once using Tyche. The observers carried out length, angle, and area measurements and determined the Kallgren and Lawrence score for each radiograph. The results were compared using intra-class correlations (ICCs), Fleiss’ kappa coefficients, and Bland–Altman plots.

Figure 4

Non-inferiority of Tyche vs. mediCAD^® for length (A), area (B), and angle measurements (C), as shown via Bland–Altman plots.