Evaluation and understanding of automated urinary stone recognition methods

Abstract

Objective: To assess the potential of automated machine-learning methods for recognizing urinary stones in endoscopy.

Materials and methods: Surface and section images of 123 urinary calculi (109 ex vivo and 14 in vivo stones) were acquired using ureteroscopes. The stones were more than 85% 'pure'. Six classes of urolithiasis were represented: Groups I (calcium oxalate monohydrate, whewellite), II (calcium oxalate dihydrate, weddellite), III (uric acid), IV (brushite and struvite stones), and V (cystine). The automated stone recognition methods that were developed for this study followed two types of approach: shallow classification methods and deep-learning-based methods. Their sensitivity, specificity and positive predictive value (PPV) were evaluated by simultaneously using stone surface and section images to classify them into one of the main morphological groups (subgroups were not considered in this study).

Results: Using shallow methods (based on texture and colour criteria), relatively high sensitivity, specificity and PPV for the six classes were attained: 91%, 90% and 89%, respectively, for whewellite; 99%, 98% and 99% for weddellite; 88%, 89% and 88% for uric acid; 91%, 89% and 90% for struvite; 99%, 99% and 99% for cystine; and 94%, 98% and 99% for brushite. Using deep-learning methods, the sensitivity, specificity and PPV for each of the classes were as follows: 99%, 98% and 97% for whewellite; 98%, 98% and 98% for weddellite; 97%, 98% and 98% for uric acid; 97%, 97% and 96% for struvite; 99%, 99% and 99% for cystine; and 94%, 97% and 98% for brushite.

Conclusion: Endoscopic stone recognition is challenging, and few urologists have sufficient expertise to achieve a diagnosis performance comparable to morpho-constitutional analysis. This work is a proof of concept that artificial intelligence could be a solution, with promising results achieved for pure stones. Further studies on a larger panel of stones (pure and mixed) are needed to further develop these methods.

Keywords: automated kidney stone recognition; deep learning; morphoconstitutional analysis; ureteroscopy; urolithiasis.

Fig. 1

Image acquisition technique and examples of acquired images. (A) Device (left) and installation (right) during image acquisition. (B) Examples of calculi images used. COM, calcium oxalate monohydrate; COD, calcium oxalate dihydrate.

Fig. 2

Scatterplot using feature extractions. Each cloud represents a type of stone after classification by ‘shallow methods’ (A) and by the ‘deep‐learning method’ (B). For each method, the results are shown for surface patches (a), section patches (b) and the effect of mixing both of them (c). The more distant each cloud was from other clouds, the more discriminating the features extracted from the images were. Using the deep‐learning method (B), clouds are more distant from each other, offering higher discrimination than that achieved using the shallow method. AU, uric acid (dark purple); BR, brushite (green); CYS, cystine (blue); STR, struvite (yellow); WD, weddellite (calcium oxalate dihydrate [COD]; red); WW whewellite (calcium oxalate monohydrate [COM]; light purple); Sur, surface; Sec, section. HSI, hue, saturation intensity colour model; LBP, local binary patterns. (a) Scatterplot for individual surface patches. (b) Scatterplot for individual section patches. (c) Scatterplot obtained using both types of patches.

Fig. 3

Review of the literature.