Cascaded learning in intravascular ultrasound: coronary stent delineation in manual pullbacks

Abstract

Purpose: Implanting stents to re-open stenotic lesions during percutaneous coronary interventions is considered a standard treatment for acute or chronic coronary syndrome. Intravascular ultrasound (IVUS) can be used to guide and assess the technical success of these interventions. Automatically segmenting stent struts in IVUS sequences improves workflow efficiency but is non-trivial due to a challenging image appearance entailing manifold ambiguities with other structures. Manual, ungated IVUS pullbacks constitute a challenge in this context. We propose a fully data-driven strategy to first longitudinally detect and subsequently segment stent struts in IVUS frames. Approach: A cascaded deep learning approach is presented. It first trains an encoder model to classify frames as "stent," "no stent," or "no use." A segmentation model then delineates stent struts on a pixel level only in frames with a stent label. The first stage of the cascade acts as a gateway to reduce the risk for false positives in the second stage, the segmentation, which is trained on a smaller and difficult-to-annotate dataset. Training of the classification and segmentation model was based on 49,888 and 1826 frames of 74 sequences from 35 patients, respectively. Results: The longitudinal classification yielded Dice scores of 92.96%, 82.35%, and 94.03% for the classes stent, no stent, and no use, respectively. The segmentation achieved a Dice score of 65.1% on the stent ground truth (intra-observer performance: 75.5%) and 43.5% on all frames (including frames without stent, with guidewires, calcium, or without clinical use). The latter improved to 49.5% when gating the frames by the classification decision and further increased to 57.4% with a heuristic on the plausible stent strut area. Conclusions: A data-driven strategy for segmenting stents in ungated, manual pullbacks was presented-the most common and practical scenario in the time-critical clinical workflow. We demonstrated a mitigated risk for ambiguities and false positive predictions.

Keywords: coronary; detection; intravascular ultrasound; segmentation; stent.

Fig. 1

Illustration of the cascaded concept. Frames of the manual pullback are first analyzed by an encoder network, which decides on one of three classes per frame: stent, no stent, or no use. Only stent frames are then passed on to the encoder–decoder to segment the stent struts. Apart from the favorable training setup, this also targets a reduction of false positive predictions on frames that do not show a stent anyway.

Fig. 2

(a) Network architectures for stent detection (encoder network) and (b) stent segmentation (encoder–decoder network with skip connections between both parts).

Fig. 3

Confusion matrix for the stent detection network after pooling the results on all five test folds.

Fig. 4

(a) ROC curve for varying thresholds on the output probability maps. ROC-AUC values are listed for a threshold of $t=0.5$. (b) Dice curves showing the dependence of the Dice score on the chosen threshold. A good compromise is achieved when thresholding all classes at 0.5.

Fig. 5

Example pullback along with encoder results. The first two rows show orthogonal cuts through the pullback in the longitudinal direction, and the lower three graphs show the ground truth (dash-dotted red) and predictions (solid blue line Monte Carlo mean and blue shading Monte Carlo estimate of the standard deviation). Regions where none of the three classes has a red ground truth value at 1 have not been annotated by the expert. Here, predictions cannot be compared with a target label. During the last frames, the transducer was covered by the catheter, which is correctly recognized as no use.

Fig. 6

Example frames labeled with their corresponding predictions from the stent output of the detector network: (a) and (b) true positive decisions (typical to challenging examples from left to right), (c) false negative decisions (detector missed ground truth annotations), and (d) false positive detections (wrong predictions without ground truth label).

Fig. 7

Sorted rank plots for the precision score computed per frame. Dashed lines indicate average metric on the intra-observer variance set. (a) Precision ranking for all samples with annotated ground truth. (b) Precision ranking for all frames containing automatic segmentations.

Fig. 8

Sorted rank plots for recall and Dice scores computed per frame. Dashed lines indicate average metric on the intra-observer variance set. (a) Recall ranking for all frames with annotated ground truth. (b) Dice score ranking for all frames with annotated ground truth.

Fig. 9

Score matrices for the segmentation network. Scores are presented for different frame supports and post-processing (pp) steps: no post-processing (no-pp), segmentation rejection based on a $<N\text{pixel}$ threshold (N_px) and based on detector decisions (detect). (a) Dice scores for three different post-processing scenarios (vertical) on different frame supports (horizontal). The effect of the detector is evaluated on all frames (segmentations are corrected based on detector decision) and on frames with positive detector decision for stent only. (b) FPR (listed as average number of pixels in a $224\times 224$ frame) for four different post-processing scenarios (vertical) on different frame supports (horizontal). A FPR of 0.25% or 125 pixels ($0.249{\mathrm{mm}}^2$) roughly corresponds to the area of one stent strut as pictured by the imaging modality.

Fig. 10

DCA and SMSD scores of the segmentation network. The former is reported in percent and the latter in pixels, i.e., multiples of the image resolution 0.0446 mm. Equivalent to the Dice scores in Fig. 9, the results are shown without post-processing (no-pp) and after applying the 400 or 500 px heuristic. Again, we evaluate on all frames, stent-only frames, and the two cascading scenarios.

Fig. 11

Example frames drawn in steps of constant frame proportions from the score rankings. Therefore, the plots provide a representative selection of frames covering the full range of scores achieved per frame (scores decrease from left to right). Green: true positive pixels, Yellow: false negative pixels, Red: false positive pixels. (a) Frames drawn at equal spacings from the Dice rankings [Fig. 7(b)]. (b) Frames drawn at equal spacings from the precision rankings [Fig. 8(a)].