Unsupervised Learning for Robust Respiratory Signal Estimation From X-Ray Fluoroscopy

Abstract

Respiratory signals are required for image gating and motion compensation in minimally invasive interventions. In X-ray fluoroscopy, extraction of a respiratory signal can be challenging due to characteristics of interventional imaging, in particular injection of contrast agent and automatic exposure control. We present a novel method for respiratory signal extraction based on dimensionality reduction that can tolerate these events. Images are divided into patches of multiple sizes. Low-dimensional embeddings are generated for each patch using illumination-invariant kernel PCA. Patches with respiratory information are selected automatically by agglomerative clustering. The signals from this respiratory cluster are combined robustly to a single respiratory signal. In the experiments, we evaluate our method on a variety of scenarios. If the diaphragm is visible, we track its superior-inferior motion as ground truth. Our method has a correlation coefficient of more than 91% with the ground truth irrespective of whether or not contrast agent injection or automatic exposure control occur. Additionally, we show that very similar signals are estimated from biplane sequences and from sequences without visible diaphragm. Since all these cases are handled automatically, the method is robust enough to be considered for use in a clinical setting.

Fig. 1

A biplane X-ray system used for interventional applications (Fig. 1a) and an X-ray image of a pig with contrast agent injection in the pulmonary arteries (Fig. 1b).

Fig. 2

Illustration of the proposed respiratory signal estimation. From the medical image, non-overlapping patches are extracted at multiple resolutions (a). Kernel PCA is applied to each patch leading to a set of low-dimensional embeddings, where each color represents a different dimension. Hierarchical clustering, represented by a dendrogram, finds similar signals (b). The respiratory cluster is identified and the corresponding signals are combined and normalized to give the respiratory signal (c).

Fig. 3

Comparison of images with and without contrast agent. The contrast agent strongly attenuates the X-rays, which is visible as an additional peak at intensity 75 in the histogram Fig. 3d. In consequence, AEC increases the X-ray tube power, which shifts the rest of the histogram to the right.

Fig. 4

Simultaneously acquired biplane images from an X-ray system as in Fig. 1a should lead to identical respiratory signals.

Fig. 5

The reference respiratory signal created from diaphragm tracking (–) and the output of each method ( ) are shown. For this simple sequence without contrast agent injection, all methods deliver good results. The gray background indicates the learning phase. The vertical axis is normalized to 0–1. Best viewed in color.

Fig. 6

Respiratory signals extracted by all methods ( ) for an exemplary sequence with contrast agent injection. The gray background indicates the learning phase. In Fig. 6a, the contrast agent is injected in the application phase and in Fig. 6b in the learning phase. Diaphragm tracking is shown as a reference signal (–) in each plot. Best viewed in color.

Fig. 7

In Fig. 7a, signals from plane A ( ) and plane B ( ) are juxtaposed. In Fig. 7b, the signals are extracted from the full image ( ) and a region of interest that excludes the diaphragm ( ). The gray background indicates the learning phase. Best viewed in color.

Fig. 8

Respiratory signals extracted by the proposed methods ( ) on free-breathing humans (top 5 sequences) and manually ventilated pigs (bottom 8 sequences). The gray background indicates the learning phase. Diaphragm tracking is shown as a reference signal (–) in each plot. Best viewed in color.

Fig. 9

Average number of patches used in the respiratory cluster for each patch size ( ). For comparison, the total number of available patches in each image is given for each patch size ( ). Best viewed in color.

Fig. 10

Patches automatically selected by clustering (largest cluster , second largest cluster ). On the left, the complete images are processed. On the right, a ROI without the diaphragm is used. Almost the same patches are chosen to contain respiratory information.