Intraoperative 3D reconstruction from sparse arbitrarily posed real X-rays

Abstract

Spine surgery is a high-risk intervention demanding precise execution, often supported by image-based navigation systems. Recently, supervised learning approaches have gained attention for reconstructing 3D spinal anatomy from sparse fluoroscopic data, significantly reducing reliance on radiation-intensive 3D imaging systems. However, these methods typically require large amounts of annotated training data and may struggle to generalize across varying patient anatomies or imaging conditions. Instance-learning approaches like Gaussian splatting could offer an alternative by avoiding extensive annotation requirements. While Gaussian splatting has shown promise for novel view synthesis, its application to sparse, arbitrarily posed real intraoperative X-rays has remained largely unexplored. This work addresses this limitation by extending the [Formula: see text]-Gaussian splatting framework to reconstruct anatomically consistent 3D volumes under these challenging conditions. We introduce an anatomy-guided radiographic standardization step using style transfer, improving visual consistency across views, and enhancing reconstruction quality. Notably, our framework requires no pretraining, making it inherently adaptable to new patients and anatomies. We evaluated our approach using an ex-vivo dataset. Expert surgical evaluation confirmed the clinical utility of the 3D reconstructions for navigation, especially when using 20-30 views, and highlighted the standardization's benefit for anatomical clarity. Benchmarking via quantitative 2D metrics (PSNR/SSIM) confirmed performance trade-offs compared to idealized settings, but also validated the improvement gained from standardization over raw inputs. This work demonstrates the feasibility of instance-based volumetric reconstruction from arbitrary sparse-view X-rays, advancing intraoperative 3D imaging for surgical navigation. Code and data to reproduce our results is made available at https://github.com/MrMonk3y/IXGS .

Keywords: Computer-assisted orthopedic surgery; Domain adaptation; Gaussian splatting; Intraoperative 3D reconstruction; Sparse-view X-ray; Surgical navigation.

Fig. 1

Comparison between conventional circular acquisition paths, typical for CBCT/CT imaging or idealized synthetic DRR generation (left), and the irregular, arbitrary acquisition poses representative of real intraoperative settings (right).

Fig. 2

Pipeline Overview. The training of Pix2Pix (blue) uses paired real X-rays () and synthetic DRRs () to learn style transfer. During inference (green), real X-rays with calibration information are converted to style-transferred images (). These images, along with their poses, are passed to the Gaussian splatting network, which outputs 3D reconstructions. The resulting volume can be visualized as slices or rendered from arbitrary viewpoints (orange).

Fig. 3

Comparison of 3D reconstructions of the lumbar spine (axial, coronal, and sagittal slices). Each block shows axial, coronal, and sagittal slices of the reconstructed volume, overlaid with alpha-blended masks of the segmented lumbar spine from the ground truth CT for better accuracy assessment. The columns compare reconstructions using 25 views (left) and 50 views (right). The top row shows reconstructions from baseline, while the bottom row displays reconstructions from (our approach).

Fig. 4

Comparison of slices from reconstructed volumes using different inputs and methods. From left to right: : Ground Truth CT volume, : Reconstruction from 50 synthetic DRRs generated from circular acquisition, : Reconstruction from 50 X-rays generated from arbitrary poses, (our approach): Reconstruction from 50 style-transferred X-rays.

Fig. 5

Comparison of 3D reconstructions of the lumbar spine. Each block shows AP, lateral, and isometric views of the reconstructed volume. The columns compare reconstructions using 25 views (left) and 50 views (right). The top row displays reconstructions from baseline, while the bottom row shows reconstructions from (our approach).

Fig. 6

Evaluation of reconstruction quality using Likert ratings from expert surgeons. a and b show ratings over 5–50 views, with a assessing 3D volumes and b assessing slice representations. See Sect. “Experiments and performance evaluation” for details on the different Likert scales used.

Fig. 7

Evaluation of novel view synthesis quality from unseen poses using PSNR and SSIM metrics over varying numbers of views. Higher scores indicate better reconstruction quality.