Automatic 3D Augmented-Reality Robot-Assisted Partial Nephrectomy Using Machine Learning: Our Pioneer Experience

Abstract

The aim of "Precision Surgery" is to reduce the impact of surgeries on patients' global health. In this context, over the last years, the use of three-dimensional virtual models (3DVMs) of organs has allowed for intraoperative guidance, showing hidden anatomical targets, thus limiting healthy-tissue dissections and subsequent damage during an operation. In order to provide an automatic 3DVM overlapping in the surgical field, we developed and tested a new software, called "ikidney", based on convolutional neural networks (CNNs). From January 2022 to April 2023, patients affected by organ-confined renal masses amenable to RAPN were enrolled. A bioengineer, a software developer, and a surgeon collaborated to create hyper-accurate 3D models for automatic 3D AR-guided RAPN, using CNNs. For each patient, demographic and clinical data were collected. A total of 13 patients were included in the present study. The average anchoring time was 11 (6-13) s. Unintended 3D-model automatic co-registration temporary failures happened in a static setting in one patient, while this happened in one patient in a dynamic setting. There was one failure; in this single case, an ultrasound drop-in probe was used to detect the neoplasm, and the surgery was performed under ultrasound guidance instead of AR guidance. No major intraoperative nor postoperative complications (i.e., Clavien Dindo > 2) were recorded. The employment of AI has unveiled several new scenarios in clinical practice, thanks to its ability to perform specific tasks autonomously. We employed CNNs for an automatic 3DVM overlapping during RAPN, thus improving the accuracy of the superimposition process.

Keywords: artificial intelligence; kidney cancer; nephron-sparing surgery; partial nephrectomy; renal cell carcinoma; robotic surgery; three-dimensional imaging.

Figure 1

Intraoperative automatic augmented-reality co-registration process, using the software ikidney through CNN technology. (A) Right kidney, as it appears after renal-lodge dissection; the renal artery was tagged with a red vessel loop and the renal vein with a blue vessel loop; the tumor is partially esophytic, and it is located at the middle portion of the kidney. (B) Automatic overlapping of the virtual kidney over the real one, leveraging CNN technology; upper urinary tract is in yellow, arterial vessels in green, and venous vessels in blue; the tumor is in brown.