Development and implementation of a web-enabled 3D consultation tool for breast augmentation surgery based on 3D-image reconstruction of 2D pictures

Abstract

Background: Producing a rich, personalized Web-based consultation tool for plastic surgeons and patients is challenging.

Objective: (1) To develop a computer tool that allows individual reconstruction and simulation of 3-dimensional (3D) soft tissue from ordinary digital photos of breasts, (2) to implement a Web-based, worldwide-accessible preoperative surgical planning platform for plastic surgeons, and (3) to validate this tool through a quality control analysis by comparing 3D laser scans of the patients with the 3D reconstructions with this tool from original 2-dimensional (2D) pictures of the same patients.

Methods: The proposed system uses well-established 2D digital photos for reconstruction into a 3D torso, which is then available to the user for interactive planning. The simulation is performed on dedicated servers, accessible via Internet. It allows the surgeon, together with the patient, to previsualize the impact of the proposed breast augmentation directly during the consultation before a surgery is decided upon. We retrospectively conduced a quality control assessment of available anonymized pre- and postoperative 2D digital photographs of patients undergoing breast augmentation procedures. The method presented above was used to reconstruct 3D pictures from 2D digital pictures. We used a laser scanner capable of generating a highly accurate surface model of the patient's anatomy to acquire ground truth data. The quality of the computed 3D reconstructions was compared with the ground truth data used to perform both qualitative and quantitative evaluations.

Results: We evaluated the system on 11 clinical cases for surface reconstructions and 4 clinical cases of postoperative simulations, using laser surface scan technologies showing a mean reconstruction error between 2 and 4 mm and a maximum outlier error of 16 mm. Qualitative and quantitative analyses from plastic surgeons demonstrate the potential of these new emerging technologies.

Conclusions: We tested our tool for 3D, Web-based, patient-specific consultation in the clinical scenario of breast augmentation. This example shows that the current state of development allows for creation of responsive and effective Web-based, 3D medical tools, even with highly complex and time-consuming computation, by off-loading them to a dedicated high-performance data center. The efficient combination of advanced technologies, based on analysis and understanding of human anatomy and physiology, will allow the development of further Web-based reconstruction and predictive interfaces at different scales of the human body. The consultation tool presented herein exemplifies the potential of combining advancements in the core areas of computer science and biomedical engineering with the evolving areas of Web technologies. We are confident that future developments based on a multidisciplinary approach will further pave the way toward personalized Web-enabled medicine.

Figure 1

General overview of the developed system. An Internet-based solution combining advanced technologies enables a realistic, patient-specific, simulated clinical scenario. 2D = 2-dimensional; 3D = 3-dimensional.

Figure 2

Three landmarked photos of a patient. Visual aids on where to place landmarks and a simple Web interface guide the user through the annotation of images. Cropped screenshot taken from the Web-based interface.

Figure 3

Web-based 3-dimensional (3D) annotations on a reconstructed patient model. A set of tools including 3D distances, text, and body drawing enable a personalized virtual clinical analysis. Cropped screenshot taken from the Web-based interface.

Figure 4

Selection of implant position and diameter. Cropped screenshot taken from the Web-based interface.

Figure 5

Voxelized breasts and implants. The fat layer is seen in yellow, the skin in orange, and the muscle layer in red. The implants are shown as white voxels or particles. This screenshot is taken from the simulation developer’s point of view and is not visible in the Web-based interface.

Figure 6

Overlapping scanner views (left) and the resulting surface scan generated by the commercial scanner software (right).

Figure 7

Composite figure sets showing, from left to right: patient photographs, corresponding 3-dimensional (3D) surface reconstruction, laser scan ground truth, and overlaid reconstructed surfaces. The laser scan textures were acquired in the absence of flash photography; hence, their illumination appears slightly different from that in the patient photos. The laser scan surface is shown as a transparent surface in the last column, and the 3D reconstruction is displayed as a superimposed red wire frame. The preoperative simulated images are screenshots from the Web-based interface, as seen in the Proposed System column.

Figure 8

Box plots of left and right breast: 3-dimensional (3D) surface reconstructions compared with laser scans. The patients in Figure 7 correspond to cases 1, 7, and 8, respectively, in these box plots.

Figure 9

Composite figure showing pre- and postoperative photos, pre- and postoperative 3-dimensional (3D) reconstruction, and implant simulation surface renderings from the simulation visualization. The reconstructed and simulated surfaces were computed from the preoperative photos and hence show similarities in the textures. The pre- and postoperative simulated images are screenshots from the Web-based interface (middle columns).

Figure 10

Postoperative simulation results predicted from preoperative images compared with postoperative laser scans.