Proof-of-concept of a robotic-driven photogrammetric scanner for intra-operative knee cartilage repair

Abstract

This work presents a proof-of-concept of a robotic-driven intra-operative scanner designed for knee cartilage lesion repair, part of a system for direct in vivo bioprinting. The proposed system is based on a photogrammetric pipeline, which reconstructs the cartilage and lesion surfaces from sets of photographs acquired by a robotic-handled endoscope, and produces 3D grafts for further printing path planning. A validation on a synthetic phantom is presented, showing that, despite the cartilage smooth and featureless surface, the current prototype can accurately reconstruct osteochondral lesions and their surroundings with mean error values of 0.199 ± 0.096 mm but with noticeable concentration on areas with poor lighting or low photographic coverage. The system can also accurately generate grafts for bioprinting, although with a slight tendency to underestimate the actual lesion sizes, producing grafts with coverage errors of -12.2 ± 3.7, -7.9 ± 4.9, and -15.2 ± 3.4% for the medio-lateral, antero-posterior, and craneo-caudal directions, respectively. Improvements in lighting and acquisition for enhancing reconstruction accuracy are planned as future work, as well as integration into a complete bioprinting pipeline and validation with ex vivo phantoms.

Keywords: biological tissues; biomedical imaging; surgery.

FIGURE 1

Left: Photograph of the proposed setup, including the robot‐mounted endoscope, phantom, and miniature projector. Right: schematic of the photogrammetric pipeline.

FIGURE 2

Graphical setup of the hand‐eye calibration system, where X is the homogeneous transform matrix obtained as the result of the calibration; A and B are the homogeneous transform matrices given by the optical tracker; Y is a constant homogeneous transform matrix representing the pose of the tracker's markers with respect to the calibration board; and C is the homogeneous transform matrix of the board with respect to the endoscope obtained using findChessboardCorners() function of OpenCV library [10].

FIGURE 3

Graphical description of the graft generation: A rectangular patch is warped over the reconstructed lesion, fitting the outer margins but avoiding the centre (left). Then pairs of points are found tracing lines from the patch's points p_k along their normal vectors n_k , finding the corresponding points q_k over the lesion's surface (centre). Finally, a point cloud is produced joining the sets of p_k and q_k points, which is then meshed producing the graft's final surface (right).

FIGURE 4

From left to right: photograph of the synthetic phantom with the milled lesion, acquired photograph of the same phantom illuminated with the projected textured pattern, reference high‐resolution scan (yellow) and reconstruction obtained with the proposed method (blue).

FIGURE 5

Visualisation of the surface reconstruction errors for all scans of the simulated lesion. Colour scale is given in the [0.0, 1.0] mm range. Analysis of the data reveals that reconstruction errors are focalised on the lesion's posterior (+Y) face.

FIGURE 6

Sample point clouds generated by the SfM pipeline, with (left) and without (right) illumination with the marker texture. Note that the point cloud using markers has a clear and large coverage of the cartilage surface, whereas the other has so few triangulated points that become useless for mesh generation. SfM, structure from motion.

FIGURE 7

Sample reconstruction error: A reconstructed lesion model (opaque) is overlayed with the reference model (semi‐transparent) showing a gap between them (marked with arrows). These reconstruction errors tend to appear on the lesions’ bottoms, as these zones are more difficult to photograph during acquisitions.