Computer vision-guided rapid and precise automated cranial microsurgeries in mice

Abstract

A common procedure that allows interfacing with the brain is cranial microsurgery, wherein small to large craniotomies are performed on the overlying skull for insertion of neural interfaces or implantation of optically clear windows for long-term cranial observation. Performing craniotomies requires skill, time, and precision to avoid damaging the brain and dura. Here, we present a computer vision-guided craniotomy robot (CV-Craniobot) that uses machine learning to accurately estimate the dorsal skull anatomy from optical coherence tomography images. Instantaneous information of skull morphology is used by a robotic mill to rapidly and precisely remove the skull from a desired craniotomy location. We show that the CV-Craniobot can perform small (2- to 4-millimeter diameter) craniotomies with near 100% success rates within 2 minutes and large craniotomies encompassing most of the dorsal cortex in less than 10 minutes. Thus, the CV-Craniobot enables rapid and precise craniotomies, reducing surgery time compared to human practitioners and eliminating the need for long training.

Fig. 1.. CV-Craniobot working principle and hardware architecture.

(A) Workflow: The mouse skull is imaged using an OCT scanner; an machine learning (ML) model estimates the dorsal and ventral skull surfaces from the B-scans of the OCT image; a robotic mill executes automated bone removal along the computed craniotomy path. (B) Computer-aided design rendering of the CV-Craniobot hardware. (C) Photograph of the CV-Craniobot with core elements highlighted.

Fig. 2.. Rapid and accurate estimation of skull morphology from OCT image scan.

(A) Steps involved in estimating skull morphology from OCT image scan (B) Reconstructed top view image of a modeling clay sample imprinted with mesh pattern before and after non-telecentric distortion correction. (C) Root mean square error (RMSE) between each Y-line and the reference line at that position for both raw and corrected scans. (D) RMSE between the raw and corrected X-lines. (E) U-Net model training [(E), i]. Representative average B-scan image from OCT scan of a mouse skull. [(E), ii] B-scan image with manually annotated dorsal and ventral skull surfaces. [(E), iii] Trained U-Net model estimate of the dorsal and ventral surfaces. (F) Tversky loss plot for training and validation. (G) Measured refractive index (n = 8 mice) (see Supplementary Text S1). (H) Top: Reconstructed top view of the mouse’s dorsal skull from OCT scan. Bottom: B-scan cross section with estimated dorsal and ventral surfaces by U-Net model and the refractive index-corrected ventral surface. (I) A pseudo-color map of the skull thickness of the specimen is shown in (H). (J) Comparison between the micro–computed tomography (μCT) scan measurement (left) and the estimated thickness using OCT scans in the same area.

Fig. 3.. Robotic milling using the estimated dorsal surface as reference.

(A) Photograph of the surface of an eggshell engraved with the University of Minnesota logo using the CV-Craniobot. Insets 1 and 2 show close-up images of the engravings milled to depths of milling operations with depths of 42 and 168 μm, respectively. (B) Surface profiles of the bottom left corner logo were obtained via μCT scanning of the eggshell surface after engraving the logo on it. Inset: Reconstructed μCT scan top view of the 6-pixel (84 μm)–deep logo engraving. The red dashed line indicates the cross section of the surface shown in the plot. (C) Pseudo-color plots of the engraving depth. (D) Milled trench depth along the milling path for different target milling depths, as measured using the OCT scanner. (E) Milled trench depths along the vertical paths for different target milling depths. Inset: Top-view image of the OCT scanner’s FOV after milling. The red circle indicates bregma. The dashed lines show the cross section displayed in (G). (F) Milling depths along the horizontal paths for different target milling depths. Inset: Top-view image of the OCT scanner’s FOV after milling. The red circle indicates bregma. The dashed lines show the cross section displayed in (H). (G) Coronal section of the mouse skull along the dashed line indicated in (E). (H) Coronal cross section of the skull along the dashed line indicated in (F). A, anterior; P, posterior; M, medial; L, lateral; D, dorsal; V, ventral.

Fig. 4.. Robotic milling of mouse skull using estimated ventral surface as reference.

(A) Reconstructed top view of OCT scan after milling circular trench on a mouse skull. (B) Cross-sectional view of the milled skull along the dashed line shown in (A). (C) Pseudo-color plot of measured skull thickness after milling of circular trench in (A). (D) Reconstructed top view of μCT scan of the same mouse skull shown in (A). (E) Cross-sectional view of the milled skull as shown in (D). (F) Violin plots of measured thickness of the skull along the milled trench path with target remaining thickness of 70 μm (left plot) and 98 μm (right plot), n = 4 mice. Red lines indicate targeted milling depth. (G) Reconstructed top view of OCT scan after milling over a circular area to thin the tissue down to 98 μm. (H) Pseudo-color plot of the skull thickness measured via OCT scanning in the milled skull area shown in (G). (I) Reconstructed top view of μCT scan of the same mouse shown in (G). (J) Cross-sectional view of the milled skull as shown in (G). (K) Pseudo-color plot of the skull thickness measured via μCT scanning in the milled skull area shown in (G). (L) Violin plots of the measured thickness of the skull in the milled circular area with target remaining thickness of 70 μm (left plot) and 98 μm (right plot), n = 4 mice. Red lines indicate targeted milling depth. A, anterior; P, posterior; M, medial; L, lateral; D, dorsal; V, ventral. Red circles in [(A) and (D)] indicate bregma.

Fig. 5.. Automated craniotomies using CV-Craniobot.

(A) Left: Top view of reconstructed OCT image of the surgery area, with path of circular craniotomy highlighted. Right: Photograph of the same location shown on the left taken after automated removal of the bone island. (B) The success rate of complete bone removal at different thicknesses of bone remaining on top of the ventral surface by the CV-Craniobot. (C) Top view of computationally stitched reconstructed images from four OCT scans. The image from each scan is coded with a different color. (D) Sagittal cross section of the stitched scans along the vertical dashed line shown in (C). (E) Coronal cross section of the stitched scans along the horizontal dashed line shown in (C). (F) Alignment error during stitching between each two neighboring scans. (G) An example image of a mouse implanted with a multi-planar faceted cranial window covering the whole dorsal cortex was taken 21 days after robotic surgery using the CV-Craniobot. (H) Representative images of brain slices stained for Iba-1 and 4′,6-diamidino-2-phenylindole (DAPI) of the brain directly underneath the craniotomy and control samples. Scale bar, 200 μm. (I) Average microglia cell counts in a 700 μm–by–900 μm area under the milled skull area for the test samples and similar areas for the control samples. n.s., not significant. (J) Average time to complete craniotomy using the CV-Craniobot and comparison with human surgeons. Small craniotomies (SC): n = 5 procedures using the robot and n = 5 manual procedures performed by two experimenters. Large craniotomies (LC): n = 5 procedures using CV-Craniobot and n = 7 manual procedures performed by three experimenters.