Head tracking using an optical soft tactile sensing surface

Abstract

This research proposes a sensor for tracking the motion of a human head via optical tactile sensing. It implements the use of a fibrescope a non-metal alternative to a webcam. Previous works have included robotics grippers to mimic the sensory features of human skin, that used monochrome cameras and depth cameras. Tactile sensing has shown advantages in feedback-based interactions between robots and their environment. The methodology in this paper is utilised to track motion of objects in physical contact with these sensors to replace external camera based motion capture systems. Our immediate application is related to detection of human head motion during radiotherapy procedures. The motion was analysed in two degrees of freedom, respective to the tactile sensor (translational in z-axis, and rotational around y-axis), to produce repeatable and accurate results. The movements were stimulated by a robot arm, which also provided ground truth values from its end-effector. The fibrescope was implemented to ensure the device's compatibility with electromagnetic waves. The cameras and the ground truth values were time synchronised using robotics operating systems tools. Image processing methods were compared between grayscale and binary image sequences, followed by motion tracking estimation using deterministic approaches. These included Lukas-Kanade Optical Flow and Simple Blob Detection, by OpenCV. The results showed that the grayscale image processing along with the Lukas-Kanade algorithm for motion tracking can produce better tracking abilities, although further exploration to improve the accuracy is still required.

Keywords: head and neck; motion tracking; optical flow; radiotherapy; tactile sensing.

FIGURE 1

Immobilisation methods used with Gamma Knife Radiotherapy. (A) Thermoplastic mask: This is moulded specifically for each patient based on their anatomy. It has an inlet for the nose. The ends of the mask are held securely to the base using nuts and bolts. The base has semi-firm cushioning which is also moulded around each patients’ anatomy. (B) Stereotactic frame: This is placed around the patient’s head and secured to the skull using surgically invasive pins. The frame is then secured to the bed of the radiotherapy equipment. This frame is specifically used for brain radiosurgeries using the Gamma Knife.

FIGURE 2

MCP anatomy and the rough placements for webcam and fibrescope, with their respective views. Only one camera was used at a time due to physical constraints.

FIGURE 3

PID controller to maintain the air pressure inside the pillow.

FIGURE 4

Experimental set-up of the Franka Emika Panda robot arm gripping the mannequin in a neutral position on the pillow. The frame, F _R , represents the frame for the robot arm, F _ee represents the end-effector frame, and F _P represents the frame for the MCP. The data was collected with respect to F _P . The translational values between the end-effector and the MCP, with respect to the MCP are T _x =75mm, and T _z = 200 mm.

FIGURE 5

Movement 1 showing the rotations around the y-axis.

FIGURE 6

Movement 2 showing the translational movement along the z-axis.

FIGURE 7

Flow-chart of the data collection methodology using ROS tools for time synchronising.

FIGURE 8

Demonstration of Lukas-Kanade tracking algorithm with respect to rotations in movement 1. The highlight dots represent the pins as corners via Shi-Tomasi corner detection. The displacement of these corners is tracked and displayed on the figures. The mean displacement is calculated to estimate the overall motion of the mannequin on the pillow. This is also available in a video format at: https://www.youtube.com/watch?v=Yk9Cr35gk4o.

FIGURE 9

Sample results from fibrescope with grayscale image processing for movement 1 and movement 2. Movement 1 is labelled Ry and movement 2 is labelled Tz. Prediction (pred) from Optical Flow and ground truth (gnd) values from the robot’s end-effector has been provided, assisted with expected positions of the mannequin at the extreme positions. The mannequin figures have been outlined with respective colours to their movements, where red outline shows movement 1 for rotation in y-axis and blue outline shows translation in z-axis.

FIGURE 10

Spearman’s correlation results from both movements: Movement 1 (Ry - rotation around y-axis) results from averaged Spearman’s Correlation for comparison of motion estimation with ground truth from the robot arm. Movement 2 (Tz - translation in z-axis) results from averaged Spearman’s Correlation for comparison of motion estimation with ground truth from the robot arm. Showing comparison in translational z-axis using average pixel brightness. Key: gray = grayscale, bin = binary, web = webcam, fib = fibrescope, BD = Blob Detection, LK = Lukas-Kanade.