Surgical Continuum Manipulator Control Using Multiagent Team Deep Q Learning

Abstract

Continuum manipulator has shown great potential in surgical applications. The flexibility of the continuum manipulator helps it achieve many complicated surgeries, such as neurosurgery, vascular surgery, abdominal surgery, etc. In this paper, we propose a Team Deep Q learning framework (TDQN) to control a 2-DoF surgical continuum manipulator with four cables, where two cables in a pair form one agent. During the learning process, each agent shares state and reward information with the other one, which namely is centralized learning. Using the shared information, TDQN shows better targeting accuracy than multiagent deep Q learning (MADQN) by verifying on a 2-DoF cable-driven surgical continuum manipulator. The root mean square error during tracking with and without disturbance are 0.82mm and 0.16mm respectively using TDQN, whereas 1.52mm and 0.98mm using MADQN respectively.Clinical Relevance-The proposed TDQN shows a promising future in improving control accuracy under disturbance and maneuverability in robotic-assisted endoscopic surgery.