Intelligent Spacecraft Visual GNC Architecture With the State-Of-the-Art AI Components for On-Orbit Manipulation

Abstract

Conventional spacecraft Guidance, Navigation, and Control (GNC) architectures have been designed to receive and execute commands from ground control with minimal automation and autonomy onboard spacecraft. In contrast, Artificial Intelligence (AI)-based systems can allow real-time decision-making by considering system information that is difficult to model and incorporate in the conventional decision-making process involving ground control or human operators. With growing interests in on-orbit services with manipulation, the conventional GNC faces numerous challenges in adapting to a wide range of possible scenarios, such as removing unknown debris, potentially addressed using emerging AI-enabled robotic technologies. However, a complete paradigm shift may need years' efforts. As an intermediate solution, we introduce a novel visual GNC system with two state-of-the-art AI modules to replace the corresponding functions in the conventional GNC system for on-orbit manipulation. The AI components are as follows: (i) A Deep Learning (DL)-based pose estimation algorithm that can estimate a target's pose from two-dimensional images using a pre-trained neural network without requiring any prior information on the dynamics or state of the target. (ii) A technique for modeling and controlling space robot manipulator trajectories using probabilistic modeling and reproduction to previously unseen situations to avoid complex trajectory optimizations on board. This also minimizes the attitude disturbances of spacecraft induced on it due to the motion of the robot arm. This architecture uses a centralized camera network as the main sensor, and the trajectory learning module of the 7 degrees of freedom robotic arm is integrated into the GNC system. The intelligent visual GNC system is demonstrated by simulation of a conceptual mission-AISAT. The mission is a micro-satellite to carry out on-orbit manipulation around a non-cooperative CubeSat. The simulation shows how the GNC system works in discrete-time simulation with the control and trajectory planning are generated in Matlab/Simulink. The physics rendering engine, Eevee, renders the whole simulation to provide a graphic realism for the DL pose estimation. In the end, the testbeds developed to evaluate and demonstrate the GNC system are also introduced. The novel intelligent GNC system can be a stepping stone toward future fully autonomous orbital robot systems.

Keywords: Guidance Navigation and Control; artificial intelligent; on-orbit service; pose estimation; space manipulator; space robotics.

Figure 1

The design of the intelligent orbital service spacecraft—AISAT: it comprises a monocular camera network and a 7-DOF (degrees of freedom) robotic arm as key components for the intelligent visual Guidance, Navigation, and Control (GNC) subsystem. The spacecraft also has a dexterous gripper for target capture and S-band patch antenna for basic house-keeping and the intelligent system updating.

Figure 2

The conceptual mission for capturing and servicing a non-cooperative but known 16U CubeSat.

Figure 3

The decision-making flow chart for the intelligent Guidance, Navigation, and Control (GNC) system. The achievable tasks are based on the hardware constraints and the task is selected based on the target's identify and features.

Figure 4

The transition from (A) conventional spacecraft Guidance, Navigation, and Control (GNC) subsystem with robotic arm to (B) the Artificial Intelligence (AI) GNC subsystem. The AI GNC system uses centralized sensor information with primarily 2D images as inputs to the AI computer with pre-trained neural networks to control the spacecraft and robotic arm at the same time.

Figure 5

Sample photo-realistic visuals of CubeSat model in orbit generated using the OrViS simulator.

Figure 6

Framework for spacecraft pose estimation using keypoint-based deep learning approach.

Figure 7

The STAR-LAB's testbeds for orbital space robotic Guidance, Navigation, and Control (GNC) systems (A) is the digital simulation testbed build in ROS gazebo environment; (B) the design and setup configuration of the STAR-LAB orbital robotic testbed in monochrome format is shown.

Figure 8

The simulation of the intelligent orbital Guidance, Navigation, and Control (GNC) operations: (A) the starting position of the simulation, (B) target identification and continuously pose estimation of the target within the camera array range, (C) final approaching with adjusted orbital and attitude control to an idea manipulation-ready state, (D) initiate the planned optimized trajectory to dock the end-effector to the pre-grasping point, (E) the end-effector arrives the dedicated located and executing grasping and rigidization, (F) plan and move the target CubeSat to for servicing. The animation is rendered in real-time in the Eevee engine with the actual relative position of the service spacecraft and the target, robotic arm trajectory points with kinematics, and time-synchronized camera readings in the render engine.

Figure 9

Results from the trained model (both object detection and keypoint estimation) along with computed pose from PnP.

Figure 10

The two aforementioned Artificial Intelligence (AI) packages are connected through a decision-making block, which can use a conventional method as an intermediate solution.

Figure 11

The attitude requirement for proposed free-floating spacecraft is to minimize the angular drift within the half-power beamwidth (HPBW) during manipulation period to maintain stable communication.

Figure 12

Time series of the free-floating spacecraft reaching the CubeSat position with the trajectory sampled from learned probabilistic distribution. This trajectory has minimized impact to the attitude of the free-floating spacecraft.

Figure 13

Motion trajectory sampled from learned probabilistic distribution to minimize the rotation of the free-floating spacecraft.

Figure 14

The normalized attitude drift (in rads) of the free-floating spacecraft relative to the Nadir during the manipulator operation in the simulation. The attitude drift is within the S-band patch antenna half-power beamwidth (HPBW) for stable ground communication.