An Angular Acceleration Based Looming Detector for Moving UAVs

Abstract

Visual perception equips unmanned aerial vehicles (UAVs) with increasingly comprehensive and instant environmental perception, rendering it a crucial technology in intelligent UAV obstacle avoidance. However, the rapid movements of UAVs cause significant changes in the field of view, affecting the algorithms' ability to extract the visual features of collisions accurately. As a result, algorithms suffer from a high rate of false alarms and a delay in warning time. During the study of visual field angle curves of different orders, it was found that the peak times of the curves of higher-order information on the angular size of looming objects are linearly related to the time to collision (TTC) and occur before collisions. This discovery implies that encoding higher-order information on the angular size could resolve the issue of response lag. Furthermore, the fact that the image of a looming object adjusts to meet several looming visual cues compared to the background interference implies that integrating various field-of-view characteristics will likely enhance the model's resistance to motion interference. Therefore, this paper presents a concise A-LGMD model for detecting looming objects. The model is based on image angular acceleration and addresses problems related to imprecise feature extraction and insufficient time series modeling to enhance the model's ability to rapidly and precisely detect looming objects during the rapid self-motion of UAVs. The model draws inspiration from the lobula giant movement detector (LGMD), which shows high sensitivity to acceleration information. In the proposed model, higher-order information on the angular size is abstracted by the network and fused with multiple visual field angle characteristics to promote the selective response to looming objects. Experiments carried out on synthetic and real-world datasets reveal that the model can efficiently detect the angular acceleration of an image, filter out insignificant background motion, and provide early warnings. These findings indicate that the model could have significant potential in embedded collision detection systems of micro or small UAVs.

Keywords: Bio-inspired Neural Networks; LGMD; UAV; collision detection; dynamic vision.

Figure A1

Object looming process. Black squares of different sizes approach at a constant speed and eventually collide with the observer. The object size and value close to the speed were considered to be constant, and thus $l_v$ = $l/v$ was constant.

Figure 1

Visual field angle curves of different orders based on Equations (A1)–(A3) (only plotting the segments where the curves are greater than 0). Here, $length/velocity(l/v)=0.5$ . The $d\theta ⁄dt$ and $\theta$ curves monotonically increase until a collision occurs. For second-order and higher visual field angle curves, there will be a peak, exhibiting a linear relationship with $l/v$ .

Figure 2

Model schematic. P: photo-receptor; DDC: distribution dual channel; USTC: ultra-spatiotemporal connection. The Output comprises two layers: the Soma layer and the Axon layer. The Soma layer integrates the looming information from the USTC layer, while the Izhikevich impulse neurons in the Axon layer generate the impulse information. This model extracts potential danger from image sequences and produces a sequence of pulses that warn of obstacles.

Figure 3

Examples of the low-speed and high-speed inhibitory kernels $K_{I_1}$ and $K_{I_2}$ , respectively. The left image shows the kernel grid of $K_{I_1}$ , while the right image displays the kernel grid of $K_{I_2}$ , where ${\sigma }_1=r1=1$ and ${\sigma }_2=r2=2$ .

Figure 4

$A^{\prime }$ position from the previous time step is denoted. Pixels A, B, and C move at speeds of 1, 2, and 3 pixels/ms, respectively. The inhibition1 and inhibition2 radii indicate the inhibition ranges of inhibition pathways 1 and 2. The inhibition values generated by $A^{\prime }$ , $B^{\prime }$ , and $C^{\prime }$ are represented by red, green, and blue colors, respectively. When the movement of the pixel generates a level of stimulation beyond its inhibition threshold, the DDC layer will receive speed data above the threshold (resulting in a purple output).

Figure 5

Schematic for collecting acceleration data within the USTC layer. The USTC layer generates acceleration information by aggregating data on high-speed moving pixels (blue pixels) near the pre-accelerated pixels (red pixels).

Figure 6

Soma layer input to the Izhikevich model produces an impulse sequence.

Figure 7

The capabilities of the A-LGMD model for extracting the angular speed and acceleration are demonstrated. The number of hazardous pixels in the velocity and acceleration layers is represented on the left y-axis, while the right y-axis displays the ideal magnitude of the angular velocity for a cube looming at a ratio of 0.1 ( $l/v=0.1$ ) per frame. During the 64th and 67th frames, the ideal angular velocities exceeded 60 and 120 pixels/s, respectively. Technical abbreviations are explained upon the first usage. The position changes of the cube in the field of view, and the variation in acceleration information extracted by the USTC layer of the model are illustrated in the diagram located in the lower left corner. The small diagram in the upper right corner offers a local magnification of the changing trend in the number of hazardous pixels in the velocity and acceleration layers of the A-LGMD neural network between frames 64 and 70.

Figure 8

Indoor looming detection experimental data were recorded using cameras, and the performance of various looming detection models was evaluated. Grayscale images of Group 1 and Group 2 are shown in (a–d), displaying the output images of the A-LGMD model for Group 1 and Group 2 with corresponding sample images provided in Table 2. The normalized output curves of different models in the Group 1 and Group 2 datasets are presented in (e–g), showing the sensitivity of A-LGMD and D-LGMD to the looming region of the differential input image.

Figure 9

Experiments were conducted using real drone flights involving collisions, focusing on an objective perspective. Grayscale images from Groups 3, 4, and 5 are displayed in (a–c), respectively. In Group 3, a stationary drone collided with an off-road vehicle at high speed. In Group 4, the drone rapidly approached a basketball stand. In Group 5, a quadcopter collided with a uniformly moving drone. (d–f) The output images of the A-LGMD model in Groups 3, 4, and 5, respectively, with corresponding sample images listed in Table 2. (g–i) The normalized output curves of the A-LGMD model and other models in Group 3, 4, and 5 datasets. (j) The rate of false detections by A-LGMD and D-LGMD in the background section of the differential input image.

Figure 10

Alarm time to collision (ATTC) and false alarm rate (FAR) performance of different biomimetic looming detection models based on the field of view theory under different background interference (BI). The LGMD algorithm had a high false alarm rate when the UAV was moving. The D-LGMD algorithm was only sensitive to fast-moving objects and performed well in resolving the background interference caused by its motion, but this may result in a lower ATTC, which may not give the UAV enough time to avoid obstacles. In contrast, the A-LGMD algorithm had a high ATTC and a low false alarm rate, making it ideal for UAV looming detection.