A Review of Vision-Based Multi-Task Perception Research Methods for Autonomous Vehicles

Abstract

Multi-task perception technology for autonomous driving significantly improves the ability of autonomous vehicles to understand complex traffic environments by integrating multiple perception tasks, such as traffic object detection, drivable area segmentation, and lane detection. The collaborative processing of these tasks not only improves the overall performance of the perception system but also enhances the robustness and real-time performance of the system. In this paper, we review the research progress in the field of vision-based multi-task perception for autonomous driving and introduce the methods of traffic object detection, drivable area segmentation, and lane detection in detail. Moreover, we discuss the definition, role, and classification of multi-task learning. In addition, we analyze the design of classical network architectures and loss functions for multi-task perception, introduce commonly used datasets and evaluation metrics, and discuss the current challenges and development prospects of multi-task perception. By analyzing these contents, this paper aims to provide a comprehensive reference framework for researchers in the field of autonomous driving and encourage more research work on multi-task perception for autonomous driving.

Keywords: autonomous driving; deep learning; detection; drivable area segmentation; multi-task learning.

Figure 1

Common algorithms for object detection.

Figure 2

Common algorithms for semantic segmentation.

Figure 3

Classification of multi-task learning sharing mechanisms: (a) hard parameter sharing; (b) soft parameter sharing; and (c) hybrid sharing.

Figure 4

A partial image of the BDD100K dataset: (a) daytime; (b) night; (c) rain; (d) snow; (e) city road; (f) motorway; (g) sunny; and (h) overcast.

Figure 5

Schematic of IoU definition.