Semi-supervised contour-driven broad learning system for autonomous segmentation of concealed prohibited baggage items

Abstract

With the exponential rise in global air traffic, ensuring swift passenger processing while countering potential security threats has become a paramount concern for aviation security. Although X-ray baggage monitoring is now standard, manual screening has several limitations, including the propensity for errors, and raises concerns about passenger privacy. To address these drawbacks, researchers have leveraged recent advances in deep learning to design threat-segmentation frameworks. However, these models require extensive training data and labour-intensive dense pixel-wise annotations and are finetuned separately for each dataset to account for inter-dataset discrepancies. Hence, this study proposes a semi-supervised contour-driven broad learning system (BLS) for X-ray baggage security threat instance segmentation referred to as C-BLX. The research methodology involved enhancing representation learning and achieving faster training capability to tackle severe occlusion and class imbalance using a single training routine with limited baggage scans. The proposed framework was trained with minimal supervision using resource-efficient image-level labels to localize illegal items in multi-vendor baggage scans. More specifically, the framework generated candidate region segments from the input X-ray scans based on local intensity transition cues, effectively identifying concealed prohibited items without entire baggage scans. The multi-convolutional BLS exploits the rich complementary features extracted from these region segments to predict object categories, including threat and benign classes. The contours corresponding to the region segments predicted as threats were then utilized to yield the segmentation results. The proposed C-BLX system was thoroughly evaluated on three highly imbalanced public datasets and surpassed other competitive approaches in baggage-threat segmentation, yielding 90.04%, 78.92%, and 59.44% in terms of mIoU on GDXray, SIXray, and Compass-XP, respectively. Furthermore, the limitations of the proposed system in extracting precise region segments in intricate noisy settings and potential strategies for overcoming them through post-processing techniques were explored (source code will be available at https://github.com/Divs1159/CNN_BLS .).

Keywords: Baggage X-ray imagery; Broad learning systems; Threat detection; Threat segmentation.

Fig. 1

Block diagram of the proposed C-BLX framework. The RCR block employs the polydirectional coherent tensor extraction approach recursively to extract the candidate segments containing items of varying density effectively from the input scan. These candidate region segments are passed to the CNN backbones, trained via cross-entropy loss, to extract features that are concatenated and eventually classified by the BLS. Detected threats are localized using bounding boxes of the region segments identified as prohibited items, and masks are created from the contours generated by RCR corresponding to these threat objects, ensuring precise segmentation

Fig. 2

Top row shows extracted region segments with isolated objects, while the bottom row shows segmented regions containing occluded, merged and cluttered objects

Fig. 3

Six unique tensor representations yielded for L = 3. Each row corresponds to a different scan, with the first column showing the original scan and the subsequent columns displaying tensor maps. These maps, arranged from left to right, represent ${\rho }_0^0,{\rho }_0^1,{\rho }_0^2,{\rho }_1^1,{\rho }_1^2,{\rho }_2^2$, obtained by computing image gradients along the angles 0, $\frac{2\pi }{3}$ , and $\frac{4\pi }{3}$

Fig. 4

Visualization of threat detection process: a Original scan; b Generated coherent tensor map highlighting object boundaries and local patterns; c Objects post non-maximum suppression; d, e Regions of interest after rectangle fitting; f, g Mask generation by filling inner pixels; and h Final threat segmentation overlaid on the original image

Algorithm 1 Proposed RCR approach

Fig. 5

BLS Parameter Optimization. a and c represent the performance variation with different numbers of enhancement nodes and feature nodes per window using SIXray and GDXray, respectively; b and d show the accuracy achieved with different numbers of windows of feature nodes for the SIXray and GDXray datasets, respectively

Fig. 6

Qualitative evaluations of the proposed framework on SIXray, GDXray and COMPASS-XP

Fig. 7

Illustration of failure cases