Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Nov 3;22(21):8459.
doi: 10.3390/s22218459.

Three-Stage Pavement Crack Localization and Segmentation Algorithm Based on Digital Image Processing and Deep Learning Techniques

Zhen Yang  1 , Changshuang Ni  1 , Lin Li  1   2 , Wenting Luo  2 , Yong Qin  3
Affiliations

Three-Stage Pavement Crack Localization and Segmentation Algorithm Based on Digital Image Processing and Deep Learning Techniques

Zhen Yang et al. Sensors (Basel). .

Abstract

The image of expressway asphalt pavement crack disease obtained by a three-dimensional line scan laser is easily affected by external factors such as uneven illumination distribution, environmental noise, occlusion shadow, and foreign bodies on the pavement. To locate and extract cracks accurately and efficiently, this article proposes a three-stage asphalt pavement crack location and segmentation method based on traditional digital image processing technology and deep learning methods. In the first stage of this method, the guided filtering and Retinex methods are used to preprocess the asphalt pavement crack image. The processed image removes redundant noise information and improves the brightness. At the information entropy level, it is 63% higher than the unpreprocessed image. In the second stage, the newly proposed YOLO-SAMT target detection model is used to locate the crack diseases in asphalt pavement. The model is 5.42 percentage points higher than the original YOLOv7 model on mAP@0.5, which enhances the recognition and location ability of crack diseases and reduces the calculation amount for the extraction of crack contour in the next stage. In the third stage, the improved k-means clustering algorithm is used to extract cracks. Compared with the traditional k-means clustering algorithm, this method improves the accuracy by 7.34 percentage points, the true rate by 6.57 percentage points, and the false positive rate by 18.32 percentage points to better extract the crack contour. To sum up, the method proposed in this article improves the quality of the pavement disease image, enhances the ability to identify and locate cracks, reduces the amount of calculation, improves the accuracy of crack contour extraction, and provides a new solution for highway crack inspection.

Keywords: Retinex; YOLOv7; asphalt pavement crack; attention mechanism; deep learning; digital image processing technology; guided filter.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
The network structure diagram of the proposed method.
Figure 2
Figure 2
The flowchart of the proposed method.
Figure 3
Figure 3
Flow chart of two-dimensional wavelet transform.
Figure 4
Figure 4
The results of various algorithms for crack disease image processing. (a) Original image, (b) SSR, (c) AutoMSRCR, (d) OpenCV, (e) MATLAB, (f) Gimp, (g) MSR, (h) MSRCP, (i) MSRCR, (j) method proposed in this paper.
Figure 4
Figure 4
The results of various algorithms for crack disease image processing. (a) Original image, (b) SSR, (c) AutoMSRCR, (d) OpenCV, (e) MATLAB, (f) Gimp, (g) MSR, (h) MSRCP, (i) MSRCR, (j) method proposed in this paper.
Figure 5
Figure 5
YOLOv7 network structure diagram.
Figure 6
Figure 6
Backbone structure of YOLOv7.
Figure 7
Figure 7
BConv structure layer diagram.
Figure 8
Figure 8
E-ELAN layer diagram.
Figure 9
Figure 9
MP layer diagram.
Figure 10
Figure 10
Head structure diagram of YOLOv7.
Figure 11
Figure 11
SPPCPC layer diagram.
Figure 12
Figure 12
Catconv layer diagram.
Figure 13
Figure 13
REP layer diagram.
Figure 14
Figure 14
Comparison of the implementation process of different attention mechanisms.
Figure 15
Figure 15
Detection results under different attention mechanisms. (a) Baseline. (b) Baseline + SE. (c) Baseline + CBAM. (d) Baseline + GC. (e) Baseline + ECA. (f) Baseline + SRM. (g) Baseline + SimAM.
Figure 16
Figure 16
Transformer encoder diagram.
Figure 17
Figure 17
Detection results under different networks. (a) Baseline. (b) Baseline + SimAM. (c) Baseline + SimAM + Transformer.
Figure 18
Figure 18
Detection results under different loss functions. (a) Baseline + SimAM + Transformer + GIoU. (b) Baseline + SimAM + Transformer + DIoU. (c) Baseline + SimAM + Transformer + CIoU. (d) Baseline + SimAM + Transformer + SIoU.
Figure 19
Figure 19
YOLO-SAMT network structure diagram.
Figure 20
Figure 20
Road multi-function detection vehicle.
Figure 21
Figure 21
Diagram of crack disease sample: (a) Transverse crack; (b) Longitudinal crack; (c) Map crack.
Figure 22
Figure 22
Comparative experiment of transverse crack segmentation. (a) original images, (b) image enhanced by guided filtering and Retinex method, (c) images generated by traditional k-means clustering algorithm, (d) ours.
Figure 23
Figure 23
Longitudinal crack segmentation contrast experiment. (a) original images, (b) image enhanced by guided filtering and Retinex method, (c) images generated by traditional k-means clustering algorithm, (d) ours.
Figure 24
Figure 24
Comparative experiment of map crack segmentation. (a) original images, (b) image enhanced by guided filtering and Retinex method, (c) images generated by traditional k-means clustering algorithm, (d) ours.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Cheng H.-D., Chen J.-R., Glazier C. Novel fuzzy logic approach to pavement distress detection. In: Chase S.B., editor. Nondestructive Evaluation of Bridges and Highways. SPIE; Bellingham, WA, USA: 1996.
    1. Zuo Y., Wang G., Zuo C. Wavelet Packet Denoising for Pavement Surface Cracks Detection; Proceedings of the 2008 International Conference on Computational Intelligence and Security; Suzhou, China. 13–17 December 2008.
    1. Bhutani K.R., Battou A. An Application of Fuzzy Relations to Image Enhancement. Pattern Recognit. Lett. 1995;16:901–909. doi: 10.1016/0167-8655(95)00035-F. - DOI
    1. Kirschke K.R., Velinsky S.A. Histogram-based Approach for Automated Pavement-crack Sensing. J. Transp. Eng. 1992;118:700–710. doi: 10.1061/(ASCE)0733-947X(1992)118:5(700). - DOI
    1. Oliveira H., Correia L. Automatic Road Crack Segmentation Using Entropy and Image Dynamic Thresholding; Proceedings of the European Signal Processing Conference; Glasgow, UK. 24–28 August 2009; pp. 2252–2259.

LinkOut - more resources