Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Feb 10;21(4):1265.
doi: 10.3390/s21041265.

Super Resolution Infrared Thermal Imaging Using Pansharpening Algorithms: Quantitative Assessment and Application to UAV Thermal Imaging

Javier Raimundo  1 , Serafin Lopez-Cuervo Medina  1 , Juan F Prieto  1 , Julian Aguirre de Mata  1
Affiliations

Super Resolution Infrared Thermal Imaging Using Pansharpening Algorithms: Quantitative Assessment and Application to UAV Thermal Imaging

Javier Raimundo et al. Sensors (Basel). .

Abstract

The lack of high-resolution thermal images is a limiting factor in the fusion with other sensors with a higher resolution. Different families of algorithms have been designed in the field of remote sensors to fuse panchromatic images with multispectral images from satellite platforms, in a process known as pansharpening. Attempts have been made to transfer these pansharpening algorithms to thermal images in the case of satellite sensors. Our work analyses the potential of these algorithms when applied to thermal images from unmanned aerial vehicles (UAVs). We present a comparison, by means of a quantitative procedure, of these pansharpening methods in satellite images when they are applied to fuse high-resolution images with thermal images obtained from UAVs, in order to be able to choose the method that offers the best quantitative results. This analysis, which allows the objective selection of which method to use with this type of images, has not been done until now. This algorithm selection is used here to fuse images from thermal sensors on UAVs with other images from different sensors for the documentation of heritage, but it has applications in many other fields.

Keywords: infrared; multispectral; pansharpening; remote sensing; resolution enhancement; super-resolution; thermal imaging.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure A1
Figure A1
UAV Illescas dataset RGB: (a) Reference image; (b) Low-resolution upsampled image; (c) PCA; (d) IHS; (e) BDSD; (f) GS; (g) PRACS; (h) HPF; (i) SFIM; (j) INDUSION; (k) MTF-GLP; (l) MTF-GLP-HPM; (m) MTF-GLP-HPM-PP; (n) MTF-GLP-ECB; (o) MTF-GLP-CBD.
Figure A1
Figure A1
UAV Illescas dataset RGB: (a) Reference image; (b) Low-resolution upsampled image; (c) PCA; (d) IHS; (e) BDSD; (f) GS; (g) PRACS; (h) HPF; (i) SFIM; (j) INDUSION; (k) MTF-GLP; (l) MTF-GLP-HPM; (m) MTF-GLP-HPM-PP; (n) MTF-GLP-ECB; (o) MTF-GLP-CBD.
Figure A2
Figure A2
UAV Illescas dataset grayscale: (a) Reference image; (b). PCA, (c) IHS; (d) BDSD; (e) GS; (f) PRACS; (g) HPF; (h) SFIM; (i) INDUSION; (j) MTF-GLP; (k) MTF-GLP-HPM; (l) MTF-GLP-HPM-PP; (m) MTF-GLP-ECB; (n) MTF-GLP-CBD.
Figure A2
Figure A2
UAV Illescas dataset grayscale: (a) Reference image; (b). PCA, (c) IHS; (d) BDSD; (e) GS; (f) PRACS; (g) HPF; (h) SFIM; (i) INDUSION; (j) MTF-GLP; (k) MTF-GLP-HPM; (l) MTF-GLP-HPM-PP; (m) MTF-GLP-ECB; (n) MTF-GLP-CBD.
Figure A3
Figure A3
FLIR ADAS dataset [47] grayscale: (a) Reference image; (b) Low-resolution upsampled image; (c) PCA; (d) IHS; (e) BDSD; (f) GS; (g) PRACS; (h) HPF; (i) SFIM; (j) INDUSION; (k) MTF-GLP; (l) MTF-GLP-HPM; (m) MTF-GLP-HPM-PP; (n) MTF-GLP-ECB; (o) MTF-GLP-CBD.
Figure A3
Figure A3
FLIR ADAS dataset [47] grayscale: (a) Reference image; (b) Low-resolution upsampled image; (c) PCA; (d) IHS; (e) BDSD; (f) GS; (g) PRACS; (h) HPF; (i) SFIM; (j) INDUSION; (k) MTF-GLP; (l) MTF-GLP-HPM; (m) MTF-GLP-HPM-PP; (n) MTF-GLP-ECB; (o) MTF-GLP-CBD.
Figure A4
Figure A4
FLIR ADAS dataset [47] RGB: (a) Reference image; (b) PCA, (c) IHS; (d) BDSD; (e) GS; (f) PRACS; (g) HPF; (h) SFIM; (i) INDUSION; (j) MTF-GLP; (k) MTF-GLP-HPM; (l) MTF-GLP-HPM-PP; (m) MTF-GLP-ECB; (n) MTF-GLP-CBD.
Figure A4
Figure A4
FLIR ADAS dataset [47] RGB: (a) Reference image; (b) PCA, (c) IHS; (d) BDSD; (e) GS; (f) PRACS; (g) HPF; (h) SFIM; (i) INDUSION; (j) MTF-GLP; (k) MTF-GLP-HPM; (l) MTF-GLP-HPM-PP; (m) MTF-GLP-ECB; (n) MTF-GLP-CBD.
Figure 1
Figure 1
Proposed workflow for the pansharpening assessment methodology of thermal and RGB images with pseudo-multispectral image composition, and the down- and upsampling resolution steps.
Figure 2
Figure 2
Graphic representation of computed quality indices. The different indices have been categorized in false color and grayscale. Dots represent quality index value, and vertical line length shows sample standard deviations. Graphics in ERGAS row are not directy comparable due to different y-axis scale.
See this image and copyright information in PMC

References

    1. Kohin M., Butler N.R. Performance limits of uncooled VO x microbolometer focal plane arrays; Proceedings of the Infrared Technology and Applications XXX; Orlando, FL, USA. 12–16 April 2004; p. 447. - DOI
    1. Yue L., Shen H., Li J., Yuan Q., Zhang H., Zhang L. Image super-resolution: The techniques, applications, and future. Signal Process. 2016;128:389–408. doi: 10.1016/j.sigpro.2016.05.002. - DOI
    1. Chavez P.S., Sides S.C., Anderson J.A. Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic. Photogramm. Eng. Remote Sens. 1991 doi: 10.1306/44b4c288-170a-11d7-8645000102c1865d. - DOI
    1. Lagüela S., Armesto J., Arias P., Herráez J. Automation of thermographic 3D modelling through image fusion and image matching techniques. Autom. Constr. 2012;27:24–31. doi: 10.1016/j.autcon.2012.05.011. - DOI
    1. Kuenzer C., Dech S. Thermal remote sensing Sensors, Methods, Applications. In: Kuenzer C., Dech S., editors. Remote Sensing and Digital Image Processing. Volume 17. Springer; Dordrecht, The Netherlands: 2013. pp. 287–313. - DOI

LinkOut - more resources