Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Nov 27;21(23):7906.
doi: 10.3390/s21237906.

Extending Camera's Capabilities in Low Light Conditions Based on LIP Enhancement Coupled with CNN Denoising

Maxime Carré  1 , Michel Jourlin  2
Affiliations

Extending Camera's Capabilities in Low Light Conditions Based on LIP Enhancement Coupled with CNN Denoising

Maxime Carré et al. Sensors (Basel). .

Abstract

Using a sensor in variable lighting conditions, especially very low-light conditions, requires the application of image enhancement followed by denoising to retrieve correct information. The limits of such a process are explored in the present paper, with the objective of preserving the quality of enhanced images. The LIP (Logarithmic Image Processing) framework was initially created to process images acquired in transmission. The compatibility of this framework with the human visual system makes possible its application to images acquired in reflection. Previous works have established the ability of the LIP laws to perform a precise simulation of exposure time variation. Such a simulation permits the enhancement of low-light images, but a denoising step is required, realized by using a CNN (Convolutional Neural Network). A main contribution of the paper consists of using rigorous tools (metrics) to estimate the enhancement reliability in terms of noise reduction, visual image quality, and color preservation. Thanks to these tools, it has been established that the standard exposure time can be significantly reduced, which considerably enlarges the use of a given sensor. Moreover, the contribution of the LIP enhancement and denoising step are evaluated separately.

Keywords: CNN; Logarithmic Image Processing; denoising; image enhancement; image quality; low-light; metrics.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Brightening/darkening an image thanks to LIP laws. (a) Original image f; (b) Image 0.75 ⨻ f; (c) Image 1.25 ⨻ f; (d) Image f ⨺ 100; (e) Image f ⨹ 100.
Figure 2
Figure 2
Ability of LIP subtraction of a constant to simulate exposure time changing. (a) Image f, “Laboratory” acquired with exposure time = 10 ms; (b) Image g, “Laboratory” acquired with exposure time = 100 ms; (c) Starting from f, simulation of an exposure time of 100 ms.
Figure 3
Figure 3
LIP tone mapping algorithm using LIP subtractions to enhance a color low-light image. (a) Original image; (b) LIP Tone Mapping applied on (a).
Figure 4
Figure 4
Color chart acquired at various exposure times. (a) image f3 at 3 ms; (b) image f6 at 6 ms; (c) image f18 at 18 ms; (d) image f24 at 24 ms; (e) image f33 at 33 ms; (f) image f45 at 45 ms.
Figure 5
Figure 5
Enhanced images associated to f3, f6, f18, f24, f33, f45, from left to right.
Figure 6
Figure 6
Proof of the linear relation between the exposure time and the constant grey level subtracted from the acquired image to get the enhanced one.
Figure 7
Figure 7
Grey level image enhancement. (a) image acquired at low exposure; (b) LIP enhancement of (a); (c) CNN denoising applied on (b); (d) zoom on (a); (e) zoom on (b); (f) zoom on (c).
Figure 8
Figure 8
Color image enhancement. (a) image acquired at low exposure; (b) LIP enhancement of (a); (c) CNN denoising applied on (b); (d) zoom on (a); (e) zoom on (b); (f) zoom on (c).
Figure 9
Figure 9
Acquisitions from 1.11 ms to 50 ms. Top: Original images; Middle; LIP stabilization; Bottom: LIP + CNN denoising.
Figure 10
Figure 10
Evolution of PSNR, SSIM, and Delta E according to exposure time for each processing step (raw image, LIP enhancement, LIP + CNN denoising. For PSNR, LIP + denoised value is not represented for the clean target (50 ms) because it reaches infinity.
Figure 11
Figure 11
Example of motion acquisition with a standard exposure time (here 50 ms). (a) Acquisition of a motionless car; (b) Acquisition of a moving car.
Figure 12
Figure 12
Short exposure acquisition (2 ms) on a moving object and application of LIP enhancement + denoising. (a) Motion acquisition with a short exposure time; (b) Noisy image after LIP enhancement; (c) Image after enhancement and denoising.
Figure 13
Figure 13
Zoom on Figure 11 and Figure 12. (a) Acquisition of a motionless car at 50 ms; (b) Acquisition of a moving car at 50 ms; (c) Motion acquisition with a short exposure time (2 ms); (d) Noisy image after LIP enhancement; (e) Image after enhancement and denoising.
See this image and copyright information in PMC

References

    1. Gorgel P., Sertbas A., Ucan O.N. A wavelet-based mammographic image denoising and enhancement with homomorphic filtering. J. Med. Syst. 2010;34:993–1002. doi: 10.1007/s10916-009-9316-3. - DOI - PubMed
    1. Gassenmaier S., Afat S., Nickel D., Kannengiesser S., Herrmann J., Hoffmann R., Othman A. Application of a novel iterative denoising and image enhancement technique in T1-weighted precontrast and postcontrast gradient echo imaging of the abdomen. Improvement of image quality and diagnostic confidence. Investig. Radiol. 2021;56:328–334. doi: 10.1097/RLI.0000000000000746. - DOI - PubMed
    1. Remez T., Litany O., Giryes R., Bronstein A.M. Deep class-aware image denoising; Proceedings of the 2017 International Conference on Sampling Theory and Applications (SampTA); Tallinn, Estonia. 3–7 July 2017; pp. 138–142.
    1. Li L., Wang R., Wang W., Gao W. A low-light image enhancement method for both denoising and contrast enlarging; Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP); Quebec City, QC, Canada. 27–30 September 2015; pp. 3730–3734.
    1. Chen C., Chen Q., Xu J., Koltun V. Learning to see in the dark; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Salt Lake City, UT, USA. 18–23 June 2018; pp. 3291–3300.

LinkOut - more resources