Context-Aware Image Compression

Abstract

We describe a physics-based data compression method inspired by the photonic time stretch wherein information-rich portions of the data are dilated in a process that emulates the effect of group velocity dispersion on temporal signals. With this coding operation, the data can be downsampled at a lower rate than without it. In contrast to previous implementation of the warped stretch compression, here the decoding can be performed without the need of phase recovery. We present rate-distortion analysis and show improvement in PSNR compared to compression via uniform downsampling.

Fig 1. Overview schematic for image compression codec with warped stretch.

The input is split into two components: i) the downsampled warped image and ii) the metadata, which contains a compressed version of the warp kernel. These two components are jointly used for recovering the original input. Since the warp kernel is image-dependent, we must send it as part of the compressed file, which creates extra overhead relative to an image-independent compression technique, such as uniform sampling. However, if the metadata can be compressed extremely compactly, the overall compression ratio can still be significant.

Fig 2. Flowchart for warped stretch compression using piecewise decomposition.

(A) A separate warp kernel is generated for each row of the image. The warp is based on the local row bandwidth, i.e. the derivative of the image intensity. The input image is then warped by the kernel and downsampled at a uniform rate. The compressed image is then saved into a custom binary file format (WST), along with the warp kernel, which itself is compressed via piecewise decomposition. To reconstruct the image, we decompress the piecewise-decomposed warp kernel and use it to perform non-uniform upsampling on the reloaded compressed image. (B) For comparison purposes, we also uniformly downsample the input image in 1D with a lower downsampling rate that accounts for the warp kernel overhead saved in the WST binary format.

Fig 3. Results for row 838 (out of 1672) of the fractal clock image at 6X warped stretch compression with overhead compensation.

The line signal (A) is first rescaled using non-uniform cubic interpolation as defined by the warp kernel, generated according to Eq (7). In this warped space (B), the signal can now be downsampled at a uniform rate (indicated by the red circles) that is lower than what is possible using uniform downsampling, at a given reconstruction quality. The number of downsampled points is less than 1/8^th of the number of pixels in the original line signal so that the compression ratio becomes 8X after taking the warp kernel overhead into consideration when saving to file. Both the warping and the downsampling operations can be reversed to reconstruct the line signal (C). The corresponding locations of the downsampled points (red circles in (B)) overlay the (A) original and (C) reconstructed line signals for visual reference. The dashed frames in (A) and (B) are shown in closeup form in Fig 4.

Fig 4. Closeup of Fig 3A and 3B.

(A) A particular subsection of row 838 of the fractal clock image (dashed frame in Fig 3A), which contains a mixture of feature-sparse and feature-dense regions, is shown in expanded form. (B) The same subsection, after warping by the warp kernel (dashed frame in Fig 3B). The subsection in (B) matches the length of the original line subsection (A) to show the redistribution of the feature density caused by the warped stretch transform. The corresponding locations of the downsampled points (red circles in (B)) overlay the original line signal (B) for visual reference.

Fig 5. Comparison of compression performance with the fractal clock image.

The original input image (A-B) and the 4X uniformly downsampled case (C-E), as compared with the reconstructed image after 5.25X warped stretch compression (F-H). The downsampling rate for the uniform case was increased (hence the image quality improvement) such that the resultant file sizes for both warped and uniform compression become equal (to compensate for the warp kernel overhead). After reconstruction, the warped case (G-H) achieved a PSNR of 37.7 dB, which was 6.32 dB better than the uniform downsampling case (D-E).

Fig 6. Comparison of compression performance with the colour portrait image.

(A) The 8X uniformly downsampled image and (B) the 10.2X warp stretch-compressed image are shown with (C) the original image and (D-E) their respective reconstructions, while (F-H) are, in turn, their respective close-up portions. Further zoom-ins on the rims of the glasses are shown in (I-K), highlighting the failure of uniform downsampling to capture this sharp feature. The downsampling rate for the uniform case was adjusted such that the resultant file sizes for both warped and uniform compression become equal; however, since all three colour channels share the same warp kernel, the overhead burden is reduced by a third in this scenario. After reconstruction, the warped case (E,H,K) achieved a PSNR of 39.1 dB, which was 3.11 dB better than the uniform downsampling case (D,G,I).

Fig 7. Empirical rate distortion plot for fractal clock and portrait images.

The PSNR of warped stretch compression (solid) is compared with uniform downsampling (dotted) over a range of compression ratios for (A) the grayscale fractal clock image and (B) the colour portrait image. At a compression ratio of 4X, warped stretch outperforms in PSNR by 6.32 dB in the clock, and by 4.10 dB in the portrait. Beyond the compression ratios of 9X and 20X respectively, the overhead from the warp kernel completely compromises the performance.