Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Jan 1;5(1):7.
doi: 10.3390/jimaging5010007.

Optimized Memory Allocation and Power Minimization for FPGA-Based Image Processing

Paulo Garcia  1 , Deepayan Bhowmik  2 , Robert Stewart  3 , Greg Michaelson  3 , Andrew Wallace  4
Affiliations

Optimized Memory Allocation and Power Minimization for FPGA-Based Image Processing

Paulo Garcia et al. J Imaging. .

Abstract

Memory is the biggest limiting factor to the widespread use of FPGAs for high-level image processing, which require complete frame(s) to be stored in situ. Since FPGAs have limited on-chip memory capabilities, efficient use of such resources is essential to meet performance, size and power constraints. In this paper, we investigate allocation of on-chip memory resources in order to minimize resource usage and power consumption, contributing to the realization of power-efficient high-level image processing fully contained on FPGAs. We propose methods for generating memory architectures, from both Hardware Description Languages and High Level Synthesis designs, which minimize memory usage and power consumption. Based on a formalization of on-chip memory configuration options and a power model, we demonstrate how our partitioning algorithms can outperform traditional strategies. Compared to commercial FPGA synthesis and High Level Synthesis tools, our results show that the proposed algorithms can result in up to 60% higher utilization efficiency, increasing the sizes and/or number of frames that can be accommodated, and reduce frame buffers' dynamic power consumption by up to approximately 70%. In our experiments using Optical Flow and MeanShift Tracking, representative high-level algorithms, data show that partitioning algorithms can reduce total power by up to 25% and 30%, respectively, without impacting performance.

Keywords: design; field programmable gate array (FPGA); image processing; memory; power.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Figures

Figure 1
Figure 1
Mapping a 3-D array into row-major and colum-major order 2-D arrays.
Figure 2
Figure 2
Mapping 2-D array of dimensions x=Bw and y=W×H to a×b BRAMs configured for width M and height N.
Figure 3
Figure 3
Partitioning across two BRAMs horizontally. Each access consumes two power consumption quantums.
Figure 4
Figure 4
Partitioning across one BRAM horizontally. Each access consumes 1 power consumption quantum.
Figure 5
Figure 5
Proposed design flow from HDL and HLS, highlighting the additional steps required for minimizing utilization and power.
Figure 6
Figure 6
BRAM utilization efficiency for RGB frames: Vivado HLS versus proposed methods.
Figure 7
Figure 7
Static power consumption: Vivado HLS versus proposed methods.
Figure 8
Figure 8
Total dynamic power consumption for sequential read/write: Vivado HLS versus proposed methods.
Figure 9
Figure 9
Total dynamic power consumption for 3 × 3 sliding window read: Vivado HLS versus proposed methods.
Figure 10
Figure 10
BRAM power consumption for sequential read/write: Vivado HLS versus proposed methods.
Figure 11
Figure 11
BRAM power consumption for 3 × 3 sliding window read: Vivado HLS versus proposed methods.
Figure 12
Figure 12
Optical Flow results using the implementation from [40]. (a,b): source frames. (c): output from 1 scale optical flow (used in our FPGA implementation). (d): output from 5 scales optical flow.
Figure 13
Figure 13
TV-L1 Optical Flow power consumption on Virtex 7.
Figure 14
Figure 14
Zedboard connected to PC through Ethernet.
Figure 15
Figure 15
MeanShift Tracking: real-time face tracking displayed on PC. Image sent from Zedboard over Ethernet connection.
Figure 16
Figure 16
MeanShift Tracking power consumption on Zedboard.
See this image and copyright information in PMC

References

    1. Wang J., Zhong S., Yan L., Cao Z. An Embedded System-on-Chip Architecture for Real-time Visual Detection and Matching. IEEE Trans. Circuits Syst. Video Technol. 2014;24:525–538. doi: 10.1109/TCSVT.2013.2280040. - DOI
    1. Mondal P., Biswal P.K., Banerjee S. FPGA based accelerated 3D affine transform for real-time image processing applications. Comput. Electr. Eng. 2016;49:69–83. doi: 10.1016/j.compeleceng.2015.04.017. - DOI
    1. Wang W., Yan J., Xu N., Wang Y., Hsu F.H. Real-Time High-Quality Stereo Vision System in FPGA. IEEE Trans. Circuits Syst. Video Technol. 2015;25:1696–1708. doi: 10.1109/TCSVT.2015.2397196. - DOI
    1. Jin S., Cho J., Pham X.D., Lee K.M., Park S.K., Kim M., Jeon J.W. FPGA Design and Implementation of a Real-Time Stereo Vision System. IEEE Trans. Circuits Syst. Video Technol. 2010;20:15–26.
    1. Perri S., Frustaci F., Spagnolo F., Corsonello P. Design of Real-Time FPGA-based Embedded System for Stereo Vision; Proceedings of the 2018 IEEE International Symposium on Circuits and Systems (ISCAS); Florence, Italy. 27–30 May 2018; pp. 1–5.

LinkOut - more resources