Stereo Imaging Using Hardwired Self-Organizing Object Segmentation

Abstract

Stereo vision utilizes two cameras to acquire two respective images, and then determines the depth map by calculating the disparity between two images. In general, object segmentation and stereo matching are some of the important technologies that are often used in establishing stereo vision systems. In this study, we implement a highly efficient self-organizing map (SOM) neural network hardware accelerator as unsupervised color segmentation for real-time stereo imaging. The stereo imaging system is established by pipelined, hierarchical architecture, which includes an SOM neural network module, a connected component labeling module, and a sum-of-absolute-difference-based stereo matching module. The experiment is conducted on a hardware resources-constrained embedded system. The performance of stereo imaging system is able to achieve 13.8 frames per second of 640 × 480 resolution color images.

Keywords: SOM; object segmentation; stereo vision.

Figure 1

System structure of the proposed embedded stereo system.

Figure 2

Stereo matching flowchart.

Figure 3

Dual camera vision module.

Figure 4

RTL-schematic diagram of the dual camera vision module.

Figure 5

SOM-based image segmentation module.

Figure 6

System architecture of the SOM training module.

Figure 7

Grafcet discrete-event model of the SOM training module.

Figure 8

RTL-schematic diagram of the random generator.

Figure 9

System architecture of the SOM color classification module.

Figure 10

Discrete-event model of the SOM color classification module.

Figure 11

RTL-schematic diagram of the SOM-based image segmentation module.

Figure 12

Hardware architecture of the CCL module.

Figure 13

System architecture of the SAD-based stereo matching module.

Figure 14

Discrete-event model of the pipeline controller. (a) Stage controller. (b) Top controller.

Figure 15

Segmentation results of an image of the Sydney Opera House. (a) Original image. (b) Ideal image segmentation. (c) Segmented image using K-means. (d) Segmented image using SOM.

Figure 16

Results of the proposed stereo vision method. (a) Left image. (b) Right image. (c) Image after color classification. (d) Image after CCL. (e) Depth map.

Figure 17

Labeled objects.

Figure 18

Hardware configuration of the embedded stereo vision system. (a) System hardware architecture. (b) Photo of the system.

Figure 19

(a) Left image. (b) Right image. (c) Image after left-image object segmentation. (d) Image after left-image CCL processing. (e) Image after stereo matching.