Classification of Holograms with 3D-CNN

Abstract

A hologram, measured by using appropriate coherent illumination, records all substantial volumetric information of the measured sample. It is encoded in its interference patterns and, from these, the image of the sample objects can be reconstructed in different depths by using standard techniques of digital holography. We claim that a 2D convolutional network (CNN) cannot be efficient in decoding this volumetric information spread across the whole image as it inherently operates on local spatial features. Therefore, we propose a method, where we extract the volumetric information of the hologram by mapping it to a volume-using a standard wavefield propagation algorithm-and then feed it to a 3D-CNN-based architecture. We apply this method to a challenging real-life classification problem and compare its performance with an equivalent 2D-CNN counterpart. Furthermore, we inspect the robustness of the methods to slightly defocused inputs and find that the 3D method is inherently more robust in such cases. Additionally, we introduce a hologram-specific augmentation technique, called hologram defocus augmentation, that improves the performance of both methods for slightly defocused inputs. The proposed 3D-model outperforms the standard 2D method in classification accuracy both for in-focus and defocused input samples. Our results confirm and support our fundamental hypothesis that a 2D-CNN-based architecture is limited in the extraction of volumetric information globally encoded in the reconstructed hologram image.

Keywords: 3D-CNN; CNN; deep learning; digital holography; neural networks.

Figure 1

(A) Samples from the classes. (B) Distribution of the number of samples in classes. (C) The volumetric input creation for the 3D neural network. Note that only the hologram’s amplitude image is shown and the phase image is omitted in this illustration for the sake of simplicity. The backward and forward propagation terms denote the direction of the propagation.

Figure 2

Illustration of the 3D-model architecture. The input of the 3D network is the amplitude and phase image (C = 2) of the initial hologram together with its 6 steps forward and 6 steps backward propagated forms (D = 13). The output of the network is the class log probabilities. The symbol k denotes the 3 dimensional kernel size (D,H,W), p the padding, and s the stride.

Figure 3

Examples of the application of hologram defocus augmentation for a sample in class 2-TRCs. In this illustration, from the original in-focus input hologram, we generate 8 slightly altered examples by propagating the hologram in the range of [−13.72, 13.72] $\mathsf{\mu }$m. The backward and forward propagation terms denote the direction of the propagation. Backward propagation means that we propagate in the negative direction: for example, propagating one step backward is equivalent to propagating with −3.43 $\mathsf{\mu }$m.

Figure 4

(A) Boxplot of the accuracy of the models with different input types; (B) F1-score matrix of the 2D-model in the case of in-focus training and test inputs. (C) F1-score matrix of the 3D-model in the case of in-focus training and test inputs.