Vision-based egg quality prediction in Pacific bluefin tuna (Thunnus orientalis) by deep neural network

Abstract

Closed-cycle aquaculture using hatchery produced seed stocks is vital to the sustainability of endangered species such as Pacific bluefin tuna (Thunnus orientalis) because this aquaculture system does not depend on aquaculture seeds collected from the wild. High egg quality promotes efficient aquaculture production by improving hatch rates and subsequent growth and survival of hatched larvae. In this study, we investigate the possibility of a simple, low-cost, and accurate egg quality prediction system based only on photographic images using deep neural networks. We photographed individual eggs immediately after spawning and assessed their qualities, i.e., whether they hatched normally and how many days larvae survived without feeding. The proposed system predicted normally hatching eggs with higher accuracy than human experts. It was also successful in predicting which eggs would produce longer-surviving larvae. We also analyzed the image aspects that contributed to the prediction to discover important egg features. Our results suggest the applicability of deep learning techniques to efficient egg quality prediction, and analysis of early developmental stages of development.

Figure 1

Overview of the proposed framework for egg quality prediction and analysis. (a) Only the egg region is extracted from a photographed image by Faster R-CNN. (b) By training the VGG16-based network using the egg images, NH and SD predictions are performed. (c) The features of the eggs that contributed to the NH prediction were examined by visualization.

Figure 2

Examples of three types of the input egg images. Images focused on the (a) cytoplasm, (b) contour of the egg, and (c) oil droplet.

Figure 3

Result of NH prediction. (a) (1), (2), (3), and (4) were NH, NH, not NH, and not NH, respectively. The system predicted (1), (2), (3), and (4) as NH, NH, not NH, and NH, respectively. That is, the system successfully predicted for (1), (2), and (3). (b) Averages and standard errors of accuracy and F-measure for NH prediction in ten-fold cross-validation.

Figure 4

Result of SD prediction. (a) The system successfully predicted (1) and (2) but failed in (3) and (4). (1) and (4) were more than four SD; (2) and (3) were NH but within four SD. (b) Accuracy averages and standard errors and F-measure for SD prediction in ten-fold cross-validation.

Figure 5

Examples of visualized image aspects that contributed to NH prediction. The images in the first and third rows are the input contour images; those in the second and bottom rows show the parts of the eggs that contributed to the prediction as visualized by Grad-CAM. Images (a) and (c) were predicted as NH; (b) and (d) were predicted as not NH. Each acronym below the visualized images indicates the hatching status. That is, examples of true-positive (a), false-negative (b), false-positive (c), and true-negative (d), respectively.

Figure 6

Test image examples to compare the prediction accuracy of experts with the network. (a,b) Actually NH. The network predicted them correctly while four experts failed. (c) Actually not NH. The network and only one expert predicted correctly. (d) Actually not NH. The network failed to predict, and three experts predicted correctly.