Recognition of natural objects in the archerfish

Abstract

Recognition of individual objects and their categorization is a complex computational task. Nevertheless, visual systems can perform this task in a rapid and accurate manner. Humans and other animals can efficiently recognize objects despite countless variations in their projection on the retina due to different viewing angles, distance, illumination conditions and other parameters. To gain a better understanding of the recognition process in teleosts, we explored it in archerfish, a species that hunts by shooting a jet of water at aerial targets and thus can benefit from ecologically relevant recognition of natural objects. We found that archerfish not only can categorize objects into relevant classes but also can do so for novel objects, and additionally they can recognize an individual object presented under different conditions. To understand the mechanisms underlying this capability, we developed a computational model based on object features and a machine learning classifier. The analysis of the model revealed that a small number of features was sufficient for categorization, and the fish were more sensitive to object contours than textures. We tested these predictions in additional behavioral experiments and validated them. Our findings suggest the existence of a complex visual process in the archerfish visual system that enables object recognition and categorization.

Keywords: Computational model; Object categorization; Visual object recognition; Visual system.

Fig. 1.

Object recognition problem. Object recognition involves the identification of objects regardless of transformations in size, contrast, orientation or viewing angle. (A) An example of object recognition. In this case, a specific spider needs to be identified in the presence of other insects. (B) An example of object recognition of an object class. In this case, an animal (insect or spider) needs to be recognized in the presence of non-animate objects (leaves or flowers).

Fig. 2.

Behavioral experimental setup. (A) The archerfish is presented with two objects on the screen: a target and a distractor. The fish is rewarded if it selects the target image. (B) An example of success rate per session in the training process (red) and experiment (blue) of fish 1 in the spider recognition task (20 trials each session). (C) Learning curve of five example fish. The fish reached 70% success rate or higher in three successive sessions after 5–10 sessions of training.

Fig. 3.

Archerfish are capable of invariant object recognition. (A) Examples of a single target spider from different viewpoints and with different contrast levels (top row) and other distractor objects and spiders (bottom row). (B) Success rate of three archerfish in recognizing the target spider: means±95% highest density interval (HDI). Shading around the chance level of 0.5 represents a region of practical equivalence (ROPE) of 0.05. Fish responded in 149–199 trials in each experiment (out of 200 trials). (C) Examples of a single target ant from different viewpoints and with different contrast levels (top row) and other distractor objects and ants (bottom row). (D) Success rate of three archerfish in recognizing the target ant: means±95% HDI. Shading represents a ROPE of 0.05 around the chance level of 0.5. Fish responded in 182–200 trials in each experiment (out of 200 trials).

Fig. 4.

Archerfish can categorize novel objects into groups. The fish were trained to categorize animal and non-animal objects. (A) Examples of animal objects (insects and spiders, top row) and non-animal objects (leaves and flowers, bottom row). (B) Success rate of 10 fish in selecting an object from its designated category: means±95% HDI. Shading represents a ROPE of 0.05 around the chance level of 0.5. Fish 5–9 were rewarded for choosing an animal; fish 10–14 were rewarded for choosing a non-animal. Fish responded in 694–814 trials (out of 820 trials).

Fig. 5.

Model building. (A) In a behavioral experiment, the fish was exposed to two objects, made a decision about the object category and executed a shot. The support vector machine (SVM) classifier was fed with the features extracted from the images. (B) Examples of the extracted visual features. (C) Success rate of different classifiers: SVM classifier using raw images, SVM classifier using extracted features, k-nearest neighbor, discriminant analysis and neural network. (D) SVM classifier success rate in predicting the objects' true category (blue line, which corresponds to the blue outlined box in the model) and the model's success rate in predicting fish selection (red line, which corresponds to the red outlined box in the model). Separate features were added in the order of their contribution to the classifier's success. (E) SVM classifier success rate for combinations of features: two features of shape and two features of texture.

Fig. 6.

Shape features are more important to recognition than texture. (A) Examples of animal target silhouettes and textures (top row) and non-animal target silhouettes and textures (bottom row). (B) Success rate at recognizing the target category in the original experiment with a full object (green bars) and with silhouettes alone (blue bars) and texture alone (red bars) in 6 fish: means±95% HDI. Shading represents a ROPE of 0.05 around the chance level of 0.5. For all fish, no significant difference was found in the response rate between the original and the silhouette experiments: 95% HDI was above the chance level in all fish. In the texture experiment, the 95% HDI range included the chance level of 0.5. For all but two fish, there was a significant difference between the original and the texture experiment. For the shape experiment, we had 200 trials and fish responded in 186–200 trials. For the texture experiment, we had 100 trials and fish responded in 87–100 trials.

Fig. 7.

Fish errors are not correlated with object identity. (A) The original experiment in object categorization was repeated for selected sets of objects: objects that were previously selected correctly by the fish, objects that were previously selected incorrectly by the fish, objects that the model labeled correctly and objects that the model labeled incorrectly (means±95% HDI). The 95% HDI of the fish success rate for all sets of objects was above chance level for all three fish that finished all sets. (B) Left: portion of images identified correctly and incorrectly by the fish and by the fish-trained model from two datasets: the dataset of images selected correctly in the original categorization task by the fish (top rows) and the set of images selected incorrectly by the fish (bottom rows). Right: success rate for the same groups expected under independence. For the model correct and originally correct conditions, there were 200 trials and fish responded in 192–200 trials. For the originally incorrect condition, there were 140 trials, and fish responded in 136–140 trials. For the model incorrect condition, there were 30 trials and fish responded in 29–30 trials.