petiteFinder: an automated computer vision tool to compute Petite colony frequencies in baker's yeast

Abstract

Background: Mitochondrial respiration is central to cellular and organismal health in eukaryotes. In baker's yeast, however, respiration is dispensable under fermentation conditions. Because yeast are tolerant of this mitochondrial dysfunction, yeast are widely used by biologists as a model organism to ask a variety of questions about the integrity of mitochondrial respiration. Fortunately, baker's yeast also display a visually identifiable Petite colony phenotype that indicates when cells are incapable of respiration. Petite colonies are smaller than their Grande (wild-type) counterparts, and their frequency can be used to infer the integrity of mitochondrial respiration in populations of cells. Unfortunately, the computation of Petite colony frequencies currently relies on laborious manual colony counting methods which limit both experimental throughput and reproducibility.

Results: To address these problems, we introduce a deep learning enabled tool, petiteFinder, that increases the throughput of the Petite frequency assay. This automated computer vision tool detects Grande and Petite colonies and computes Petite colony frequencies from scanned images of Petri dishes. It achieves accuracy comparable to human annotation but at up to 100 times the speed and outperforms semi-supervised Grande/Petite colony classification approaches. Combined with the detailed experimental protocols we provide, we believe this study can serve as a foundation to standardize this assay. Finally, we comment on how Petite colony detection as a computer vision problem highlights ongoing difficulties with small object detection in existing object detection architectures.

Conclusion: Colony detection with petiteFinder results in high accuracy Petite and Grande detection in images in a completely automated fashion. It addresses issues in scalability and reproducibility of the Petite colony assay which currently relies on manual colony counting. By constructing this tool and providing details of experimental conditions, we hope this study will enable larger-scale experiments that rely on Petite colony frequencies to infer mitochondrial function in yeast.

Keywords: Baker’s yeast; Colony morphology; Computer vision; Mitochondrial respiration; Object detection.

Fig. 1

An overview of the labeled dataset. a A composite image showing Petri dishes containing synthetic (top 4 Petri dishes) and complete media (bottom 2 Petri dishes) with yeast colonies on their surface. Six petri dishes at a time were placed in a 3D printed insert (black structure bordering petri dish images) and scanned on a computer scanner bottom-up. The synthetic media was SC-ura-trp and the complete media was YPADG (both 0.1% glucose and 3% glycerol carbon source). b Individual plate images were cropped out of the large image, and 83 of these images were annotated using the LabelMe annotation tool [24]. Grande and Petite colonies are indicated by blue and orange bounding boxes, respectively. Experimental variation in media preparation/pouring of plates produces diffuse low-quality scans (non-ideal) and sharper plate images (ideal) as shown by example plate crops (i) and (ii) for synthetic media. Plate crop (iii) is complete media that has uniform agar opacity across all images

Fig. 2

The architecture of the object detection pipeline. The entire plate image is first sliced into cropped images (red boxes) using a sliding window with the SAHI package. The image slice is scaled to ($1024\times 1024$) regardless of its size and is passed into a ResNet-50-FPN convolutional neural network backbone. Here, Sx represents the convolutional stage of the ResNet-50 network. Solid blue indicates stages that were frozen during training. Convolution block outputs are rescaled (dotted arrow), undergo element-wise addition (intersection of solid and dotted arrows), and in one case, undergo max pooling (red arrow). This is the feature pyramid underlying the FPN (feature pyramid network) nomenclature. The output of this step is a collection of feature maps of various scales (resolutions) and semantic values. All feature maps are passed into the Faster R-CNN object detection architecture. In this architecture, the region proposal network (RPN), which is fully convolutional, generates region proposals that are likely to contain objects. These proposals are then passed to a final regressor and classifier which outputs a predicted bounding box and probability score for the class of the predicted object. Bounding boxes per image slice are filtered through non-maximal suppression (NMS). Finally, bounding box predictions on all image slices are merged into the final image using a non-maximum merging algorithm (NMM). The final output is a set of predicted bounding boxes on the entire image with associated Grande/Petite class probabilities (lower left image and zoomed portion)

Fig. 3

An example of the interface in the amend function within petiteFinder. a The default view upon opening the GUI. Orange and blue boxes are Petites and Grandes, respectively. Arrow buttons in the top left can be used to move through images that have predictions. Hovering over colonies with a cursor reveals their class and probability score in the upper right. b An example where a user is zooming into an image to add a new Grande bounding box that is being drawn in white in the center of the frame. Upon finishing the drawing movement with the mouse, the bounding box switches to the appropriate colour (blue) in this case. Functions also exist to remove bounding boxes after being selected with the cursor

Fig. 4

petiteFinder performance compared to manual counting alongside a test of model robustness. a A plate-wise comparison of predicted and manually counted Petite colony frequencies. The red curves are the predictions and the black curves are manual counting. The average absolute deviation between predicted and ground truth Petite percentage (prediction error) is 1.7%. The gray envelope is the binomial sampling error (ground truth ± standard deviation), assuming that Petite production is a Bernoulli process with a probability equal to the ground truth frequency when sampling 78 colonies per plate image. b Top panel: Precision and recall of each colony category as a function of test image resolution normalized by the image resolution of the training data. Bottom panel: Absolute error in petite frequency as a function of test image resolution. The error is defined as the deviation between predicted and ground truth Petite frequency. Labels are also included to denote absolute image resolutions