Multi-template matching: a versatile tool for object-localization in microscopy images

Abstract

Background: The localization of objects of interest is a key initial step in most image analysis workflows. For biomedical image data, classical image-segmentation methods like thresholding or edge detection are typically used. While those methods perform well for labelled objects, they are reaching a limit when samples are poorly contrasted with the background, or when only parts of larger structures should be detected. Furthermore, the development of such pipelines requires substantial engineering of analysis workflows and often results in case-specific solutions. Therefore, we propose a new straightforward and generic approach for object-localization by template matching that utilizes multiple template images to improve the detection capacity.

Results: We provide a new implementation of template matching that offers higher detection capacity than single template approach, by enabling the detection of multiple template images. To provide an easy-to-use method for the automatic localization of objects of interest in microscopy images, we implemented multi-template matching as a Fiji plugin, a KNIME workflow and a python package. We demonstrate its application for the localization of entire, partial and multiple biological objects in zebrafish and medaka high-content screening datasets. The Fiji plugin can be installed by activating the Multi-Template-Matching and IJ-OpenCV update sites. The KNIME workflow is available on nodepit and KNIME Hub. Source codes and documentations are available on GitHub (https://github.com/multi-template-matching).

Conclusion: The novel multi-template matching is a simple yet powerful object-localization algorithm, that requires no data-pre-processing or annotation. Our implementation can be used out-of-the-box by non-expert users for any type of 2D-image. It is compatible with a large variety of applications including, for instance, analysis of large-scale datasets originating from automated microscopy, detection and tracking of objects in time-lapse assays, or as a general image-analysis step in any custom processing pipelines. Using different templates corresponding to distinct object categories, the tool can also be used for classification of the detected regions.

Keywords: Classification; Fiji; KNIME; Medaka; Object-localization; Object-recognition; OpenCV; Pattern recognition; Template matching; Zebrafish.

Fig. 1

Head region detection in oriented zebrafish larvae using single template matching. a Searched image (2048 × 2048 pixels, scale bar: 1 mm) with template as inset (188 × 194 pixels), search region in orange (1820 × 452 pixels) and predicted location in blue. The red cross corresponds to the position of the global maximum of the correlation map (as in b). b Correlation map with global maximum (red cross) indicating the position of the bounding box in a. The grid area indicates the smaller size of the correlation map compared to the image in which the search is performed (see also Additional file 13). c Montage of detected head regions within a 96 well plate

Fig. 2

Multi-template matching and Non-Maxima Suppression for the detection of randomly oriented and positioned medaka embryos. a Image in which the search is performed (2048 × 2048 pixels - scale bar: 1 mm) and template as inset (400 × 414 pixels). The search was performed with a set of templates (original template, vertical and horizontal flip, each rotated by 90°, 180° and 270°). Parameters for the detection: score type: 0-mean normalized cross-correlation, N = 4 expected objects per image, score threshold: 0.35, maximal overlap between bounding boxes: 0.25. b One of the derived correlation maps from A: red crosses indicate possible local maxima before Non-Maxima Suppression (NMS). The grid area indicates the smaller size of the correlation map compared to the image in which the search is performed as explained in Additional file 13. c Bounding boxes associated to the maxima shown in b and overlaid on the searched image. Colours are highlighting overlapping bounding boxes. The bounding box dimensions are identical to the dimensions of the template used for the search. d, e Preventing overlapping detections by NMS. Shown are 2 overlapping bounding boxes predicting possible object locations. Each predicted location is associated to a probability score S to contain an object. The ratio between the intersection (d) and the union (e) area of the bounding boxes (Intersection over Union or IoU) is computed to decide whether the 2 overlapping bounding boxes are likely to predict the location of the same object (IoU close to 1) or the locations of distinct objects that are close to each other (IoU close to 0). For a detailed description of Non-Maxima Suppression see Additional file 13. f Yielded object detections after NMS with a maximal IoU of 0.25, to return the N_objects = 4 best detections