Studierfenster: an Open Science Cloud-Based Medical Imaging Analysis Platform

Abstract

Imaging modalities such as computed tomography (CT) and magnetic resonance imaging (MRI) are widely used in diagnostics, clinical studies, and treatment planning. Automatic algorithms for image analysis have thus become an invaluable tool in medicine. Examples of this are two- and three-dimensional visualizations, image segmentation, and the registration of all anatomical structure and pathology types. In this context, we introduce Studierfenster ( www.studierfenster.at ): a free, non-commercial open science client-server framework for (bio-)medical image analysis. Studierfenster offers a wide range of capabilities, including the visualization of medical data (CT, MRI, etc.) in two-dimensional (2D) and three-dimensional (3D) space in common web browsers, such as Google Chrome, Mozilla Firefox, Safari, or Microsoft Edge. Other functionalities are the calculation of medical metrics (dice score and Hausdorff distance), manual slice-by-slice outlining of structures in medical images, manual placing of (anatomical) landmarks in medical imaging data, visualization of medical data in virtual reality (VR), and a facial reconstruction and registration of medical data for augmented reality (AR). More sophisticated features include the automatic cranial implant design with a convolutional neural network (CNN), the inpainting of aortic dissections with a generative adversarial network, and a CNN for automatic aortic landmark detection in CT angiography images. A user study with medical and non-medical experts in medical image analysis was performed, to evaluate the usability and the manual functionalities of Studierfenster. When participants were asked about their overall impression of Studierfenster in an ISO standard (ISO-Norm) questionnaire, a mean of 6.3 out of 7.0 possible points were achieved. The evaluation also provided insights into the results achievable with Studierfenster in practice, by comparing these with two ground truth segmentations performed by a physician of the Medical University of Graz in Austria. In this contribution, we presented an online environment for (bio-)medical image analysis. In doing so, we established a client-server-based architecture, which is able to process medical data, especially 3D volumes. Our online environment is not limited to medical applications for humans. Rather, its underlying concept could be interesting for researchers from other fields, in applying the already existing functionalities or future additional implementations of further image processing applications. An example could be the processing of medical acquisitions like CT or MRI from animals [Clinical Pharmacology & Therapeutics, 84(4):448-456, 68], which get more and more common, as veterinary clinics and centers get more and more equipped with such imaging devices. Furthermore, applications in entirely non-medical research in which images/volumes need to be processed are also thinkable, such as those in optical measuring techniques, astronomy, or archaeology.

Keywords: Augmented reality; CNN; Client/server; Cloud; Deep learning; GAN; ITK; Medical image analysis; Python; VTK; Virtual reality; Whitepaper.

Fig. 1

Overall platform architecture of Studierfenster with its modules and communications

Fig. 2

The current Studierfenster landing page

Fig. 3

Client-sided DICOM browser and converter from Studierfenster [30]

Fig. 4

After the selection and conversion of specific (or all) studies or series to compressed .nrrd (nearly raw raster data) files, theses can be downloaded as a single .zip file

Fig. 5

Medical 3D Viewer of Studierfenster with the classical 2D views in axial, coronal, and sagittal directions (right) and volume rendering (middle) [31, 32]

Fig. 6

Manual segmentation of a brain tumor (glioblastoma multiforme (GBM), blue) in a magnetic resonance imaging (MRI) scan of a patient [37]

Fig. 7

An initial centerline (left, red) and the corresponding smoothed centerline (right, red) calculated and visualized with Studierfenster [45]

Fig. 8

Skull reconstruction under Studierfenster: defected skull (left window), reconstructed skull (window in the middle), and subtraction (right window) [47]

Fig. 9

3D Face Reconstruction and Registration module of Studierfenster: extracted surface of a medical head/face CT scan (left window), reconstructed 3D model from a single photo from a person’s face (window in the middle), and registration of both 3D models (right window)

Fig. 10

Studierfenster functionality of calculating dice similarity coefficient (DSC) and directed and undirected Hausdorff distance (HD) scores for two uploaded volumes

Fig. 11

After the calculation of DSCs and HDs for several volumes, our tool provides the options filtering for specific values, searching for specific values in the calculated metrics, and exporting filtered metric lists in different file formats, such as CSV, Excel, and PDF

Fig. 12

User study results visualized as a bar chart, presenting the mean of the ratings of all users grouped per question [33]

Fig. 13

Map of the worldwide distribution of the first 1000 users, accessing and interaction with Studierfenster (monitored with Google Analytics)