Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Oct;54(3):248-258.
doi: 10.5152/eurasianjmed.2022.22096.

Artificial Intelligence in Healthcare Competition (TEKNOFEST-2021): Stroke Data Set

Ural Koç  1 , Ebru Akçapınar Sezer  2 , Yaşar Alper Özkaya  3 , Yasin Yarbay  4 , Onur Taydaş  5 , Veysel Atilla Ayyıldız  6 , Hüseyin Alper Kızıloğlu  7 , Uğur Kesimal  8 , İmran Çankaya  9 , Muhammed Said Beşler  1 , Emrah Karakaş  4 , Fatih Karademir  10 , Nihat Barış Sebik  4 , Murat Bahadır  11 , Özgür Sezer  4 , Batuhan Yeşilyurt  10 , Songul Varlı  10   12 , Erhan Akdoğan  10   13 , Mustafa Mahir Ülgü  4 , Şuayip Birinci  14
Affiliations

Artificial Intelligence in Healthcare Competition (TEKNOFEST-2021): Stroke Data Set

Ural Koç et al. Eurasian J Med. 2022 Oct.

Abstract

Objective: The artificial intelligence competition in healthcare was organized for the first time at the annual aviation, space, and technology festival (TEKNOFEST), Istanbul/Türkiye, in September 2021. In this article, the data set preparation and competition processes were explained in detail; the anonymized and annotated data set is also provided via official website for further research.

Materials and methods: Data set recorded over the period covering 2019 and 2020 were centrally screened from the e-Pulse and Teleradiology System of the Republic of Türkiye, Ministry of Health using various codes and filtering criteria. The data set was anonymized. The data set was prepared, pooled, curated, and annotated by 7 radiologists. The training data set was shared with the teams via a dedicated file transfer protocol server, which could be accessed using private usernames and passwords given to the teams under a nondisclosure agreement signed by the representative of each team.

Results: The competition consisted of 2 stages. In the first stage, teams were given 192 digital imaging and communications in medicine images that belong to 1 of 3 possible categories namely, hemorrhage, ischemic, or non-stroke. Teams were asked to classify each image as either stroke present or absent. In the second stage of the competition, qualifying 36 teams were given 97 digital imaging and communications in medicine images that contained hemorrhage, ischemia, or both lesions. Among the employed methods, Unet and DeepLabv3 were the most frequently observed ones.

Conclusion: Artificial intelligence competitions in healthcare offer good opportunities to collect data reflecting various cases and problems. Especially, annotated data set by domain experts is more valuable.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Data annotation and workflow management software, RAD-X Picture Archiving and Communication System Version 2.0 (Simplex IT Inc.-Ankara-Türkiye).
Figure 2.
Figure 2.
Data viewing and segmentation window.
Figure 3.
Figure 3.
Radiologist task sharing and data set annotation methodology.
Figure 4.
Figure 4.
The F1 scores for 49 teams.
Figure 5.
Figure 5.
The IoU scores of the 36 teams.
Figure 6.
Figure 6.
Top 3 IoU scores for 4 images (misleading images). First row CT image consisted of both ischemic core and aneurysm ipsilaterally, second row CT image had both ischemic core and vascular anomaly contralaterally, third row CT image had arachnoid cyst and ischemic core contralaterally, and fourth row CT image had both ischemic cores and hemorrhagic areas. CT, computed tomography.
Figure 7.
Figure 7.
Top IoU scores of 3 images were demonstrated. All of the images are of the hyperacute ischemic stroke cases. Imaging findings that are hard to catch, even for the radiologist.
Figure 8.
Figure 8.
Average IoU scores based on training model.
See this image and copyright information in PMC

References

    1. Aydoğan A, Türk Havacılık AD, Faaliyetlerinin Gelişim FVH. Süreci. Uluslararası Tarih Araştırmaları Derg. 2020;4(2):69 81.
    1. TEKNOFEST Official Website; 2021. Available at: https://teknofest.org/tr/.
    1. Kirshner H, Schrag M. Management of intracerebral hemorrhage: update and future therapies. Curr Neurol Neurosci Rep. 2021;21(10):57. 10.1007/s11910-021-01144-9) - DOI - PubMed
    1. Brott T, Bogousslavsky J. Treatment of acute ischemic stroke. N Engl J Med. 2000;343(10):710 722. 10.1056/NEJM200009073431007) - DOI - PubMed
    1. Dyer T, Chawda S, Alkilani R, Morgan TN, Hughes M, Rasalingham S. Validation of an artificial intelligence solution for acute triage and rule-out normal of non-contrast CT head scans. Neuroradiology. 2022;64(4):735 743. 10.1007/s00234-021-02826-4) - DOI - PubMed

LinkOut - more resources