Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 May 1;35(9):1610-1612.
doi: 10.1093/bioinformatics/bty855.

CANDI: an R package and Shiny app for annotating radiographs and evaluating computer-aided diagnosis

Marcus A Badgeley  1   2   3 , Manway Liu  3 , Benjamin S Glicksberg  4 , Mark Shervey  1   2 , John Zech  5 , Khader Shameer  6 , Joseph Lehar  7 , Eric K Oermann  8 , Michael V McConnell  3   9 , Thomas M Snyder  3 , Joel T Dudley  1   2
Affiliations

CANDI: an R package and Shiny app for annotating radiographs and evaluating computer-aided diagnosis

Marcus A Badgeley et al. Bioinformatics. .

Abstract

Motivation: Radiologists have used algorithms for Computer-Aided Diagnosis (CAD) for decades. These algorithms use machine learning with engineered features, and there have been mixed findings on whether they improve radiologists' interpretations. Deep learning offers superior performance but requires more training data and has not been evaluated in joint algorithm-radiologist decision systems.

Results: We developed the Computer-Aided Note and Diagnosis Interface (CANDI) for collaboratively annotating radiographs and evaluating how algorithms alter human interpretation. The annotation app collects classification, segmentation, and image captioning training data, and the evaluation app randomizes the availability of CAD tools to facilitate clinical trials on radiologist enhancement.

Availability and implementation: Demonstrations and source code are hosted at (https://candi.nextgenhealthcare.org), and (https://github.com/mbadge/candi), respectively, under GPL-3 license.

Supplementary information: Supplementary material is available at Bioinformatics online.

PubMed Disclaimer

Figures

Fig. 1.
Fig. 1.
Annotation modalities and distinct uses. (A) The CANDI radiograph annotation (RAD) and computer-aided diagnosis (CAD) applications provide human-algorithm interfaces to generate training annotations and evaluate the subsequent models. Different annotation data modalities provide training data for distinct deep learning model utilities. We use convolutional neural networks (CNNs) to generate predictions in CANDI-CAD. (B) Various input/output systems are set up that conform to the security needs of different types of users
See this image and copyright information in PMC

References

    1. Badgeley M.A. et al. (2016). EHDViz: clinical dashboard development using open-source technologies. BMJ Open, 6(3). - PMC - PubMed
    1. Benedikt R.A. et al. (2017) Concurrent computer-aided detection improves reading time of digital breast tomosynthesis and maintains interpretation performance in a multireader multicase study. Am. J. Roentgenol., 1–10. - PubMed
    1. Cheplygina V. et al. (2017) Early experiences with crowdsourcing airway annotations in chest CT. arXiv [cs.CV]. http://arxiv.org/abs/1706.02055.
    1. Demner-Fushman D. et al. (2016) Preparing a collection of radiology examinations for distribution and retrieval. J. Am. Med. Inform. Assn., 23, 304–310. - PMC - PubMed
    1. Fenton J.J. et al. (2007) Influence of computer-aided detection on performance of screening mammography. N. Engl. J. Med., 356, 1399–1409. - PMC - PubMed

Publication types

LinkOut - more resources