Automated detection of critical results in radiology reports

Abstract

The goal of this study was to develop and validate text-mining algorithms to automatically identify radiology reports containing critical results including tension or increasing/new large pneumothorax, acute pulmonary embolism, acute cholecystitis, acute appendicitis, ectopic pregnancy, scrotal torsion, unexplained free intraperitoneal air, new or increasing intracranial hemorrhage, and malpositioned tubes and lines. The algorithms were developed using rule-based approaches and designed to search for common words and phrases in radiology reports that indicate critical results. Certain text-mining features were utilized such as wildcards, stemming, negation detection, proximity matching, and expanded searches with applicable synonyms. To further improve accuracy, the algorithms utilized modality and exam-specific queries, searched under the "Impression" field of the radiology report, and excluded reports with a low level of diagnostic certainty. Algorithm accuracy was determined using precision, recall, and F-measure using human review as the reference standard. The overall accuracy (F-measure) of the algorithms ranged from 81% to 100%, with a mean precision and recall of 96% and 91%, respectively. These algorithms can be applied to radiology report databases for quality assurance and accreditation, integrated with existing dashboards for display and monitoring, and ported to other institutions for their own use.

Fig. 1

This is a general schema of the critical results algorithms consisting of the concepts outlined above

Fig. 2

Impression parser: The “Impression” field of every report in the database was parsed using a PHP script based on regular expressions. The script was able to parse the “Impression” section no matter its location in the report. The parser could also handle synonyms and misspellings of “Impression”

Fig. 3

Explanation: The negative modifier “no” is within close proximity to “hematoma.” The algorithm is currently designed to exclude these reports. However, in this example, “no” is used to negate “midline shift” and not “hematoma.” Future efforts are underway to remedy this problem using sentence-specific searches

Fig. 4

This is a general schema of the algorithm for acute appendicitis. The algorithm consisted of modality and exam specific searches under the “Impression” field of the radiology report. A negation detection algorithm was employed. Reports with low diagnostic certainty and with certain pertinent negatives were also excluded. (1) For example, the algorithm excluded radiology reports containing phrases such as “No peri-appendicular inflammatory stranding.” (2) The algorithm excluded reports such as “The appendix is not seen and therefore appendicitis cannot be excluded”

Fig. 5

This is a partial generalized schema of the algorithm for Unexplained Free Intraperitoneal Air. The entire algorithm had over 20 general and 100 specific rules. (1) Negation distance varied depending on the context and the negation term in question. (2) Certain statements containing negation terms were permitted such as “the pneumoperitoneum that was not clearly seen on yesterday’s study has increased in size”