Investigating the impact of structured reporting on the linguistic standardization of radiology reports through natural language processing over a 10-year period

Abstract

Objectives: To investigate how a transition from free text to structured reporting affects reporting language with regard to standardization and distinguishability.

Methods: A total of 747,393 radiology reports dictated between January 2011 and June 2020 were retrospectively analyzed. The body and cardiothoracic imaging divisions introduced a reporting concept using standardized language and structured reporting templates in January 2016. Reports were segmented by a natural language processing algorithm and converted into a 20-dimension document vector. For analysis, dimensionality was reduced to a 2D visualization with t-distributed stochastic neighbor embedding and matched with metadata. Linguistic standardization was assessed by comparing distinct report types' vector spreads (e.g., run-off MR angiography) between reporting standards. Changes in report type distinguishability (e.g., CT abdomen/pelvis vs. MR abdomen) were measured by comparing the distance between their centroids.

Results: Structured reports showed lower document vector spread (thus higher linguistic similarity) compared with free-text reports overall (21.9 [free-text] vs. 15.9 [structured]; - 27.4%; p < 0.001) and for most report types, e.g., run-off MR angiography (15.2 vs. 1.8; - 88.2%; p < 0.001) or double-rule-out CT (26.8 vs. 10.0; - 62.7%; p < 0.001). No changes were observed for reports continued to be written in free text, e.g., CT head reports (33.2 vs. 33.1; - 0.3%; p = 1). Distances between the report types' centroids increased with structured reporting (thus better linguistic distinguishability) overall (27.3 vs. 54.4; + 99.3 ± 98.4%) and for specific report types, e.g., CT abdomen/pelvis vs. MR abdomen (13.7 vs. 37.2; + 171.5%).

Conclusion: Structured reporting and the use of factual language yield more homogenous and standardized radiology reports on a linguistic level, tailored to specific reporting scenarios and imaging studies.

Clinical relevance: Information transmission to referring physicians, as well as automated report assessment and content extraction in big data analyses, may benefit from standardized reporting, due to consistent report organization and terminology used for pathologies and normal findings.

Key points: • Natural language processing and t-distributed stochastic neighbor embedding can transform radiology reports into numeric vectors, allowing the quantification of their linguistic standardization. • Structured reporting substantially increases reports' linguistic standardization (mean: - 27.4% in vector spread) and distinguishability (mean: + 99.3 ± 98.4% increase in vector distance) compared with free-text reports. • Higher standardization and homogeneity outline potential benefits of structured reporting for information transmission and big data analyses.

Keywords: Language; Linguistics; Radiology; Report; Standardization.

Fig. 1

Schematic drawings how report standardization (A) and distinguishability (B) were assessed. Linguistic standardization (A) is represented by the spread (= standard deviation) of distinct radiology report types (expressed by color coding) around their centroid (= mean) in vector space. Less spread equals higher document similarity (thus higher standardization of a distinct report type). Distinguishability (B) between distinct types of radiology reports (expressed by color coding) is represented by the distance between their centroids in vector space. A higher distance between two centroids equals lower document type similarity (thus better distinguishability of the two report types)

Fig. 2

Flowchart of the study sample

Fig. 3

Distribution of radiology reports in vector space before (A) and after (B) the introduction of structured reporting in body imaging. Distinguishability and clustering of distinct radiology report increased with structured templates compared with overlapping data points for free-text reporting. Neuroradiology report distribution remained unchanged between 2014 (C) and 2018 (D), given continued free-text format reporting

Fig. 4

Comparison of changes in vector spread between 2014 and 2019 for different levels of reporting. Level I structured reports represent templates with a structured layout; level II represents templates with a structured content [7]. SR = structured report

Fig. 5

Distribution of chest radiograph reports in vector space before (A) and after (B) the introduction of structured reporting templates in cardiothoracic imaging, demonstrating higher distinguishability and clustering when reported with dedicated structured reporting templates

Fig. 6

Distribution of radiology reports in vector space in 2018 (A) and 2019 (B) following the introduction of structured reporting templates. Centroid location in vector space and spread around the distinct report types’ centroids remain similar with structured reporting, indicating high reporting consistency