Assessing the readability of ClinicalTrials.gov

Abstract

Objective: ClinicalTrials.gov serves critical functions of disseminating trial information to the public and helping the trials recruit participants. This study assessed the readability of trial descriptions at ClinicalTrials.gov using multiple quantitative measures.

Materials and methods: The analysis included all 165,988 trials registered at ClinicalTrials.gov as of April 30, 2014. To obtain benchmarks, the authors also analyzed 2 other medical corpora: (1) all 955 Health Topics articles from MedlinePlus and (2) a random sample of 100,000 clinician notes retrieved from an electronic health records system intended for conveying internal communication among medical professionals. The authors characterized each of the corpora using 4 surface metrics, and then applied 5 different scoring algorithms to assess their readability. The authors hypothesized that clinician notes would be most difficult to read, followed by trial descriptions and MedlinePlus Health Topics articles.

Results: Trial descriptions have the longest average sentence length (26.1 words) across all corpora; 65% of their words used are not covered by a basic medical English dictionary. In comparison, average sentence length of MedlinePlus Health Topics articles is 61% shorter, vocabulary size is 95% smaller, and dictionary coverage is 46% higher. All 5 scoring algorithms consistently rated CliniclTrials.gov trial descriptions the most difficult corpus to read, even harder than clinician notes. On average, it requires 18 years of education to properly understand these trial descriptions according to the results generated by the readability assessment algorithms.

Discussion and conclusion: Trial descriptions at CliniclTrials.gov are extremely difficult to read. Significant work is warranted to improve their readability in order to achieve CliniclTrials.gov's goal of facilitating information dissemination and subject recruitment.

Keywords: CliniclTrials.gov; clinical trial; comprehension; electronic health records; natural language processing; readability.

Figure 1:

A sample Purpose section from ClinicalTrials.gov.

Figure 2:

A sample MedlinePlus Health Topics article on “Aortic Aneurysm”.

Figure 3:

A sample clinical note from University of Michigan Health System.

Figure 4:

Distributions of average readability scores of the 3 study corpora (average scores were computed by pooling the results generated by the 4 conventional readability measures).

Figure 5:

Variability of average readability scores (average scores were computed by pooling the results generated by the 4 conventional readability measures).