Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Feb 11;6(2):e32443.
doi: 10.2196/32443.

Assessing COVID-19 Health Information on Google Using the Quality Evaluation Scoring Tool (QUEST): Cross-sectional and Readability Analysis

Vismaya S Bachu  1 , Heba Mahjoub  1 , Albert E Holler  1 , Tudor Crihalmeanu  2 , Dheevena M Bachu  3 , Varun Ayyaswami  4 , Pearman D Parker  5 , Arpan V Prabhu  6
Affiliations

Assessing COVID-19 Health Information on Google Using the Quality Evaluation Scoring Tool (QUEST): Cross-sectional and Readability Analysis

Vismaya S Bachu et al. JMIR Form Res. .

Abstract

Background: The COVID-19 pandemic spurred an increase in online information regarding disease spread and symptomatology.

Objective: Our purpose is to systematically assess the quality and readability of articles resulting from frequently Google-searched COVID-19 terms in the United States.

Methods: We used Google Trends to determine the 25 most commonly searched health-related phrases between February 29 and April 30, 2020. The first 30 search results for each term were collected, and articles were analyzed using the Quality Evaluation Scoring Tool (QUEST). Three raters scored each article in authorship, attribution, conflict of interest, currency, complementarity, and tone. A readability analysis was conducted.

Results: Exactly 709 articles were screened, and 195 fulfilled inclusion criteria. The mean article score was 18.4 (SD 2.6) of 28, with 7% (14/189) scoring in the top quartile. National news outlets published the largest share (70/189, 36%) of articles. Peer-reviewed journals attained the highest average QUEST score compared to national/regional news outlets, national/state government sites, and global health organizations (all P<.05). The average reading level was 11.7 (SD 1.9, range 5.4-16.9). Only 3 (1.6%) articles were written at the recommended sixth grade level.

Conclusions: COVID-19-related articles are vastly varied in their attributes and levels of bias, and would benefit from revisions for increased readability.

Keywords: COVID-19; COVID-19 pandemic; QUEST; United States; cross-sectional; health literacy; internet; online health information; quality; readability; spread; symptom; trend.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Description and examples of basic inclusion (article) vs exclusion (nonarticle) criteria. Article, Example 1: “Gastrointestinal Symptoms Common in COVID-19 Patients, Stanford Medicine Study Reports.” Article, Example 2: “What to Know about Coronaviruses.” Nonarticle, Example 1: “COVID-19 in San Diego County.” Nonarticle, Example 2: “Covid-19 Coronavirus Pandemic."
Figure 2
Figure 2
(1) Google search words/phrases used to collect most viewed articles. (2) Increase in Google search popularity of the five most commonly searched health-related phrases in the United States between February 29 to April 30, 2020.
Figure 3
Figure 3
CONSORT (Consolidated Standards of Reporting Trials) diagram depicting article flow and eventual sample size. QUEST: Quality Evaluation Scoring Tool.
Figure 4
Figure 4
QUEST score by categorization of 195 articles into the article type. Peer-reviewed journal (PRJ; mean 22.7, SD 2.20); entertainment or cultural outlet (mean 20.6, SD 1.95); online encyclopedia (mean 20.3, SD not applicable); private health-focused entity (mean 18.9, SD 3.17); private entity (mean 18.9, SD 3.17); national news outlet (NNO; mean 18.3, SD 2.2); regional news outlet (RNO; mean 17.6, SD 2.24); national government site (NGS; mean 17.5, SD 1.81); state government site (SGS; mean 17.4, SD 1.65); global health organization (GHO; mean 17.2, SD 1.18). PRJ quality score was significantly higher than NNO (P=.002), RNO (P=.003), NGS (P=.001), SGS (P<.001), and GHO (P=.046) scores. QUEST: Quality Evaluation Scoring Tool.
Figure 5
Figure 5
QUEST score stratified by search order product (R=–0.16; P>.05); there is no hierarchy in score based on the order of search results. QUEST: Quality Evaluation Scoring Tool.
Figure 6
Figure 6
Number of articles (n=187) stratified by educational stages; 187 COVID-19–related articles’ readability score breakdown: middle school, <9th grade; some high school, 9th to 11th grade; high school degree, 12th grade; some college, 13th to 15th grade; college degree or higher, ≥16th grade.
See this image and copyright information in PMC

References

    1. Hydroxychloroquine has about 90 percent chance of helping COVID-19 patients. Association of American Physicians and Surgeons. [2020-09-06]. https://aapsonline.org/hcq-90-percent-chance/
    1. Boulware DR, Pullen MF, Bangdiwala AS, Pastick KA, Lofgren SM, Okafor EC, Skipper CP, Nascene AA, Nicol MR, Abassi M, Engen NW, Cheng MP, LaBar D, Lother SA, MacKenzie LJ, Drobot G, Marten N, Zarychanski R, Kelly LE, Schwartz IS, McDonald EG, Rajasingham R, Lee TC, Hullsiek KH. A randomized trial of hydroxychloroquine as postexposure prophylaxis for Covid-19. N Engl J Med. 2020 Aug 06;383(6):517–525. doi: 10.1056/NEJMoa2016638. http://europepmc.org/abstract/MED/32492293 - DOI - PMC - PubMed
    1. Skipper CP, Pastick KA, Engen NW, Bangdiwala AS, Abassi M, Lofgren SM, Williams DA, Okafor EC, Pullen MF, Nicol MR, Nascene AA, Hullsiek KH, Cheng MP, Luke D, Lother SA, MacKenzie LJ, Drobot G, Kelly LE, Schwartz IS, Zarychanski R, McDonald EG, Lee TC, Rajasingham R, Boulware DR. Hydroxychloroquine in nonhospitalized adults with early COVID-19: a randomized trial. Ann Intern Med. 2020 Oct 20;173(8):623–631. doi: 10.7326/M20-4207. https://www.acpjournals.org/doi/abs/10.7326/M20-4207?url_ver=Z39.88-2003... - DOI - PMC - PubMed
    1. Ruiz JB, Bell RA. Understanding vaccination resistance: vaccine search term selection bias and the valence of retrieved information. Vaccine. 2014 Oct 07;32(44):5776–80. doi: 10.1016/j.vaccine.2014.08.042.S0264-410X(14)01165-7 - DOI - PubMed
    1. Farooq A, Laato S, Islam AKMN. Impact of online information on self-isolation intention during the COVID-19 pandemic: cross-sectional study. J Med Internet Res. 2020 May 06;22(5):e19128. doi: 10.2196/19128. https://www.jmir.org/2020/5/e19128/ v22i5e19128 - DOI - PMC - PubMed