Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Oct 20;18(1):801.
doi: 10.1186/s12913-018-3566-z.

Harvesting the wisdom of the crowd: using online ratings to explore care experiences in regions

Roy J P Hendrikx  1   2 , Marieke D Spreeuwenberg  3   4 , Hanneke W Drewes  5 , Jeroen N Struijs  5   6 , Dirk Ruwaard  4 , Caroline A Baan  7   5
Affiliations

Harvesting the wisdom of the crowd: using online ratings to explore care experiences in regions

Roy J P Hendrikx et al. BMC Health Serv Res. .

Abstract

Background: Regional population health management (PHM) initiatives need an understanding of regional patient experiences to improve their services. Websites that gather patient ratings have become common and could be a helpful tool in this effort. Therefore, this study explores whether unsolicited online ratings can provide insight into (differences in) patient's experiences at a (regional) population level.

Methods: Unsolicited online ratings from the Dutch website Zorgkaart Nederland (year = 2008-2017) were used. Patients rated their care providers on six dimensions from 1 to 10 and these ratings were geographically aggregated based on nine PHM regions. Distributions were explored between regions. Multilevel analyses per provider category, which produced Intraclass Correlation Coefficients (ICC), were performed to determine clustering of ratings of providers located within regions. If ratings were clustered, then this would indicate that differences found between regions could be attributed to regional characteristics (e.g. demographics or regional policy).

Results: In the nine regions, 70,889 ratings covering 4100 care providers were available. Overall, average regional scores (range = 8.3-8.6) showed significant albeit small differences. Multilevel analyses indicated little clustering between unsolicited provider ratings within regions, as the regional level ICCs were low (ICC pioneer site < 0.01). At the provider level, all ICCs were above 0.11, which showed that ratings were clustered.

Conclusions: Unsolicited online provider-based ratings are able to discern (small) differences between regions, similar to solicited data. However, these differences could not be attributed to the regional level, making unsolicited ratings not useful for overall regional policy evaluations. At the provider level, ratings can be used by regions to identify under-performing providers within their regions.

Keywords: Online ratings; Population health management; Quality of care; Regional evaluation; Unsolicited data.

PubMed Disclaimer

Conflict of interest statement

Ethics approval and consent to participate

The Medical Research Involving Human Subjects Act (WMO) does not apply to this study, and official approval was not required [34]. Participants agreed to the terms of service of Zorgkaart Nederland, which states that their submission can be used anonymously for research purposes [35].

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interest.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig. 1
Fig. 1
Comparison of mean ratings per dimension and overall per pioneer site with confidence intervals (Range 8–9)
Fig. 2
Fig. 2
Comparison of mean dental care ratings between pioneer sites (GZGR = Gezonde Zorg, Gezonde Regio; SmZ = Slimmer met Zorg; SSiZ = Samen Sterk in Zorg)
Fig. 3
Fig. 3
Comparison of mean hospital care ratings between pioneer sites (GZGR = Gezonde Zorg, Gezonde Regio; SmZ = Slimmer met Zorg; SSiZ = Samen Sterk in Zorg)
Fig. 4
Fig. 4
Comparison of mean general practitioner’s ratings between pioneer sites (GZGR = Gezonde Zorg, Gezonde Regio; SmZ = Slimmer met Zorg; SSiZ = Samen Sterk in Zorg)
Fig. 5
Fig. 5
Comparison of mean nursing home ratings between pioneer sites (GZGR = Gezonde Zorg, Gezonde Regio; SmZ = Slimmer met Zorg; SSiZ = Samen Sterk in Zorg)
See this image and copyright information in PMC

References

    1. Corrigan JM, Fisher ES. Accountable health communities: insights from state health reform initiatives. In. The Dartmouth Institute for Health Policy & Clinical Practice: Lebanon; 2014.
    1. Alderwick H, Ham C, Buck D. Population health systems: Going beyond integrated care. 2015. http://www.kingsfund.org.uk/sites/files/kf/field/field_publication_file/....
    1. Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff. 2008;27(3):759–769. doi: 10.1377/hlthaff.27.3.759. - DOI - PubMed
    1. McNally D, Sharples MA. Improving experience of care through people who use services. In. London: NHS England; 2015.
    1. Agency for Healthcare Research and Quality. Section 2. Why Improve Patient Experience? 2016. https://www.ahrq.gov/cahps/quality-improvement/improvement-guide/2-why-i.... Accessed 13 Oct 2017.