Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Oct 5;23(10):e30701.
doi: 10.2196/30701.

Improvement and Evaluation of the TOPCOP Taxonomy of Patient Portals: Taxonomy-Evaluation-Delphi (TED) Approach

Michael Glöggler  1 , Elske Ammenwerth  1
Affiliations

Improvement and Evaluation of the TOPCOP Taxonomy of Patient Portals: Taxonomy-Evaluation-Delphi (TED) Approach

Michael Glöggler et al. J Med Internet Res. .

Abstract

Background: Patient portals have been introduced in many countries over the last 10 years, but many health information managers still feel they have too little knowledge of patient portals. A taxonomy can help them to better compare and select portals. This has led us to develop the TOPCOP taxonomy for classifying and comparing patient portals. However, the taxonomy has not been evaluated by users.

Objective: This study aimed to evaluate the taxonomy's usefulness to support health information managers in comparing, classifying, defining a requirement profile for, and selecting patient portals and to improve the taxonomy where needed.

Methods: We used a modified Delphi approach. We sampled a heterogeneous panel of 13 health information managers from 3 countries using the criterion sampling strategy. We conducted 4 anonymous survey rounds with qualitative and quantitative questions. In round 1, the panelists assessed the appropriateness of each dimension, and we collected new ideas to improve the dimensions. In rounds 2 and 3, the panelists iteratively evaluated the taxonomy that was revised based on round 1. In round 4, the panelists assessed the need for a taxonomy and the appropriateness of patient engagement as a distinguishing concept. Then, they compared 2 real portals with the final taxonomy and evaluated its usefulness for comparing portals, creating an initial requirement profile, and selecting patient portals. To determine group consensus, we applied the RAND/UCLA Appropriateness Method.

Results: The final taxonomy consists of 25 dimensions with 65 characteristics. Five new dimensions were added to the original taxonomy, with 8 characteristics added to already existing dimensions. Group consensus was achieved on the need for such a taxonomy to compare portals, on patient engagement as an appropriate distinguishing concept, and on the comprehensibility of the taxonomy's form. Further, consensus was achieved on the taxonomy's usefulness for classifying and comparing portals, assisting users in better understanding portals, creating a requirement profile, and selecting portals. This allowed us to test the usefulness of the final taxonomy with the intended users.

Conclusions: The TOPCOP taxonomy aims to support health information managers in comparing and selecting patient portals. By providing a standardized terminology to describe various aspects of patient portals independent of clinical setting or country, the taxonomy will also be useful for advancing research and evaluation of patient portals.

Keywords: Delphi study; EHR portal; classification system; digital health; electronic health records; evaluation; health information; information management; online EHR access; patient portal; taxonomy; user perspectives.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
The TOPCOP taxonomy of patient portals [32]: A patient portal is described regarding 7 aspects that cover 20 dimensions. A patient portal can be described by selecting 1 characteristic per dimension.
Figure 2
Figure 2
Example for the display of a single dimension for rating.
Figure 3
Figure 3
Example of a panelist’s rating in comparison with the group ratings.
Figure 4
Figure 4
Achieved consensus for the existing dimensions of the TOPCOP taxonomy after round 1. The grey column shows the median value without disagreement. All 13 panelists assessed all dimensions.
Figure 5
Figure 5
Proposals from round 1 for new characteristics and dimensions to improve the taxonomy.
Figure 6
Figure 6
Achieved consensus by all 13 panelists on new characteristics for existing dimensions proposed in round 1. The grey column shows the median without disagreement. *New characteristics to improve the dimension.
Figure 7
Figure 7
Achieved consensus by all 13 panelists on new dimensions proposed in round 1.
Figure 8
Figure 8
The TOPCOP taxonomy in progress after round 2.
Figure 9
Figure 9
Achieved consensus by all 13 panelists on dimensions D21 and D26 after round 3 in comparison with the consensus in round 2.
Figure 10
Figure 10
The final and user-evaluated TOPCOP taxonomy of patient portals.
Figure 11
Figure 11
Achieved consensus by all 13 panelists on the research questions Q1, Q2, Q4–Q9. The grey column shows the median without disagreement. p: patient.
Figure 12
Figure 12
Example of a comparison of 2 real patient portals carried out with the TOPCOP taxonomy by marking the respective characteristics.
See this image and copyright information in PMC

References

    1. Beal Lauren L, Kolman Jacob M, Jones Stephen L, Khleif Aroub, Menser Terri. Quantifying Patient Portal Use: Systematic Review of Utilization Metrics. J Med Internet Res. 2021 Feb 25;23(2):e23493. doi: 10.2196/23493. https://www.jmir.org/2021/2/e23493/ v23i2e23493 - DOI - PMC - PubMed
    1. Nøst Torunn Hatlen, Faxvaag Arild, Steinsbekk Aslak. Participants' views and experiences from setting up a shared patient portal for primary and specialist health services- a qualitative study. BMC Health Serv Res. 2021 Feb 24;21(171):2–9. doi: 10.1186/s12913-021-06188-8. https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-021-0... 10.1186/s12913-021-06188-8 - DOI - DOI - PMC - PubMed
    1. Nazi KM, Turvey CL, Klein DM, Hogan TP. A Decade of Veteran Voices: Examining Patient Portal Enhancements Through the Lens of User-Centered Design. J Med Internet Res. 2018 Jul 10;20(7):e10413. doi: 10.2196/10413. https://www.jmir.org/2018/7/e10413/ v20i7e10413 - DOI - PMC - PubMed
    1. Wiljer D, Urowitz S, Apatu E, DeLenardo C, Eysenbach G, Harth T, Pai H, Leonard KJ. Patient accessible electronic health records: exploring recommendations for successful implementation strategies. J Med Internet Res. 2008 Oct 31;10(4):e34. doi: 10.2196/jmir.1061. https://www.jmir.org/2008/4/e34/ v10i4e34 - DOI - PMC - PubMed
    1. Vreugdenhil MMT, Ranke S, de Man Y, Haan MM, Kool RB. Patient and Health Care Provider Experiences With a Recently Introduced Patient Portal in an Academic Hospital in the Netherlands: Mixed Methods Study. J Med Internet Res. 2019 Aug 20;21(8):13743. doi: 10.2196/13743. https://www.jmir.org/2019/8/13743/ v21i8e13743 - DOI - PMC - PubMed

LinkOut - more resources