Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jul 1;25(1):868.
doi: 10.1186/s12913-025-12733-6.

Development of key performance indicators for a telemedicine setting in Egypt using an electronic modified Delphi approach

Sara Ebraheem  1 , Ayat Farouk Manzour  2 , Ahmed Elbokl  3   4 , Tamer Emara  3   4 , Moustafa El Houssinie  2 , Mahi Mahmoud Al-Tehewy  5   6
Affiliations

Development of key performance indicators for a telemedicine setting in Egypt using an electronic modified Delphi approach

Sara Ebraheem et al. BMC Health Serv Res. .

Abstract

Background: Performance measurement is crucial for maintaining quality of care in health systems, and so telemedicine is no exception. Key Performance Indicators (KPIs) are some of the primary tools for performance measurement. While there are many global studies on telemedicine quality, studies that are done specifically on indicators are limited. Based on this, the purpose of this study is to develop KPIs for evaluating the performance of telemedicine settings.

Methods: A consensus-based methodological study employing the Delphi method was conducted. The study was conducted in Ain Shams Virtual Hospital (AVH). It started with defining the selection criteria to be used in Delphi method. Then a review of scientific sources was carried out to enlist the potential indicators of telemedicine. An electronic modified Delphi technique was then used. The panel members were invited to rate the indicators on a 10-point Likert scale, and subsequently, the experts then reconsidered their previous voting in the light of the aggregated results. The agreement was reached as the indicator showed a 70% level of agreement. Cohen Kappa (K) was calculated to show the percentage of agreement between the two rounds.

Results: The multi-voting process established three selection criteria for indicators: clarity, feasibility, and importance. A review of research sources led to an initial list of 52 potential indicators, categorized into operations, clinical services, and customer satisfaction. The Delphi method was carried out in two rounds, with expert participation rates of 35% in the first round, and 80% in the second. In the end, 31 indicators achieved the 70% agreement level, with the majority falling under the operations category, while all customer satisfaction indicators were selected. Cohen's Kappa analysis showed that nearly 85% of comparisons between the two rounds demonstrated substantial agreement.

Conclusions: This study identifies a thorough collection of indicators that offers a valuable tool for enhancing quality in telemedicine settings. The indicators cover important domains in telemedicine services, such as: access, utilization, system efficiency, and customer satisfaction. Future directions should include piloting these KPIs in telemedicine systems and assessing the acceptability of their application.

Keywords: Delphi method; Key performance indicators; Quality indicators; Telemedicine.

PubMed Disclaimer

Conflict of interest statement

Declarations. Ethics approval and consent to participate: Ethical approval number (FMASU MD 96/20221) was obtained from the Institutional Research Board (IRB) in Ain Shams Faculty of Medicine called “Research Ethics Committee REC” FWA 000017585. Administrative approval came from hospital management. Confidentiality was maintained. Informed consent by email was obtained from the Delphi experts. Published data were handled anonymously and used only for research. This study adhered to the Declaration of Helsinki. Consent for publication: Not applicable. Competing interests: The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Flow of participants through the Delphi method
Fig. 2
Fig. 2
Numbers of KPIs in different phases in the study
Fig. 3
Fig. 3
Flowchart of KPIs through the study
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Sood S, Mbarika V, Jugoo S, Dookhy R, Doarn CR, Prakash N, et al. What is telemedicine? A collection of 104 peer-reviewed perspectives and theoretical underpinnings. Telemed e-Health. 2007;13(5):573–90. - PubMed
    1. Uscher-Pines L, Mulcahy A, Cowling D, Hunter G, Burns R, Mehrotra A. Access and quality of care in direct-to-consumer telemedicine. Telemed e-Health. 2016;22(4):282–7. - PMC - PubMed
    1. Cassar MR, Borg D, Camilleri L, Schembri A, Anastasi EA, Buhagiar K, et al. A novel use of telemedicine during the COVID-19 pandemic. Int J Infect Dis. 2021;103:182–7. - PMC - PubMed
    1. Mahajan V, Singh T, Azad C. Using telemedicine during the COVID-19 pandemic. Indian Pediatr. 2020;57(7):658–61. - PubMed
    1. Kobeissi MM, Ruppert SD. Remote patient triage: shifting toward safer telehealth practice. J Am Assoc Nurse Pract. 2022;34(3):444. - PMC - PubMed

LinkOut - more resources