Developing Key Performance Indicators for Emergency Department of Teaching Hospitals: A Mixed Fuzzy Delphi and Nominal Group Technique Approach

doi:10.21315/mjms2022.29.2.11

. 2022 Apr;29(2):114-125.

doi: 10.21315/mjms2022.29.2.11. Epub 2022 Apr 21.

Developing Key Performance Indicators for Emergency Department of Teaching Hospitals: A Mixed Fuzzy Delphi and Nominal Group Technique Approach

Rahman Nik Hisamuddin^{1

2}, Tuan Kamauzaman Tuan Hairulnizam^{1

2}

Affiliations

¹ Department of Emergency Medicine, School of Medical Sciences, Universiti Sains Malaysia, Kelantan, Malaysia.
² Hospital Universiti Sains Malaysia, Kelantan, Malaysia.

PMID: 35528822
PMCID: PMC9036937
DOI: 10.21315/mjms2022.29.2.11

Developing Key Performance Indicators for Emergency Department of Teaching Hospitals: A Mixed Fuzzy Delphi and Nominal Group Technique Approach

Rahman Nik Hisamuddin et al. Malays J Med Sci. 2022 Apr.

. 2022 Apr;29(2):114-125.

doi: 10.21315/mjms2022.29.2.11. Epub 2022 Apr 21.

Authors

Rahman Nik Hisamuddin^{1

2}, Tuan Kamauzaman Tuan Hairulnizam^{1

2}

Affiliations

¹ Department of Emergency Medicine, School of Medical Sciences, Universiti Sains Malaysia, Kelantan, Malaysia.
² Hospital Universiti Sains Malaysia, Kelantan, Malaysia.

PMID: 35528822
PMCID: PMC9036937
DOI: 10.21315/mjms2022.29.2.11

Abstract

Background: This article demonstrates combination of the fuzzy Delphi method (FDM) and the nominal group technique (NGT) to consolidate consensus agreement within a panel of experts regarding key performance indicators (KPIs) development for emergency department (ED).

Methods: Twenty-four participants for NGT and 10 experts for FDM were randomly chosen from the emergency medicine (EM) department staff list obtained from the human resource department of study centres. A set of item constructs related to KPIs was developed from the NGT session and used for the FDM session in the second phase of the study.

Results: We found that 16 out of 22 and 11 out of 15 items satisfied the first prerequisite 'd' value ≤ 0.2. Ten items (45%) from the service KPI domain and six items (40%) from the academic KPI domain had expert consensus of more than 75%. A total of 16 out of 22 (73%) and 14 out of 15 (93%) fit the criteria of an average fuzzy number (A value) of more than 0.5. Fifteen items that fulfilled the prerequisites were retained for the final KPI draft.

Conclusion: The FDM and NGT analyses reached experts' consensus on the suitability of the pre-selected items in the ED KPIs. The development of the KPI framework is expected to enhance future improvement of EM services and academic activities in all teaching hospitals in the country.

Keywords: academic; benchmarking; emergency department; key performance indicators; quality; teaching hospital.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest None.

Figures

**Figure 2**
Formula for determining the distance between two fuzzy numbers

**Figure 1**
The triangular fuzzy numbers

**Figure 3**
Summary of content validation using the FDM

See this image and copyright information in PMC

Cited by

Identifying relevant topics and their competency levels for dental materials science: a fuzzy Delphi study.
Lin GSS, Pow KJ, Yahya NA, Foong CC, Noorani TY. Lin GSS, et al. BMC Oral Health. 2023 Apr 27;23(1):243. doi: 10.1186/s12903-023-02946-8. BMC Oral Health. 2023. PMID: 37106354 Free PMC article.
Hospital performance evaluation indicators: a scoping review.
Hadian SA, Rezayatmand R, Shaarbafchizadeh N, Ketabi S, Pourghaderi AR. Hadian SA, et al. BMC Health Serv Res. 2024 May 1;24(1):561. doi: 10.1186/s12913-024-10940-1. BMC Health Serv Res. 2024. PMID: 38693562 Free PMC article.
Identifying High-Priority Ethical Challenges for Precision Emergency Medicine: Nominal Group Study.
Rose C, Shearer E, Woller I, Foster A, Ashenburg N, Kim I, Newberry J. Rose C, et al. JMIR Form Res. 2025 Feb 6;9:e68371. doi: 10.2196/68371. JMIR Form Res. 2025. PMID: 39916376 Free PMC article.
Outcome metrics in recommended performance measurement tools for emergency department performance evaluation: a systematic review.
Toukola TK, Mertanen RKK, Kaartinen JM, Tuunainen EA, Castrén MK, Torkki PM. Toukola TK, et al. Scand J Trauma Resusc Emerg Med. 2025 Jul 16;33(1):127. doi: 10.1186/s13049-025-01442-y. Scand J Trauma Resusc Emerg Med. 2025. PMID: 40671154 Free PMC article.
Expert consensus on relevant topics for undergraduate paediatric dental curriculum using the fuzzy Delphi method: a new direction for Malaysian dental education.
Lin GSS, Chin YJ, Chong RS, Baharin F, Syed Saadun Tarek Wafa SWW, Dziaruddin N. Lin GSS, et al. BMC Oral Health. 2023 Jul 5;23(1):452. doi: 10.1186/s12903-023-03130-8. BMC Oral Health. 2023. PMID: 37407955 Free PMC article.

References

1. Reznek MA, Michael SS, Harbertson CA, Scheulen JJ, Augustine JJ. Clinical operations of academic versus non-academic emergency departments: a descriptive comparison of two large emergency department operations surveys. BMC Emerg Med. 2019;19:72. doi: 10.1186/s12873-019-0285-7. - DOI - PMC - PubMed
1. Nowacki AK, Landes M, Azazh A, Ritchie LM. A review of published literature on emergency medicine training programs in low- and middle-income countries. Int J Emerg Med. 2013;6:26. doi: 10.1186/1865-1380-6-26. - DOI - PMC - PubMed
1. Pimentel L, Hirshon JM, Barrueto F, Jr, Browne BJ. Development of a university-based emergency department network: lessons learned. J Emerg Med. 2012;43(4):728–735. doi: 10.1016/j.jemermed.2011.05.055. - DOI - PubMed
1. Breen LM, Trepp R, Jr, Gavin N. Lean process improvement in the emergency department. Emerg Med Clin North Am. 2020;38(3):633–646. doi: 10.1016/j.emc.2020.05.001. - DOI - PubMed
1. Drynda S, Schindler W, Slagman A, Pollmanns J, Horenkamp-Sonntag D, Schirrmeister W, et al. Evaluation of outcome relevance of quality indicators in the emergency department (ENQuIRE): study protocol for a prospective multicentre cohort study. BMJ Open. 2020;10(9):e038776. doi: 10.1136/bmjopen-2020-038776. - DOI - PMC - PubMed

LinkOut - more resources

Full Text Sources
- Europe PubMed Central
- PubMed Central

[1] Reznek MA, Michael SS, Harbertson CA, Scheulen JJ, Augustine JJ. Clinical operations of academic versus non-academic emergency departments: a descriptive comparison of two large emergency department operations surveys. BMC Emerg Med. 2019;19:72. doi: 10.1186/s12873-019-0285-7. - DOI - PMC - PubMed

[2] Reznek MA, Michael SS, Harbertson CA, Scheulen JJ, Augustine JJ. Clinical operations of academic versus non-academic emergency departments: a descriptive comparison of two large emergency department operations surveys. BMC Emerg Med. 2019;19:72. doi: 10.1186/s12873-019-0285-7. - DOI - PMC - PubMed

[3] Nowacki AK, Landes M, Azazh A, Ritchie LM. A review of published literature on emergency medicine training programs in low- and middle-income countries. Int J Emerg Med. 2013;6:26. doi: 10.1186/1865-1380-6-26. - DOI - PMC - PubMed

[4] Nowacki AK, Landes M, Azazh A, Ritchie LM. A review of published literature on emergency medicine training programs in low- and middle-income countries. Int J Emerg Med. 2013;6:26. doi: 10.1186/1865-1380-6-26. - DOI - PMC - PubMed

[5] Pimentel L, Hirshon JM, Barrueto F, Jr, Browne BJ. Development of a university-based emergency department network: lessons learned. J Emerg Med. 2012;43(4):728–735. doi: 10.1016/j.jemermed.2011.05.055. - DOI - PubMed

[6] Pimentel L, Hirshon JM, Barrueto F, Jr, Browne BJ. Development of a university-based emergency department network: lessons learned. J Emerg Med. 2012;43(4):728–735. doi: 10.1016/j.jemermed.2011.05.055. - DOI - PubMed

[7] Breen LM, Trepp R, Jr, Gavin N. Lean process improvement in the emergency department. Emerg Med Clin North Am. 2020;38(3):633–646. doi: 10.1016/j.emc.2020.05.001. - DOI - PubMed

[8] Breen LM, Trepp R, Jr, Gavin N. Lean process improvement in the emergency department. Emerg Med Clin North Am. 2020;38(3):633–646. doi: 10.1016/j.emc.2020.05.001. - DOI - PubMed

[9] Drynda S, Schindler W, Slagman A, Pollmanns J, Horenkamp-Sonntag D, Schirrmeister W, et al. Evaluation of outcome relevance of quality indicators in the emergency department (ENQuIRE): study protocol for a prospective multicentre cohort study. BMJ Open. 2020;10(9):e038776. doi: 10.1136/bmjopen-2020-038776. - DOI - PMC - PubMed

[10] Drynda S, Schindler W, Slagman A, Pollmanns J, Horenkamp-Sonntag D, Schirrmeister W, et al. Evaluation of outcome relevance of quality indicators in the emergency department (ENQuIRE): study protocol for a prospective multicentre cohort study. BMJ Open. 2020;10(9):e038776. doi: 10.1136/bmjopen-2020-038776. - DOI - PMC - PubMed

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Developing Key Performance Indicators for Emergency Department of Teaching Hospitals: A Mixed Fuzzy Delphi and Nominal Group Technique Approach

Affiliations

Developing Key Performance Indicators for Emergency Department of Teaching Hospitals: A Mixed Fuzzy Delphi and Nominal Group Technique Approach

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

Similar articles

Cited by

References

LinkOut - more resources

Full Text Sources

Abstract

Conflict of interest statement

Figures

Similar articles

Cited by

References

Related information

LinkOut - more resources

Full Text Sources