Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Dec;27(1):2053401.
doi: 10.1080/10872981.2022.2053401.

Do entrustment scales make a difference in the inter-rater reliability of the workplace-based assessment?

Ayat Nabil Eltayar  1 , Soha Rashed Aref  2 , Hoda Mahmoud Khalifa  3 , Abdullah Said Hammad  4
Affiliations

Do entrustment scales make a difference in the inter-rater reliability of the workplace-based assessment?

Ayat Nabil Eltayar et al. Med Educ Online. 2022 Dec.

Abstract

Background: A workplace-based assessment (WBA) is used to assess learners' competencies in their workplaces. Many workplace assessment tools are available and validated to assess various constructs. The implementation of workplace-based assessment requires proper training of the staff.

Objective: This study aimed to explore the impact of staff training on WBA practices and evaluate the inter-rater reliability of these practices while using entrustment scales, performance descriptors, and personal judgment.

Design: A quasi-experimental study, in which the staff members of the orthopedic department were invited to participate in a training program on the use of entrustment scales and assessment descriptors within the WBA tools. As a response to the training, subjective judgment was replaced by entrustment scales and performance descriptors in a trauma course offered by the orthopedic department. The inter-rater reliability of the WBA was evaluated using various rating scales.

Results: The entrustment scales had higher inter-rater reliability of the assessment tools than performance descriptors and the personal judgment.

Conclusion: The inter-rater reliability was highest when using entrustment scales for WBAs, which could indicate that the entrustment scales achieve good psychometric properties as regards consistency among different raters. Thus, they decrease the confounding effect of differences in assessors. They may also give a clearer image of the actual academic level of the learners.

Keywords: Entrustment scales; inter-rater reliability; performance descriptors; workplace-based assessment.

PubMed Disclaimer

Conflict of interest statement

No potential conflict of interest was reported by the author(s).

References

    1. Yousuf Guraya S. Workplace-based assessment; applications and educational impact. Malays J Med Sci. 2015;22(6):5–6. - PMC - PubMed
    1. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9):S63–7. doi:10.1097/0000188-199009000-00045. - DOI - PubMed
    1. Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide no. 31. Med Teach. 2007;29(9–10):855–871. - PubMed
    1. Ten Cate O. Trust, competence, and the supervisor’s role in postgraduate training. BMJ. 2006;333(7571):748–751. - PMC - PubMed
    1. Haneline M, Cooperstein R. Weighing the reliability and validity of clinical tests. J Chiropr Med. 2006;43:19–22.