Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Dec 16:6:1425040.
doi: 10.3389/fspor.2024.1425040. eCollection 2024.

Reliability agreement in foul and penalty judgements between officials in the Swedish hockey league

Glenn Björklund  1 , Olivia Procter  1 , Mikael Swarén  1   2
Affiliations

Reliability agreement in foul and penalty judgements between officials in the Swedish hockey league

Glenn Björklund et al. Front Sports Act Living. .

Abstract

Introduction: Officials are essential in terms of player safety and injury prevention, especially in contact team sports such as ice hockey, where numerous fast pace and high force contacts occur. If against the rules, these collisions can result in penalties. However, there is limited literature on the inter-rater reliability of the officials' decisions. Hence, the purpose was to investigate the theoretical reliability agreement between professional ice hockey officials in the Swedish Hockey League (SHL).

Method: Fifty video clips with different match situations were shown to 33 professional ice hockey officials in the SHL. Each situation was shown three times and the officials had 20 s between each video clip to answer which offence and penalty they would judge. The answers were anonymously collected using an online questionnaire. Fleiss' kappa was used to assess the reliability agreement between the referees, for each situation.

Results: The Fleiss' kappa values for all officials were 0.63 and 0.35 for offences and penalties, respectively. Referees and linesmen had similar kappa values for offences (0.64 vs. 0.64), as well as for penalties (0.38 vs. 0.35).

Conclusion: The results show that the suggested methodology can be used to identify situations where officials agree and disagree. In ice hockey, poor agreement regarding penalties can depend on the chosen offence as the rulebook limits the availability of penalties, based on the chosen offence. This can create issues, as there are situations where different offences are equally correct but will result in different penalties.

Keywords: Fleiss' kappa; SHL; ice hockey; inter-rater agreement; sports.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Figure 1
Figure 1
Calculated kappa values for offence and penalty for each video situation.
Figure 2
Figure 2
Key images from situation 13 where either checking from behind, boarding, charging or illegal check to the head can be considered correct.
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Raab M, Avugos S, Bar-Eli M, MacMahon C. The referee’s challenge: a threshold process model for decision making in sport games. Int Rev Sport Exerc Psychol. (2021) 14(1):208–28. 10.1080/1750984X.2020.1783696 - DOI
    1. Mascarenhas D. The psychology of officiating. Southland NZFU Referees’ Meeting; Southland, New Zealand (2006).
    1. Webb T. Referees and the media: a difficult relationship but an unavoidable necessity. Soccer & Society. (2018) 19(2):205–21. 10.1080/14660970.2015.1133414 - DOI
    1. Hancock DJ, Bennett S, Roaten H, Chapman K, Stanley C. An analysis of literature on sport officiating research. Res Q Exerc Sport. (2021) 92(4):607–17. 10.1080/02701367.2020.1756198 - DOI - PubMed
    1. Anderson KJ, Pierce DA. Officiating bias: the effect of foul differential on foul calls in NCAA basketball. J Sports Sci. (2009) 27(7):687–94. 10.1080/02640410902729733 - DOI - PubMed

LinkOut - more resources