Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2025 Mar;52(1):45-55.
doi: 10.1177/14653125241268763. Epub 2024 Aug 16.

Orthodontic treatment protocol versus Peer Assessment Rating: Assessing the quality of orthodontic treatment

Jonathan D Shelswell  1 , Brian M Kelly  2 , Trevor M Hodge  3 , Sophy K Barber  1
Affiliations
Comparative Study

Orthodontic treatment protocol versus Peer Assessment Rating: Assessing the quality of orthodontic treatment

Jonathan D Shelswell et al. J Orthod. 2025 Mar.

Abstract

Objective: To apply the Peer Assessment Rating (PAR) to cases that have been assessed by the NHS Business Service Authority (NHSBSA) using the orthodontic treatment protocol (OTO), then compare the NHSBSA outcome assessment with weighted (W) and unweighted (U) PAR scores.

Design: Cross-sectional study.

Setting: UK.

Cases: Anonymised orthodontic cases submitted to the NHSBSA.

Methods: A sample of 30 reports from 2021/2022 were randomly selected to include different standard of treatment grades. The records were de-identified and the pre- and post-treatment study models were PAR scored by a calibrated assessor.

Results: The mean percentage change in PAR was higher in cases from green reports (W: 78%; U: 79%) than amber (W: 68%; U: 67%) and red reports (W: 65%; U: 65%). Alignment and poor buccal segment interdigitation were the most reported concerns for cases included in the red and amber graded reports. A residual increased overjet was the most common occlusal feature leading to PAR scores not being more than 70% improved. Only slight agreement was shown between OTP and PAR using the kappa statistic, and the chi-square statistical test found that outcome measures are statistically significantly different.

Conclusion: There are fundamental differences between OTP and PAR, and general agreement between them has not been demonstrated. The NHSBSA Report provides a more critical outcome assessment than PAR, identifying elements that are not assessed or measured by the PAR index.

Keywords: economics of orthodontic delivery systems; epidemiology in orthodontics (including occlusal indices); health services and quality of life aspects; risk/benefit assessment.

PubMed Disclaimer

Conflict of interest statement

Declaration of conflicting interestsThe authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: BK is employed by the NHSBSA and is subject to NHSBSA reporting as part of their clinical practice.

Figures

Figure 1.
Figure 1.
Summary of data collection, extraction and analysis.
Figure 2.
Figure 2.
Bland–Altman plot to determine intra-rater agreement.
Figure 3.
Figure 3.
The frequency that components of the PAR index contribute to a greater than 20% reduction in PAR score. PAR, Peer Assessment Rating.
Figure 4.
Figure 4.
The frequency that components of the PAR index contributed to a high post-treatment PAR score (greater than 8).
See this image and copyright information in PMC

References

    1. Brook PH, Shaw WC. (1989) The development of an index of orthodontic treatment priority. The European Journal of Orthodontics 11: 309–320. - PubMed
    1. Daniels C, Richmond S. (2000) The development of the index of complexity, outcome and need (ICON). Journal of Orthodontics 27: 149–162. - PubMed
    1. DeGuzman L, Bahiraei D, Vig K, et al.. (1995) The validation of the peer assessment rating index for malocclusion severity and treatment difficulty. American Journal of Orthodontics and Dentofacial Orthopedics 107: 172–176. - PubMed
    1. Department of Health (2005) Primary care dental services: Commissioning specialist dental services. Available at: https://webarchive.nationalarchives.gov.uk/ukgwa/20130123205317/http://w... (accessed 29 November 2023).
    1. Egress Software Technologies Ltd. (2023) Egress secure workspace (NHS Business Services Authority). Available at: https://switch.egress.com/ui/signin.aspx?rt=https%3A%2F%2Fnhsbsa.egressc... (accessed 6 December 2023).

Publication types