Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jul 5.
doi: 10.1111/cyt.70006. Online ahead of print.

From the Daily Peer Review of Abnormal Pap Test Slides to the Monitoring of Individual and Laboratory Performances: 5 Years of Data Collection and New Potential (Key) Performance Indicators

Claudia Giachini  1 , Giampaolo Pompeo  1 , Donella Puliti  1 , Giuseppe Gorini  1 , Irene Paganini  1 , Ornella Cutaia  1 , Marco Meoni  1 , Stefania Cannistrà  1 , Marzia Matucci  1   2 , Francesca Carozzi  1   2 , Chiara Di Stefano  1 , Cristina Sani  1 ; Ispro Cytology Working Group
Collaborators, Affiliations

From the Daily Peer Review of Abnormal Pap Test Slides to the Monitoring of Individual and Laboratory Performances: 5 Years of Data Collection and New Potential (Key) Performance Indicators

Claudia Giachini et al. Cytopathology. .

Abstract

Objectives: The Peer Review (PR) consists of the daily examination, by all cytologists, of Pap slides that resulted abnormal/difficult, in order to reach a consensus on the final diagnosis (FD). We explore data from 5 years (2017-2021) of PR to: (i) evaluate the agreement (both inter-observer and versus FD) over time; (ii) identify new quality indicators.

Methods: 5673 slides were submitted to PR and examined by an average of 8 cytologists (range: 4-13). The agreement between cytologists and between the individual diagnosis with FD were evaluated by Kappa (k) and weighted Kappa (wK) and compared between 'experts' and 'less experienced' readers.

Results: The inter-observer agreement showed a moderate agreement among readers (whole team k = 0.44; experts k = 0.48). The highest and the lowest agreement was reported in HSIL and ASC-H, respectively. In 2018 and 2021, a significant reduction of kappa was observed, likely attributable to team turnover. The laboratory agreement versus FD resulted in significantly higher scores in experts (wk = 0.73, 95% CI 0.73-0.74) compared to less experienced individuals (wk = 0.65, 95% CI 0.64-0.66), with a general reduction of wk recorded in 2021. The individual agreement versus FD (calculated for 16 cytologists) achieved a moderate/substantial level of agreement (wK range: 0.57-0.80), with a shift toward higher wk in experts.

Conclusion: The levels of agreement are influenced by cytologist experience and team turnover. We propose new potential (key) performance indicators to strictly monitor the occurrence of systematic differences in interpretation criteria among cytologists. The proposed reference values are based on preliminary data and should be validated prospectively over a longer monitoring period.

Keywords: cervical cancer screening; cytology; internal quality control; pap test; peer review.

PubMed Disclaimer

References

    1. F. Carozzi, E. Burroni, M. Confortini, et al., “Implementation of a Centralized HPV‐Based Cervical Cancer Screening Programme in Tuscany: First Round Results and Comparison With the Foregoing Pap‐Based Screening Programme,” Journal of Medical Screening 29, no. 2 (2022): 110–122, https://doi.org/10.1177/09691413211067922.
    1. M. Confortini, G. Montanari, S. Prandi, et al., “Gruppo Italiano Screening del Cervicocarcinoma (GISCi): Recommendations for Quality Control of Cervico‐Vaginal Cytology,” Epidemiologia e Prevenzione 28, no. suppl (2004): 1–16.
    1. A. Placidi, P. Capparucci, A. Di Luzio, et al., “Team Reading (Peer Review) of Suspicious/Positive Slides for Continuous Quality Improvement in Cervical‐Vaginal Cytology: A Comparison Between Methods and Indicators,” Acta Cytologica 60, no. 5 (2016): 458–464, https://doi.org/10.1159/000448370.
    1. M. R. Melamed, “Quality Control in the Cytology Laboratory,” Acta Cytologica 20, no. 3 (1976): 203–206.
    1. C. N. Carney, “Quality Control in Cytopathology. A System for Simultaneous Monitoring of Accuracy and Education and for Proficiency Testing,” Acta Cytologica 28, no. 5 (1984): 535–540.

LinkOut - more resources