Inter-and intra-observer agreement on the judgment of toluidine blue staining for screening of oral potentially malignant disorders and oral cancer

Abstract

Objective: Toluidine blue staining (TBS) is used for screening of oral potentially malignant disorders (OPMDs) and oral cancer. Nevertheless, the judgment of TBS is based on the observer's personal experience and largely considered subjective. The aim of this study was to evaluate the inter- and intra-observer agreement on the judgment of TBS for screening of OPMDs and oral cancer.

Materials and methods: Two hundred randomly ordered TBS images of OPMDs and oral cancer were judged twice independently by four clinicians and four graduated students majored in oral medicine, with an interval of 3 months. Inter- and intra-observer agreement was evaluated by the Cohen's kappa (κ) coefficient. The correlation between κ-value and individual experience was assessed by Spearman's correlation coefficient.

Results: For the two sets of observation, good overall inter-observer agreement was found in the study (κ = 0.620 and 0.667). Sub-group analysis yielded good inter-observer agreement both for clinicians (κ = 0.613 and 0.611) and students (κ = 0.610 and 0.665). Pairwise inter-observer agreement was presented to be moderate to excellent between clinician and clinician (κ = 0.41~0.87, 0.43~0.81), moderate to excellent between student and student (κ = 0.43~0.71, 0.53~0.82), and poor to excellent between clinician and student (κ = 0.25~0.81, 0.35~0.87). Good to excellent intra-observer agreement (κ = 0.69~0.85) was presented for eight observers. However, no correlation was illustrated between years of experience and inter- or intra-observer agreement.

Conclusion: Judgment of TBS has a good overall inter-observer agreement and good to excellent intra-observer agreement, which guarantees its utilization and popularity in detecting OPMDs and oral cancer.

Clinical relevance: To our knowledge, this is the first study to investigate the inter- and intra-observer agreement of TBS judgment. The present results revealed a good inter-observer agreement and good to excellent intra-observer agreement, which provide a beneficial supplement to the knowledge of TBS technique.

Keywords: Inter- and intra-observer agreement; Oral cancer screening; Oral potentially malignant disorders; Toluidine blue staining.