Developing the Benchmark: Establishing a Gold Standard for the Evaluation of AI Caries Diagnostics

Abstract

Background/Objectives: The aim of this study was to establish a histology-based gold standard for the evaluation of artificial intelligence (AI)-based caries detection systems on proximal surfaces in bitewing images. Methods: Extracted human teeth were used to simulate intraoral situations, including caries-free teeth, teeth with artificially created defects and teeth with natural proximal caries. All 153 simulations were radiographed from seven angles, resulting in 1071 in vitro bitewing images. Histological examination of the carious lesion depth was performed twice by an expert. A total of thirty examiners analyzed all the radiographs for caries. Results: We generated in vitro bitewing images to evaluate the performance of AI-based carious lesion detection against a histological gold standard. All examiners achieved a sensitivity of 0.565, a Matthews correlation coefficient (MCC) of 0.578 and an area under the curve (AUC) of 76.1. The histology receiver operating characteristic (ROC) curve significantly outperformed the examiners' ROC curve (p < 0.001). All examiners distinguished induced defects from true caries in 54.6% of cases and correctly classified 99.8% of all teeth. Expert caries classification of the histological images showed a high level of agreement (intraclass correlation coefficient (ICC) = 0.993). Examiner performance varied with caries depth (p ≤ 0.008), except between E2 and E1 lesions (p = 1), while central beam eccentricity, gender, occupation and experience had no significant influence (all p ≥ 0.411). Conclusions: This study successfully established an unbiased dataset to evaluate AI-based caries detection on bitewing surfaces and compare it to human judgement, providing a standardized assessment for fair comparison between AI technologies and helping dental professionals to select reliable diagnostic tools.

Keywords: artificial intelligence; benchmarking; bitewing; dental caries; diagnostics; radiography.

Figure 1

Trial profile.

Figure 2

Photographic and radiological documentation of all teeth.

Figure 3

Occlusion holder with fixed teeth simulating clinical bitewing scenarios.

Figure 4

Digital in vitro bitewing images. Top: color-coded setup—yellow: examination tooth, red: carious lesion, blue: adjacent tooth, green: antagonistic tooth. Below: The mesial-eccentric series shows increased superimposition as the ray path becomes increasingly eccentric in the proximal region of teeth 46 and 47. Conversely, the distal-eccentric series shows increased superimposition as the ray path becomes increasingly eccentric in the interproximal region of teeth 15 and 16.

Figure 5

Sample preparation steps.

Figure 6

Equipment for the preparation of histological specimens. (A): EXAKT-HISTOLUX light polymerization unit, (B) EXAKT precision vacuum bonding press, (C): Thermo Heraeus B6060 Incubator, (D): EXAKT band saw, (E): EXAKT horizontal microgrinding system, (F): EXAKT micrometer screw.

Figure 7

Histological specimen with different proximal carious lesion depths. E0 = Caries-free, E1 = Caries limited to the outer half of the enamel, E2 = Caries extending to the inner half of the enamel, D1 = Caries in the outer third of dentin, D2 = Caries in the middle third of dentin, D3 = Caries in the dentinal third close to the pulp or up to the pulp.

Figure 8

AUC of examiners and histology.

Figure 9

MCC by lesion class.

Figure 10

MCC by gender.

Figure 11

MCC by occupation.

Figure 12

MCC by experience.

Figure 13

MCC by eccentricity angle. m = mesial-eccentric, d = distal-eccentric.

Figure 14

Assessment of artificially induced lesions.

Figure 15

Tooth classification according to the FDI scheme.