Reproducibility of the histologic diagnosis of pneumonia among a panel of four pathologists: analysis of a gold standard

Abstract

Study objective: To establish a histologic diagnosis of pneumonia by consensus of a panel of pathologists, to test the interobserver and intraobserver variation in the histologic diagnosis of pneumonia, to compare the diagnostic accuracy of diagnosing pneumonia with and without preselected histologic criteria, and to establish more specific histologic criteria for the diagnosis of pneumonia.

Methods: The study group consisted of 39 patients who died after a mean of 14 days of mechanical ventilation. A postmortem open lung biopsy was performed on all patients. The tissue was reviewed independently by four pathologists who categorized the slides from each patient as showing or not showing pneumonia. Interobserver variation was calculated using the kappa statistic. Six months following the initial evaluation, the same slides were resubmitted to one of the pathologists for reevaluation to look for intraobserver error. Finally, the slides were reviewed and categorized by the criteria of Johanson et al into no pneumonia, mild, moderate, or severe bronchopneumonia. A comparison was made of the patients selected as demonstrating histologic pneumonia by each of the examinations.

Results: The reliability coefficient (kappa) measuring agreement among the four pathologists was good at 0.916. However, the prevalence of pneumonia as determined by each of the four pathologists varied; pathologist A, 15 of 39 (38%); pathologist B, 12 of 39 (31%); pathologist C, 9 of 39 (23%); and pathologist D, 7 of 39 (18%). Resubmitting the same slides to the same pathologist 6 months later resulted in reclassification of 2 of 39 patients. Using the histologic criteria of Johanson and colleagues, 14 patients were selected as having pneumonia compared with only nine patients selected by consensus of three of four pathologists.

Conclusions: Recognition of histologic pneumonia varies among pathologists. The preselected criteria of Johanson and colleagues detected histologic pneumonia in eight of nine patients picked by consensus of pathologists, but six additional patients classified as "no histologic pneumonia" by the consensus of pathologists were judged to have histologic pneumonia by these criteria. The results established the necessity for standardization of histologic criteria for studies using biopsy as the gold standard for bacterial pneumonia. An atlas showing the criteria used in our selection was developed.