Inter-rater reliability of malaria parasite counts and comparison of methods

Abstract

Background: The introduction of artemesinin-based treatment for falciparum malaria has led to a shift away from symptom-based diagnosis. Diagnosis may be achieved by using rapid non-microscopic diagnostic tests (RDTs), of which there are many available. Light microscopy, however, has a central role in parasite identification and quantification and remains the main method of parasite-based diagnosis in clinic and hospital settings and is necessary for monitoring the accuracy of RDTs. The World Health Organization has prepared a proficiency testing panel containing a range of malaria-positive blood samples of known parasitaemia, to be used for the assessment of commercially available malaria RDTs. Different blood film and counting methods may be used for this purpose, which raises questions regarding accuracy and reproducibility. A comparison was made of the established methods for parasitaemia estimation to determine which would give the least inter-rater and inter-method variation

Methods: Experienced malaria microscopists counted asexual parasitaemia on different slides using three methods; the thin film method using the total erythrocyte count, the thick film method using the total white cell count and the Earle and Perez method. All the slides were stained using Giemsa pH 7.2. Analysis of variance (ANOVA) models were used to find the inter-rater reliability for the different methods. The paired t-test was used to assess any systematic bias between the two methods, and a regression analysis was used to see if there was a changing bias with parasite count level.

Results: The thin blood film gave parasite counts around 30% higher than those obtained by the thick film and Earle and Perez methods, but exhibited a loss of sensitivity with low parasitaemia. The thick film and Earle and Perez methods showed little or no bias in counts between the two methods, however, estimated inter-rater reliability was slightly better for the thick film method.

Conclusion: The thin film method gave results closer to the true parasite count but is not feasible at a parasitaemia below 500 parasites per microlitre. The thick film method was both reproducible and practical for this project. The determination of malarial parasitaemia must be applied by skilled operators using standardized techniques.

Figure 1

Diagram of slide sets and readers.

Figure 2

Inter-rater reliability for each method, on original (notlog) count scale. (a) Earle and Perez method. x axis: Mean count across raters for each sample. y axis: Difference between each raters count and the mean of all counts from the same sample. (b) Thick film method. X axis: Mean count across raters for each sample. Y axis: Difference between each raters count and the mean of all counts from the same sample.

Figure 3

Inter-rater reliability for each method, on log count scale, and therefore showing relative differences. (a) Earle and Perez graph. X axis: Mean (geometric) count across raters for each sample. Y axis: Relative difference between each raters count and the geometric mean of all counts from the same sample. (b) Thick film method. X axis: Mean (geometric) count across raters for each sample. Y axis: Relative difference between each raters count and the geometric mean of all counts from the same sample. (c) Thick film method restricted to EDTA Blood. X axis: Mean (geometric) count across raters for each sample. Y axis: Relative difference between each raters count and the geometric mean of all counts from the same sample.

Figure 4

Bland and Altman plots to demonstrate the presence of any bias between different methods of counting the parasite loads, on a log scale, to show relative differences. (a) Comparison of Earle and Perez with thick film methods (after averaging each method across all raters). X axis: Mean (geometric) averaged across both methods. Y axis: Relative difference between mean (geometric) counts from each method (Earle and Perez over Thick film method). (b Comparison of Earle and Perez with Thick Film methods (after averaging Earle and Perez across all raters). X axis: Mean (geometric) averaged across both methods. Y axis: Relative difference between mean (geometric) counts from each method (Earle and Perez over Thick film method). (c) Comparison of Thick Film with Thin Film methods (after averaging thick Film method across all raters). X axis: Mean (geometric) averaged across both methods. Y axis: Relative difference between mean (geometric) counts from each method (Thick film over thin film method).