Standardization of Clinical Assessment and Sample Collection Across All PERCH Study Sites

Abstract

Background.: Variable adherence to standardized case definitions, clinical procedures, specimen collection techniques, and laboratory methods has complicated the interpretation of previous multicenter pneumonia etiology studies. To circumvent these problems, a program of clinical standardization was embedded in the Pneumonia Etiology Research for Child Health (PERCH) study.

Methods.: Between March 2011 and August 2013, standardized training on the PERCH case definition, clinical procedures, and collection of laboratory specimens was delivered to 331 clinical staff at 9 study sites in 7 countries (The Gambia, Kenya, Mali, South Africa, Zambia, Thailand, and Bangladesh), through 32 on-site courses and a training website. Staff competency was assessed throughout 24 months of enrollment with multiple-choice question (MCQ) examinations, a video quiz, and checklist evaluations of practical skills.

Results.: MCQ evaluation was confined to 158 clinical staff members who enrolled PERCH cases and controls, with scores obtained for >86% of eligible staff at each time-point. Median scores after baseline training were ≥80%, and improved by 10 percentage points with refresher training, with no significant intersite differences. Percentage agreement with the clinical trainer on the presence or absence of clinical signs on video clips was high (≥89%), with interobserver concordance being substantial to high (AC1 statistic, 0.62-0.82) for 5 of 6 signs assessed. Staff attained median scores of >90% in checklist evaluations of practical skills.

Conclusions.: Satisfactory clinical standardization was achieved within and across all PERCH sites, providing reassurance that any etiological or clinical differences observed across the study sites are true differences, and not attributable to differences in application of the clinical case definition, interpretation of clinical signs, or in techniques used for clinical measurements or specimen collection.

Keywords: childhood; hospital; pneumonia; standardization.; training.

Figure 1.

Distribution of multiple choice question (MCQ) scores by site and training time-point: postbaseline training (A), pre- and postrefresher training 1 (B), and refresher training 2 (C). Boxplots display the distribution of MCQ scores. The number beneath each boxplot indicates the number of Pneumonia Etiology Research for Child Health (PERCH) clinicians and nurses who took the MCQ at each site. The diamond and horizontal line within the boxes represent the mean and median, respectively. The box reflects the interquartile range (IQR) and the whiskers extend to 1.5 multiplied by the IQR in either direction, or maximum and minimum values (if no outliers). The circle indicates outliers (values lying outside 1.5 multiplied by the IQR). Abbreviations: BAN, Bangladesh; GAM, The Gambia; KEN, Kenya; MAL, Mali; MCQ, multiple-choice question; pre, pre-course MCQ; post, post-course MCQ; RF1, refresher training 1; RF2, refresher training 2; SAF, South Africa; THA, Thailand; ZAM, Zambia.

Figure 2.

Staff retention during the course of the Pneumonia Etiology Research for Child Health (PERCH) study, by site. Kaplan-Meier graph displaying the proportion of staff attending the initial baseline training (N = 137) who remained with the study; analysis includes all staff members regardless of whether or not they enrolled study participants. Drops represent staff members leaving the study over time. The table beneath the graph indicates the number of staff members who remained in the study over time. Abbreviations: BAN, Bangladesh; GAM, The Gambia; KEN, Kenya; MAL, Mali; SAF, South Africa; THA, Thailand; ZAM, Zambia.