Delphi: A Democratic and Cost-Effective Method of Consensus Generation in Transplantation

Abstract

The Thrombotic Microangiopathy Banff Working Group (TMA-BWG) was formed in 2015 to survey current practices and develop minimum diagnostic criteria (MDC) for renal transplant TMA (Tx-TMA). To generate consensus among pathologists and nephrologists, the TMA BWG designed a 3-Phase study. Phase I of the study is presented here. Using the Delphi methodology, 23 panelists with >3 years of diagnostic experience with Tx-TMA pathology listed their MDC suggesting light, immunofluorescence, and electron microscopy lesions, clinical and laboratory information, and differential diagnoses. Nine rounds (R) of consensus resulted in MDC validated during two Rs using online evaluation of whole slide digital images of 37 biopsies (28 TMA, 9 non-TMA). Starting with 338 criteria the process resulted in 24 criteria and 8 differential diagnoses including 18 pathologic, 2 clinical, and 4 laboratory criteria. Results show that 3/4 of the panelists agreed on the diagnosis of 3/4 of cases. The process also allowed definition refinement for 4 light and 4 electron microscopy lesions. For the first time in Banff classification, the Delphi methodology was used to generate consensus. The study shows that Delphi is a democratic and cost-effective method allowing rapid consensus generation among numerous physicians dealing with large number of criteria in transplantation.

Keywords: Banff; Delphi; kidney; thrombotic microangiopathy; transplantation.

FIGURE 1

The Delphi process applied to this study. Nine rounds of survey (R1–R9) were designed. At the beginning of each round or R, the facilitator presented the panelists with the results (criteria) obtained from the previous R and asked them to either approve/disapprove of the listed criteria or to rank them. The panelists individually responded to this call and sent their votes to the facilitator who would collect the responses, eliminate redundancies, and apply a cut-off (80% or 60%) to that R. The results of the cut-off application were then shared with the panelists. A new list composed of all criteria that were above the cut-off was made by the facilitator and presented in the next R to the panelists. R6 and R7 were two rounds during which the criteria obtained from R5 were validated against 37 real-life cases by the panelists. R9 was a control round during which the integrity of the entire Delphi process was assessed. R9 was used to fine tune the definitions of the lesions that the panelists had difficulty with, during the validation R and was therefore called the Definition R.

FIGURE 2

Panelists’ performance assessed at the end of R6. The facilitator observed the panelists’ performance looking at multiple agreement levels. At the end of R6, the first validation R, %AL was assessed at 50%, 60%, 70%, 80%, 90% and 100% levels. The results show that at 70%AL (middle bar), consensus was reached on 28/37 (76%) of cases. This means that almost three-quarters of the panelists agreed on three-quarters of the cases.