Comparison of three methods (an updated logistic probabilistic method, the Naranjo and Liverpool algorithms) for the evaluation of routine pharmacovigilance case reports using consensual expert judgement as reference

Abstract

Background: An updated probabilistic causality assessment method and the Liverpool algorithm presented as an improved version of the Naranjo algorithm, one of the most used and accepted causality assessment methods, have recently been proposed.

Objective: In order to test the validity of the probabilistic method in routine pharmacovigilance, results provided by the Naranjo and Liverpool algorithms, as well as the updated probabilistic method, were each compared with a consensual expert judgement taken as reference.

Methods: A sample of 59 drug-event pairs randomly sampled from spontaneous reports to the French pharmacovigilance system was assessed by expert judgement until reaching consensus and by members of a pharmacovigilance unit using the updated probabilistic method, the Naranjo and Liverpool algorithms. Probabilities given by the probabilistic method, and categories obtained by both the Naranjo and the Liverpool algorithms were compared as well as their sensitivity, specificity, positive and negative predictive values.

Results: The median probability for drug causation given by the consensual expert judgement was 0.70 (inter-quartile range, IQR 0.54-0.84) versus 0.77 (IQR 0.54-0.91) for the probabilistic method. For the Naranjo algorithm, the 'possible' causality category was predominant (61 %), followed by 'probable' (35 %), 'doubtful', and 'almost certain' categories (2 % each). Category distribution obtained with the Liverpool algorithm was similar to that obtained by the Naranjo algorithm with a majority of 'possible' (61 %) and 'probable' (30 %) followed by 'definite' (7 %) and 'unlikely' (2 %). For the probabilistic method, sensitivity, specificity, positive and negative predictive values were 0.96, 0.56, 0.92 and 0.71, respectively. For the Naranjo algorithm, depending on whether the 'possible' category was considered in favour or in disfavour of drug causation, sensitivity was, respectively, 1 or 0.42, specificity 0.11 or 0.89, negative predictive value 1 or 0.22 and positive predictive value 0.86 or 0.95; results were identical for the Liverpool algorithm.

Conclusion: The logistic probabilistic method gave results closer to the consensual expert judgment than either the Naranjo or Liverpool algorithms whose performance were strongly dependent on the meaning given to the 'possible' category. Owing to its good sensitivity and positive predictive value and by providing results as continuous probabilities, the probabilistic method seems worthy to use for a trustable assessment of adverse drug reactions in routine practice.