Reliability of the Roussel Uclaf Causality Assessment Method for assessing causality in drug-induced liver injury

Abstract

The Roussel Uclaf Causality Assessment Method (RUCAM) was developed to quantify the strength of association between a liver injury and the medication implicated as causing the injury. However, its reliability in a research setting has never been fully explored. The aim of this study was to determine test-retest and interrater reliabilities of RUCAM in retrospectively-identified cases of drug induced liver injury. The Drug-Induced Liver Injury Network is enrolling well-defined cases of hepatotoxicity caused by isoniazid, phenytoin, clavulanate/amoxicillin, or valproate occurring since 1994. Each case was adjudicated by three reviewers working independently; after an interval of at least 5 months, cases were readjudicated by the same reviewers. A total of 40 drug-induced liver injury cases were enrolled including individuals treated with isoniazid (nine), phenytoin (five), clavulanate/amoxicillin (15), and valproate (11). Mean +/- standard deviation age at protocol-defined onset was 44.8 +/- 19.5 years; patients were 68% female and 78% Caucasian. Cases were classified as hepatocellular (44%), mixed (28%), or cholestatic (28%). Test-retest differences ranged from -7 to +8 with complete agreement in only 26% of cases. On average, the maximum absolute difference among the three reviewers was 3.1 on the first adjudication and 2.7 on the second, although much of this variability could be attributed to differences between the enrolling investigator and the external reviewers. The test-retest reliability by the same assessors was 0.54 (upper 95% confidence limit = 0.77); the interrater reliability was 0.45 (upper 95% confidence limit = 0.58). Categorizing the RUCAM to a five-category scale improved these reliabilities but only marginally.

Conclusion: The mediocre reliability of the RUCAM is problematic for future studies of drug-induced liver injury. Alternative methods, including modifying the RUCAM, developing drug-specific instruments, or causality assessment based on expert opinion, may be more appropriate.

Fig. 1

Histogram of the within-reviewer differences from the first occasion to the second occasion. (A) All reviewers. (B) Site PIs only.

Fig. 2

Bland-Altman plot of test-retest differences versus their mean. The purpose is to determine if the test-retest differences were consistent throughout the range of the RUCAM score. This would be reflected by a constant level of scatter about the reference line. Deviations form this pattern suggests that consistency varies from one place to another in the scale and casts doubt on its overall reliability.

Fig. 3

Histogram of the maximum absolute difference between the two external reviewers. (A) First occasion. (B) Second occasion.