An internal standard for verifying the accuracy of serious adverse event reporting: the example of an acupuncture study of 190,924 patients

Abstract

Background: Reporting of all serious adverse events (SAEs) is a requirement for regulatory approval of a drug. Can equally rigorous reporting standards be expected in studies of non-drug treatments and how can underreporting, if any, be detected and proven? Using data from our large-scale prospective cohort study of acupuncture on outpatients, we examine the use of an internal standard, a principle taken from laboratory medicine, to quantify real event rates.

Methods: A total of 190,924 patients (68.6% women) seeking treatment for chronic pain (headache, low back pain, coxarthrosis or gonarthrosis) from 12,000 physicians in private practice in Germany were observed during a six-month period ending in May 2002. Most received ten sessions of body acupuncture. Mean treatment time was six weeks. All practitioners were certified in acupuncture and received written instructions on completing forms for basic patient data and SAE monitoring. They were also informed that payment by insurers would be made only upon return of the completed form. All SAEs occurring between start of the first acupuncture session and end of the last one were to be reported, whether causally related to the treatment or not. Multiple minor adverse events (AEs) per single patient were to be reported only once. As the internal standard we chose the expected number of deaths, based on the death rate for the German population, adjusted for age, sex distribution and mean observation time of our study patients.

Results: 45 SAEs and 14,404 AEs were reported (i.e. 2.4 SAEs and 754 AEs per 10,000 patients). The number of reported deaths (9) was only 5% of the statistically expected number (180). Applying the resulting correction factor of 20 to all reported SAEs, resulted in 900 expected SAEs (versus 45 reported) or 47 per 10,000 patients.

Conclusions: Without verifying the accuracy of a measurement, results remain speculative. Our internal standard for the first time provides a means of verifying the accuracy of the reported SAE rate and correcting it to the statistically expected SAE rate.