Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2005 Sep 1:2:8.
doi: 10.1186/1742-7622-2-8.

Assessing observational studies of medical treatments

Arthur Hartz  1 , Suzanne Bentler, Mary Charlton, Douglas Lanska, Yogita Butani, G Mustafa Soomro, Kjell Benson
Affiliations

Assessing observational studies of medical treatments

Arthur Hartz et al. Emerg Themes Epidemiol. .

Abstract

Background: Previous studies have assessed the validity of the observational study design by comparing results of studies using this design to results from randomized controlled trials. The present study examined design features of observational studies that could have influenced these comparisons.

Methods: To find at least 4 observational studies that evaluated the same treatment, we reviewed meta-analyses comparing observational studies and randomized controlled trials for the assessment of medical treatments. Details critical for interpretation of these studies were abstracted and analyzed qualitatively.

Results: Individual articles reviewed included 61 observational studies that assessed 10 treatment comparisons evaluated in two studies comparing randomized controlled trials and observational studies. The majority of studies did not report the following information: details of primary and ancillary treatments, outcome definitions, length of follow-up, inclusion/exclusion criteria, patient characteristics relevant to prognosis or treatment response, or assessment of possible confounding. When information was reported, variations in treatment specifics, outcome definition or confounding were identified as possible causes of differences between observational studies and randomized controlled trials, and of heterogeneity in observational studies.

Conclusion: Reporting of observational studies of medical treatments was often inadequate to compare study designs or allow other meaningful interpretation of results. All observational studies should report details of treatment, outcome assessment, patient characteristics, and confounding assessment.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Comparison of Confidence Intervals for Combined Odds Ratios from Observational Studies and Randomized Controlled Trials.
See this image and copyright information in PMC

References

    1. D'Agostino R, Kwan H. Measuring effectiveness. What to expect without a randomized control group. Med Care. 1995;33:AS95–105. - PubMed
    1. Pocock S. Randomized Trials or Observational Tribulations? N Eng J Med. 2000;342:1907–1908. doi: 10.1056/NEJM200006223422511. - DOI - PubMed
    1. Writing Group for the Women's Health Initiative I. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results From the Women's Health Initiative randomized controlled trial. JAMA-J Am Med Assoc. 2002;288:321–333. doi: 10.1001/jama.288.3.321. - DOI - PubMed
    1. Grodstein F, Manson JE, Colditz GA, Willett WC, Speizer FE, Stampfer MJ. A prospective, observational study of postmenopausal hormone therapy and primary prevention of cardiovascular disease. Ann Intern Med. 2000;133:933–941. - PubMed
    1. Heckbert SR, Weiss NS, Koepsell TD, Lemaitre RN, Smith NL, Siscovick DS, Lin D, Psaty BM. Duration of estrogen replacement therapy in relation to the risk of incident myocardial infarction in postmenopausal women. Arch Intern Med. 1997;157:1330–1336. doi: 10.1001/archinte.157.12.1330. - DOI - PubMed

LinkOut - more resources