Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2006 Jan-Feb;13(1):52-60.
doi: 10.1197/jamia.M1911. Epub 2005 Oct 12.

Automatically identifying health outcome information in MEDLINE records

Dina Demner-Fushman  1 , Barbara Few, Susan E Hauser, George Thoma
Affiliations

Automatically identifying health outcome information in MEDLINE records

Dina Demner-Fushman et al. J Am Med Inform Assoc. 2006 Jan-Feb.

Abstract

Objective: Understanding the effect of a given intervention on the patient's health outcome is one of the key elements in providing optimal patient care. This study presents a methodology for automatic identification of outcomes-related information in medical text and evaluates its potential in satisfying clinical information needs related to health care outcomes.

Design: An annotation scheme based on an evidence-based medicine model for critical appraisal of evidence was developed and used to annotate 633 MEDLINE citations. Textual, structural, and meta-information features essential to outcome identification were learned from the created collection and used to develop an automatic system. Accuracy of automatic outcome identification was assessed in an intrinsic evaluation and in an extrinsic evaluation, in which ranking of MEDLINE search results obtained using PubMed Clinical Queries relied on identified outcome statements.

Measurements: The accuracy and positive predictive value of outcome identification were calculated. Effectiveness of the outcome-based ranking was measured using mean average precision and precision at rank 10.

Results: Automatic outcome identification achieved 88% to 93% accuracy. The positive predictive value of individual sentences identified as outcomes ranged from 30% to 37%. Outcome-based ranking improved retrieval accuracy, tripling mean average precision and achieving 389% improvement in precision at rank 10.

Conclusion: Preliminary results in outcome-based document ranking show potential validity of the evidence-based medicine-model approach in timely delivery of information critical to clinical decision support at the point of service.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
(A) An outcome statement identified by the RN. (B) An outcome statement identified by the medical student.
Figure 2.
Figure 2.
Positions of outcome statements in 275 training abstracts.
See this image and copyright information in PMC

References

    1. Arndt KA. Information excess in medicine. Overview, relevance to dermatology, and strategies for coping. Arch Dermatol. 1992;128:1249–56. - PubMed
    1. Bakken S, Cimino JJ, Hripcsak G. Enabling patient safety and evidence-based practice through informatics. Med Care. 2004;42:S49–56. - PubMed
    1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71–2. - PMC - PubMed
    1. Covell DG, Uman GC, Manning PR. Information needs in office practice: are they being met? Ann Intern Med. 1985;103:596–9. - PubMed
    1. Northup DE, Moore-West M, Skipper B, Teaf SR. Characteristics of clinical information–searching: investigation using critical incident technique. J. Med Educ. 1983;58:873–81. - PubMed