[How effective is the teaching of critical analysis of scientific publications? Review of studies and their methodological quality]

Abstract

OBJECTIVE: To evaluate studies assessing the effectiveness of teaching critical appraisal of the literature to medical students. DATA SOURCES: French and English articles published between 1980 and 1990 indexed on MEDLINE or FAMLI as well as articles identified from the bibliographies. STUDY SELECTION: Studies were evaluated if the subjects were undergraduate or postgraduate medical students and if the teaching intervention was aimed at improving one or more of the following areas: knowledge in clinical epidemiology and biostatistics, reading habits and ability to critically appraise a scientific article. DATA EXTRACTION: The methodologic quality of the articles was assessed by three evaluators, who used a modified version of Poynard's checklist to assign a score. Articles with a score of 60% or more were considered satisfactory. The reliability of the checklist was evaluated by means of the kappa (kappa) coefficient and a coefficient of intraclass correlation. DATA SYNTHESIS: For the three evaluators the mean kappa coefficient was 0.33 and the coefficient of intraclass correlation 0.70. Five of the 10 studies had an overall score of 60% or higher. The quality of the individual sections of the articles varied: purpose of the study 85%, description of the population 58%, methods 44%, analysis of results 50%, and conclusions 90%. CONCLUSIONS: The effectiveness of teaching critical appraisal of the literature remains uncertain. More rigorous methods are needed in research in this area.