Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2008 Jul;23(7):903-7.
doi: 10.1007/s11606-008-0664-3.

Predictive validity evidence for medical education research study quality instrument scores: quality of submissions to JGIM's Medical Education Special Issue

Darcy A Reed  1 , Thomas J Beckman, Scott M Wright, Rachel B Levine, David E Kern, David A Cook
Affiliations

Predictive validity evidence for medical education research study quality instrument scores: quality of submissions to JGIM's Medical Education Special Issue

Darcy A Reed et al. J Gen Intern Med. 2008 Jul.

Abstract

Background: Deficiencies in medical education research quality are widely acknowledged. Content, internal structure, and criterion validity evidence support the use of the Medical Education Research Study Quality Instrument (MERSQI) to measure education research quality, but predictive validity evidence has not been explored.

Objective: To describe the quality of manuscripts submitted to the 2008 Journal of General Internal Medicine (JGIM) medical education issue and determine whether MERSQI scores predict editorial decisions.

Design and participants: Cross-sectional study of original, quantitative research studies submitted for publication.

Measurements: Study quality measured by MERSQI scores (possible range 5-18).

Results: Of 131 submitted manuscripts, 100 met inclusion criteria. The mean (SD) total MERSQI score was 9.6 (2.6), range 5-15.5. Most studies used single-group cross-sectional (54%) or pre-post designs (32%), were conducted at one institution (78%), and reported satisfaction or opinion outcomes (56%). Few (36%) reported validity evidence for evaluation instruments. A one-point increase in MERSQI score was associated with editorial decisions to send manuscripts for peer review versus reject without review (OR 1.31, 95%CI 1.07-1.61, p = 0.009) and to invite revisions after review versus reject after review (OR 1.29, 95%CI 1.05-1.58, p = 0.02). MERSQI scores predicted final acceptance versus rejection (OR 1.32; 95% CI 1.10-1.58, p = 0.003). The mean total MERSQI score of accepted manuscripts was significantly higher than rejected manuscripts (10.7 [2.5] versus 9.0 [2.4], p = 0.003).

Conclusions: MERSQI scores predicted editorial decisions and identified areas of methodological strengths and weaknesses in submitted manuscripts. Researchers, reviewers, and editors might use this instrument as a measure of methodological quality.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Quality of rejected and accepted manuscripts in the 2008 JGIM Medical education special issue. Legend. Calculated for 65 rejected and 35 accepted manuscripts. Columns represent mean domain-specific MERSQI scores. Error bars represent standard deviation of mean domain-specific MERSQI scores. Maximum possible domain-specific MERSQI score is 3.
See this image and copyright information in PMC

References

    1. {'text': '', 'ref_index': 1, 'ids': [{'type': 'DOI', 'value': '10.1097/00001888-200410000-00005', 'is_inner': False, 'url': 'https://doi.org/10.1097/00001888-200410000-00005'}, {'type': 'PubMed', 'value': '15383347', 'is_inner': True, 'url': 'https://pubmed.ncbi.nlm.nih.gov/15383347/'}]}
    2. Dauphinee WD, Wood-Dauphinee S. The need for evidence in medical education: the development of best evidence medical education as an opportunity to inform, guide, and sustain medical education research. Acad Med. 2004;79(10):925–30. - PubMed
    1. {'text': '', 'ref_index': 1, 'ids': [{'type': 'DOI', 'value': '10.1097/00001888-199408000-00003', 'is_inner': False, 'url': 'https://doi.org/10.1097/00001888-199408000-00003'}, {'type': 'PubMed', 'value': '8054103', 'is_inner': True, 'url': 'https://pubmed.ncbi.nlm.nih.gov/8054103/'}]}
    2. Wartman SA. Research in medical education: the challenge for the next decade. Acad Med. 1994;69(8):608–14. - PubMed
    1. {'text': '', 'ref_index': 1, 'ids': [{'type': 'DOI', 'value': '10.1097/00001888-200410000-00006', 'is_inner': False, 'url': 'https://doi.org/10.1097/00001888-200410000-00006'}, {'type': 'PubMed', 'value': '15383348', 'is_inner': True, 'url': 'https://pubmed.ncbi.nlm.nih.gov/15383348/'}]}
    2. Shea JA, Arnold L, Mann KV. A RIME perspective on the quality and relevance of current and future medical education research. Acad Med. 2004;79(10):931–8. - PubMed
    1. {'text': '', 'ref_index': 1, 'ids': [{'type': 'DOI', 'value': '10.1001/jama.290.9.1210', 'is_inner': False, 'url': 'https://doi.org/10.1001/jama.290.9.1210'}, {'type': 'PubMed', 'value': '12953006', 'is_inner': True, 'url': 'https://pubmed.ncbi.nlm.nih.gov/12953006/'}]}
    2. Lurie SJ. Raising the passing grade for studies of medical education. JAMA. 2003;290(9):1210–2. - PubMed
    1. {'text': '', 'ref_index': 1, 'ids': [{'type': 'DOI', 'value': '10.1097/00001888-200410000-00004', 'is_inner': False, 'url': 'https://doi.org/10.1097/00001888-200410000-00004'}, {'type': 'PubMed', 'value': '15383346', 'is_inner': True, 'url': 'https://pubmed.ncbi.nlm.nih.gov/15383346/'}]}
    2. Carline JD. Funding medical education research: opportunities and issues. Acad Med. 2004;79(10):918–24. - PubMed

Publication types

LinkOut - more resources