Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Jun 27;21(1):357.
doi: 10.1186/s12909-021-02797-3.

Psychometric validation of the Laval developmental benchmarks scale for family medicine

Jean-Sébastien Renaud  1   2   3 , Miriam Lacasse  4   5 , Luc Côté  4 , Johanne Théorêt  4 , Christian Rheault  4 , Caroline Simard  5
Affiliations

Psychometric validation of the Laval developmental benchmarks scale for family medicine

Jean-Sébastien Renaud et al. BMC Med Educ. .

Abstract

Background: With the implementation of competency-based education in family medicine, there is a need for summative end-of-rotation assessments that are criterion-referenced rather than normative. Laval University's family residency program therefore developed the Laval Developmental Benchmarks Scale for Family Medicine (DBS-FM), based on competency milestones. This psychometric validation study investigates its internal structure and its relation to another variable, two sources of validity evidence.

Methods: We used assessment data from a cohort of residents (n = 1432 assessments) and the Rasch Rating Scale Model to investigate its reliability, dimensionality, rating scale functioning, targeting of items to residents' competency levels, biases (differential item functioning), items hierarchy (adequacy of milestones ordering), and score responsiveness. Convergent validity was estimated by its correlation with the clinical rotation decision (pass, in difficulty/fail).

Results: The DBS-FM can be considered as a unidimensional scale with good reliability for non-extreme scores (.83). The correlation between expected and empirical items hierarchies was of .78, p < .0001.Year 2 residents achieved higher scores than year 1 residents. It was associated with the clinical rotation decision.

Conclusion: Advancing its validation, this study found that the DBS-FM has a sound internal structure and demonstrates convergent validity.

Keywords: Criterion-referenced assessment; Family medicine; Validation.

PubMed Disclaimer

Conflict of interest statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

Figures

Fig. 1
Fig. 1
Option characteristic curves for the 3-point scale
Fig. 2
Fig. 2
Standard error of measurement relative to estimated Rasch ability level of residents
Fig. 3
Fig. 3
Distribution of the Rasch ability parameters for year 1 (top) and year 2 (bottom) residents
Fig. 4
Fig. 4
Wright map of persons and items parameters
Fig. 5
Fig. 5
Average CAS with 95% confidence intervals for the 26 periods of the residency program
Fig. 6
Fig. 6
Expected CAS
See this image and copyright information in PMC

References

    1. Weggemans MM, van Dijk B, van Dooijeweert B, Veenendaal AG, Ten Cate O. The postgraduate medical education pathway: an international comparison. GMS J Med Educ. 2017;34(5):Doc63-Doc. - PMC - PubMed
    1. Edgar L, McLean S, Hogan SO, Hamstra SJ, Holmboe ES. The milestones guidebook: Accreditation Council for Graduate Medical Education; 2020. https://www.acgme.org/Portals/0/MilestonesGuidebook.pdf. Accessed 1 June 2020.
    1. Baglia J, Foster E, Dostal J, Keister D, Biery N, Larson D. Generating developmentally appropriate competency assessment at a family medicine residency. Fam Med. 2011;43(2):90–98. - PubMed
    1. Frank JR, Snell LS, Sherbino J, et al. Draft CanMEDS 2015 Milestones Guide – May 2014. Ottawa: The Royal College of Physicians and Surgeons of Canada; 2014.
    1. General Medical Council . Excellence by design: standards for postgraduate curricula. 2017.

LinkOut - more resources