Does simulator-based clinical performance correlate with actual hospital behavior? The effect of extended work hours on patient care provided by medical interns

Abstract

Purpose: The correlation between simulator-based medical performance and real-world behavior remains unclear. This study explored whether the effects of extended work hours on clinical performance, as reported in prior hospital-based studies, could be observed in a simulator-based testing environment.

Method: Intern volunteers reported to the simulator laboratory in a rested state and again in a sleep-deprived state (after a traditional 24- to 30-hour overnight shift [n=17]). A subset also presented after a shortened overnight shift (16 scheduled hours [n=8]). During each laboratory visit, participants managed two critically ill patients. An on-site physician scored each case, as did a blinded rater later watching videotapes of the performances (score=1 [worst] to 8 [best]; average of both cases=session score).

Results: Among all participants, the average simulator session score was 6.0 (95% CI: 5.6-6.4) in the rested state and declined to 5.0 (95% CI: 4.6-5.4) after the traditional overnight shift (P<.001). Among those who completed the shortened overnight shift, the average postshift simulator session score was 5.8 (95% CI: 5.0-6.6) compared with 4.3 (95% CI: 3.8-4.9) after a traditional extended shift (P<.001).

Conclusions: In a clinical simulation test, medical interns performed significantly better after working a shortened overnight shift compared with a traditional extended shift. These findings are consistent with real-time hospital studies using the same shift schedule. Such an independent correlation not only confirms the detrimental impact of extended work hours on medical performance but also supports the validity of simulation as a clinical performance assessment tool.

Figure 1

Average performance in simulated acute care sessions in both rested and post-call conditions. Post-call sessions occurred after either a traditional 24- to 30-hour extended on-call shift or a modified 16-hour overnight scheduled shift. * P<0.001 for comparison of initially rested vs. traditional post-call (paired t-test, both cohorts) + P<0.001 for comparison of traditional vs. modified post-call (paired t-test, cohort 2) ++Cohort 2 is a subset of Cohort 1 that progressed through all 4 testing cycles

Figure 2

A Individual performances of the 17 interns of Cohort 1 in simulated acute care sessions in both rested and traditional post-call conditions. Traditional post-call sessions occurred after a 24- to 30-hour extended on-call shift. B Individual performances of the 8 interns in Cohort 2 in simulated acute care sessions in both rested and post-call conditions. Post-call sessions occurred after either a traditional 24- to 30-hour extended on-call shift or a modified 16-hour overnight scheduled shift.

Figure 2

A Individual performances of the 17 interns of Cohort 1 in simulated acute care sessions in both rested and traditional post-call conditions. Traditional post-call sessions occurred after a 24- to 30-hour extended on-call shift. B Individual performances of the 8 interns in Cohort 2 in simulated acute care sessions in both rested and post-call conditions. Post-call sessions occurred after either a traditional 24- to 30-hour extended on-call shift or a modified 16-hour overnight scheduled shift.