Reliability of Clinical Assessments in Older Adults With Syncope or Near Syncope

Abstract

Objectives: Clinical prediction models for risk stratification of older adults with syncope or near syncope may improve resource utilization and management. Predictors considered for inclusion into such models must be reliable. Our primary objective was to evaluate the inter-rater agreement of historical, physical examination, and electrocardiogram (ECG) findings in older adults undergoing emergency department (ED) evaluation for syncope or near syncope. Our secondary objective was to assess the level of agreement between clinicians on the patient's overall risk for death or serious cardiac outcomes.

Methods: We conducted a cross-sectional study at 11 EDs in adults 60 years of age or older who presented with unexplained syncope or near syncope. We excluded patients with a presumptive cause of syncope (e.g., seizure) or if they were unable or unwilling to follow-up. Evaluations of the patient's past medical history and current medication use were completed by treating provider and trained research associate pairs. Evaluations of the patient's physical examination and ECG interpretation were completed by attending/resident, attending/advanced practice provider, or attending/attending pairs. All evaluations were blinded to the responses from the other rater. We calculated the percent agreement and kappa statistic for binary variables. Inter-rater agreement was considered acceptable if the kappa statistic was 0.6 or higher.

Results: We obtained paired observations from 255 patients; mean (±SD) age was 73 (±9) years, 137 (54%) were male, and 204 (80%) were admitted to the hospital. Acceptable agreement was achieved in 18 of the 21 (86%) past medical history and current medication findings, none of the 10 physical examination variables, and three of the 13 (23%) ECG interpretation variables. There was moderate agreement (Spearman correlation coefficient, r = 0.40) between clinicians on the patient's probability of 30-day death or serious cardiac outcome, although as the probability increased, there was less agreement.

Conclusions: Acceptable agreement between raters was more commonly achieved with historical rather than physical examination or ECG interpretation variables. Clinicians had moderate agreement in assessing the patient's overall risk for a serious outcome at 30 days. Future development of clinical prediction models in older adults with syncope should account for variability of assessments between raters and consider the use of objective clinical variables.

Figure 1. Interrater agreement of probability of 30-day serious cardiac event

Figure 2. Percent agreement and kappa statistic of clinical findings