Low Concordance of Patient-Reported Outcomes With Clinical and Clinical Trial Documentation

Abstract

Purpose: Health care research increasingly relies on assessment of data extracted from electronic medical records (EMRs). Clinical trial adverse event (AE) logs and patient-reported outcomes (PROs) are sources of data often available in the context of specific research projects. The aim of this study was to evaluate the extent of data concordance from these sources.

Patients and methods: Patients enrolled in clinical trials or receiving standard treatment for lung cancer (n = 62) completed validated questionnaires on physical and psychological symptoms at up to three assessment points. Temporally matched documentation was extracted from EMR notes and, for clinical trial participants (n = 41), AE logs. Evaluated data included symptom assessment, vital signs, medication logs, and laboratory values. Agreement (positive, negative) and Cohen's κ coefficients were calculated to assess concordance of symptoms among sources, with PROs considered the gold standard.

Results: Patient-reported weight loss correlated significantly with clinical measurements ( t = 2.90; P = .02), and average number of PROs correlated negatively with albumin concentration, supporting PROs as the gold standard. Comparisons of PROs versus EMR yielded poor concordance across 11 physical symptoms, anxiety, and depressive symptoms (all κ < 0.40). Providers under-reported the presence of each symptom in the EMR compared with PROs. AE logs showed similarly poor concordance with PROs (all κ < 0.40, except shortness of breath). Negative agreement among sources was higher than positive agreement for all symptoms except pain.

Conclusion: There was poor concordance between EMR notes and AE logs with PROs. Findings suggest that EMR notes and AE logs may not be reliable sources for capturing physical and psychological symptoms experienced by patients with lung cancer, supporting use of PRO assessments in oncology practices.

Fig 1.

CONSORT diagram of analysis.

Fig 2.

Comparison of symptom reports with objective patient data. (A) Patient-reported weight loss compared with objective weight change recorded in electronic medical record (n = 27; t = 2.9; P = .009). (B, C) Average number of symptom counts on (B) Memorial Symptom Assessment–Physical Symptom Subscale (MSAS; β = −1.58; P = .033) and (C) Functional Assessment of Cancer Therapy Scale (FACT; β = −2.02; P = .009) compared with average albumin concentration per patient. (D) Adverse event (AE) log reporting of mental health issues compared with medications (depression: χ² = 17.8; P < .001; anxiety: χ² = 19.8; P < .001; insomnia: χ² = 11.7; P < .001). (E) Patient-reported outcomes (PROs) of mental health issues compared with medications (depression: χ² = 0.75; P = .38; anxiety: χ² = 0.01; P = .91; insomnia: χ² = 6.4; P = .01). (*) P < .05.

Fig 3.

Patient responses to Functional Assessment of Cancer Therapy Scale questions (A) “I am able to work” and (B) “I am forced to spend time in bed” on a 5-point Likert scale by Eastern Cooperative Oncology Group (ECOG) score (ECOG v “I am able to work”: χ² = 70.8; df = 12; P < .001; ECOG v “I am forced to spend time in bed”: χ² = 57.6; df = 12; P < .001).