Statistical modeling of acute and chronic pain patient-reported outcomes obtained from ecological momentary assessment

Abstract

Ecological momentary assessment (EMA) allows for the collection of participant-reported outcomes (PROs), including pain, in the normal environment at high resolution and with reduced recall bias. Ecological momentary assessment is an important component in studies of pain, providing detailed information about the frequency, intensity, and degree of interference of individuals' pain. However, there is no universally agreed on standard for summarizing pain measures from repeated PRO assessment using EMA into a single, clinically meaningful measure of pain. Here, we quantify the accuracy of summaries (eg, mean and median) of pain outcomes obtained from EMA and the effect of thresholding these summaries to obtain binary clinical end points of chronic pain status (yes/no). Data applications and simulations indicate that binarizing empirical estimators (eg, sample mean, random intercept linear mixed model) can perform well. However, linear mixed-effect modeling estimators that account for the nonlinear relationship between average and variability of pain scores perform better for quantifying the true average pain and reduce estimation error by up to 50%, with larger improvements for individuals with more variable pain scores. We also show that binarizing pain scores (eg, <3 and ≥3) can lead to a substantial loss of statistical power (40%-50%). Thus, when examining pain outcomes using EMA, the use of linear mixed models using the entire scale (0-10) is superior to splitting the outcomes into 2 groups (<3 and ≥3) providing greater statistical power and sensitivity.

Figure 1:

Distribution of daily and average (across days) PROs of average and highest pain in the last 24 hours in two clinical samples. (a) Distribution of patients’ average and daily pain records; and (b) relationship between patients’ average pain records and the standard deviation across days. (a) Distribution of patient average daily pain scores (daily records averaged across days of assessment solid gold points) along with daily PRO pain (semitransparent black points). Each column of points on the x-axis represents a single participant, the y-axis corresponds to the participant’s average pain records across days (solid gold points) and daily records (semitransparent black points). Participants’ data are ordered by their average pain across days. Patient pain is presented for the MiCAPP (left panel) and TKA (right panel) samples which assessed average and highest daily pain, respectively. (b) Patient average pain scores (x-axis) versus standard deviation of pain scores (y-axis) are presented as solid black points. A non-parametric smoother is used to estimate the conditional mean of the standard deviation given average pain (solid blue line).

Figure 1:

Distribution of daily and average (across days) PROs of average and highest pain in the last 24 hours in two clinical samples. (a) Distribution of patients’ average and daily pain records; and (b) relationship between patients’ average pain records and the standard deviation across days. (a) Distribution of patient average daily pain scores (daily records averaged across days of assessment solid gold points) along with daily PRO pain (semitransparent black points). Each column of points on the x-axis represents a single participant, the y-axis corresponds to the participant’s average pain records across days (solid gold points) and daily records (semitransparent black points). Participants’ data are ordered by their average pain across days. Patient pain is presented for the MiCAPP (left panel) and TKA (right panel) samples which assessed average and highest daily pain, respectively. (b) Patient average pain scores (x-axis) versus standard deviation of pain scores (y-axis) are presented as solid black points. A non-parametric smoother is used to estimate the conditional mean of the standard deviation given average pain (solid blue line).

Figure 2:

Estimation accuracy of various estimators for ${\mu }_i$, participant specific average pain score as measured by mean squared error (MSE). Color corresponds to a particular estimator: empirical, LMM-CV, and LMM-NCV are presented as light blue, blue, and purple, respectively. Within (A), (B), and (C), each panel corresponds to a different number of participants in the dataset (columns, $N=100$ or $N=200$, left to right) and a different missing data generating mechanism (rows, noninformative or informative, top to bottom). Within (D) each column corresponds to MSE grouped by number of observed pain scores (1–3, 4–7, 8–10, left to right). (A) Boxplots of MSE averaged across all individuals within a simulated dataset. The x-axis corresponds to number of maximum pain scores per person (J) of either 10 or 20. (B) Average MSE across all datasets calculated separately for individuals with true average ${\mu }_i$ binned into 10 intervals of length 0.1. Line type corresponds to J of either 10 (solid lines) or 20 (dashed lines). (C) Boxplots of MSE averaged across all individuals within a simulated dataset among scenarios with maximum number of pain scores per person of 10 ($J=10$). The x-axis corresponds to number of observed pain scores per person (1–3, 4–7, 8–10, left to right). (D) Average MSE across all datasets calculated separately for individuals with true average ${\mu }_i$ binned into 10 intervals of length 0.1 for scenarios with $N=100$, $J=10$.

Figure 3:

Inferential accuracy of various estimators for (A) $\text{Pr}(y_{ij}\ge 0.3)$; and (B) 95% Coverage probabilities for scenarios with maximum number of pain scores equal to 10 ($J=10$) and number of persons equal to 100 ($N=100$). Accuracy is presented as a function of individuals’ true average pain score (x-axis, ${mu}_i$). Color corresponds to a particular estimator: LMM-CV and LMM-NCV are presented as light blue and blue, respectively. Within (A) and (B), each panel corresponds to a different number of observed pain scores per person (columns, 1–3, 4–7, 7–10, left to right) and a different missing data generating mechanism (rows, non-informative or informative, top to bottom) (A) MSE averaged across all simulated datasets. (B) Coverage probability averaged across all simulated datasets. Quantities were calculated by binning individuals into 10 intervals of length 0.1 individuals within a simulated dataset.

Figure 4:

Classification accuracy of chronic pain based on a threshold of 0.3 (3 out of 10 on a 10-point scale) using various estimators for ${\mu }_i$ for datasets simulated with 100 participants ($N=100$) and a maximum of 10 observations per participant ($J=10$) presented separately by the number of nonmissing observations per participant. Color corresponds to a particular estimator: empirical, LMMCV, and LMM-NCV are presented as light blue, blue, and purple, respectively. Within (A)-(D) each row corresponds to a different data generating mechanism (non-informative and informative, top and bottom rows, respectively). (A) Boxplots of the accuracy of classification measures (accuracy, NPV, PPV, sensitivity, specificity, from left to right panels) across simulated datasets. (B) Local accuracy of classification of chronic pain as a function of ${\mu }_i$ binned into 10 intervals of length 0.1. Line type corresponds to J of either 10 (solid lines) or 20 (dashed lines). (C) Same structure as (A), but accuracy metrics presented only for scenarios with $J=10$, separately for number of observed pain scores per participant (x-axis). (D) Same structure as (B), but accuracy metrics presented only for scenarios with $J=10$, separately for number of observed pain scores per participant (columns, 1–3, 4–7, 8–10, left to right).

Figure 5:

(a) Distribution of participants’ average and daily pain records; and (b) relationship between participants’ average pain records and the standard deviation across days in the TKA sample for the PROs of: average daily pain, percent of the day in pain, interference with non-work activities, interference with work. (a) Distribution of average daily pain scores (solid gold points) along with individual PRO pain (semitransparent black points). Each column of points on the x-axis represents a single participant, the y-axis corresponds to the participant’s average pain across days (solid gold points) and daily records (semitransparent black points). Participants’ data are ordered by their average pain across days. (b) Patient average pain scores (x-axis) versus standard deviation of pain scores (y-axis) are presented as solid black points. A non-parametric smoother is used to estimate the conditional mean of the standard deviation given average pain (solid blue line).

Figure 5:

(a) Distribution of participants’ average and daily pain records; and (b) relationship between participants’ average pain records and the standard deviation across days in the TKA sample for the PROs of: average daily pain, percent of the day in pain, interference with non-work activities, interference with work. (a) Distribution of average daily pain scores (solid gold points) along with individual PRO pain (semitransparent black points). Each column of points on the x-axis represents a single participant, the y-axis corresponds to the participant’s average pain across days (solid gold points) and daily records (semitransparent black points). Participants’ data are ordered by their average pain across days. (b) Patient average pain scores (x-axis) versus standard deviation of pain scores (y-axis) are presented as solid black points. A non-parametric smoother is used to estimate the conditional mean of the standard deviation given average pain (solid blue line).