Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 May 26:30:e45.
doi: 10.1017/S204579602100038X.

Uncovering survivorship bias in longitudinal mental health surveys during the COVID-19 pandemic

Mark É Czeisler  1   2   3 , Joshua F Wiley  1 , Charles A Czeisler  1   4   5 , Shantha M W Rajaratnam  1   2   4   5 , Mark E Howard  1   2   6
Affiliations

Uncovering survivorship bias in longitudinal mental health surveys during the COVID-19 pandemic

Mark É Czeisler et al. Epidemiol Psychiatr Sci. .

Abstract

Aims: Markedly elevated adverse mental health symptoms were widely observed early in the coronavirus disease-2019 (COVID-19) pandemic. Unlike the U.S., where cross-sectional data indicate anxiety and depression symptoms have remained elevated, such symptoms reportedly declined in the U.K., according to analysis of repeated measures from a large-scale longitudinal study. However, nearly 40% of U.K. respondents (those who did not complete multiple follow-up surveys) were excluded from analysis, suggesting that survivorship bias might partially explain this discrepancy. We therefore sought to assess survivorship bias among participants in our longitudinal survey study as part of The COVID-19 Outbreak Public Evaluation (COPE) Initiative.

Methods: Survivorship bias was assessed in 4039 U.S. respondents who completed surveys including the assessment of mental health as part of The COPE Initiative in April 2020 and were invited to complete follow-up surveys. Participants completed validated screening instruments for symptoms of anxiety, depression and insomnia. Survivorship bias was assessed for (1) demographic differences in follow-up survey participation, (2) differences in initial adverse mental health symptom prevalence adjusted for demographic factors and (3) differences in follow-up survey participation based on mental health experiences adjusted for demographic factors.

Results: Adjusting for demographics, individuals who completed only one or two out of four surveys had significantly higher prevalence of anxiety and depression symptoms in April 2020 (e.g. one-survey v. four-survey, anxiety symptoms, adjusted prevalence ratio [aPR]: 1.30, 95% confidence interval [CI]: 1.08-1.55, p = 0.0045; depression symptoms, aPR: 1.43, 95% CI: 1.17-1.75, p = 0.00052). Moreover, individuals who experienced incident anxiety or depression symptoms had significantly higher adjusted odds of not completing follow-up surveys (adjusted odds ratio [aOR]: 1.68, 95% CI: 1.22-2.31, p = 0.0015, aOR: 1.56, 95% CI: 1.15-2.12, p = 0.0046, respectively).

Conclusions: Our findings reveal significant survivorship bias among longitudinal survey respondents, indicating that restricting analytic samples to only respondents who provide repeated assessments in longitudinal survey studies could lead to overly optimistic interpretations of mental health trends over time. Cross-sectional or planned missing data designs may provide more accurate estimates of population-level adverse mental health symptom prevalence than longitudinal surveys.

Keywords: Epidemiology; non-random attrition; non-response bias; research design and methods.

PubMed Disclaimer

Conflict of interest statement

All authors report institutional grants to Monash University from the C.D.C. Foundation, with funding from BNY Mellon, and from WHOOP, Inc. M.É.C. reported grants from the Fulbright Foundation sponsored by The Kinghorn Foundation and personal fees from Vanda Pharmaceuticals Inc. C.A.C. reported receiving personal fees from Teva Pharma Australia, Inselspital Bern, the Institute of Digital Media and Child Development, the Klarman Family Foundation, Tencent Holdings Ltd, the Sleep Research Society Foundation, and Physician's Seal; receiving grants to Brigham and Women's Hospital from the Federal Aviation Administration, the National Health Lung and Blood Institute U01-HL-111478, the National Institute on Aging P01-AG09975, the National Aeronautics and Space Administration and the National Institute of Occupational Safety and Health R01-OH-011773; receiving personal fees from and equity interest in Vanda Pharmaceuticals Inc.; educational and research support from Jazz Pharmaceuticals Plc, Philips Respironics Inc., Regeneron Pharmaceuticals and Sanofi S.A.; an endowed professorship provided to Harvard Medical School from Cephalon, Inc.; an institutional gift from Alexandra Drane; and a patent on Actiwatch-2 and Actiwatch-Spectrum devices, with royalties paid from Philips Respironics, Inc. C.A.C.'s interests were reviewed and managed by Brigham and Women's Hospital and Mass General Brigham in accordance with their conflict of interest policies. C.A.C also served as a voluntary board member for the Institute for Experimental Psychiatry Research Foundation and a voluntary consensus panel chair for the National Sleep Foundation. S.M.W.R. reported receiving grants and personal fees from Cooperative Research Centre for Alertness, Safety and Productivity, receiving grants and institutional consultancy fees from Teva Pharma Australia, and institutional consultancy fees from Vanda Pharmaceuticals, Circadian Therapeutics, BHP Billiton and Herbert Smith Freehills. No other disclosures were reported.

Figures

Fig. 1.
Fig. 1.
Crude and adjusted prevalence ratios for anxiety, depression and insomnia symptoms in April 2020 by number of completed surveys. The marker * indicates that p < 0.025 (i.e. the prevalence ratio is statistically significant).
Fig. 2.
Fig. 2.
Estimated prevalence of symptoms of anxiety, depression and insomnia in April 2020 based on total number of completes surveys, with each group weighted to population estimates for gender, age and race/ethnicity. The marker * indicates that p < 0.025 (i.e. the difference in prevalence estimates is statistically significant). The rounded, weighted percentages of respondents shown in Fig. 2. based on the number of completed surveys may differ from those reported in Table 1 due to different survey weight raking and trimming.
Fig. 3.
Fig. 3.
Longitudinal comparisons of anxiety and depression symptom prevalence by number of repeated measures.The marker * indicates that p < 0.025 within the same group over the timepoints designated with brackets (i.e. the prevalence estimates differ with statistical significance). The marker † indicates that p < 0.025 between groups at a single timepoint, with the comparison designated with brackets (i.e. the prevalence estimates differ with statistical significance). The marker ns indicates that p ≥ 0.025 (i.e. the prevalence estimates do not differ with statistical significance).
Fig. 4.
Fig. 4.
Odds of lower participation in follow-up surveys based on mental health in earlier surveys. The marker * indicates that p < 0.025 (i.e. the odds ratio is statistically significant).
See this image and copyright information in PMC

Update of

References

    1. Adams MJ, Hill WD, Howard DM, Dashti HS, Davis KAS, Campbell A, Clarke TK, Deary IJ, Hayward C, Porteous D, Hotopf M and McIntosh AM (2020) Factors associated with sharing e-mail information and mental health survey participation in large population cohorts. International Journal of Epidemiology 49, 410–421. - PMC - PubMed
    1. Batabyal S (2021) COVID-19: perturbation dynamics resulting chaos to stable with seasonality transmission. Chaos, Solitons & Fractals 145, 110772. - PMC - PubMed
    1. Batabyal S and Batabyal A (2021) Mathematical computations on epidemiology: a case study of the novel coronavirus (SARS-CoV-2). Theory in Biosciences. - PMC - PubMed
    1. Boldrini M, Canoll PD and Klein RS (2021) How COVID-19 affects the brain. JAMA Psychiatry. - PMC - PubMed
    1. Bonati M, Campi R, Zanetti M, Cartabia M, Scarpellini F, Clavenna A and Segre G (2021) Psychological distress among Italians during the 2019 coronavirus disease (COVID-19) quarantine. BMC Psychiatry 21, 20. - PMC - PubMed