Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2022 Apr;17(3):887-909.
doi: 10.1007/s11739-022-02967-1. Epub 2022 Mar 24.

Analysis of common methodological flaws in the highest cited e-cigarette epidemiology research

Cother Hajat  1 , Emma Stein  2 , Arielle Selya  3   4   5 , Riccardo Polosa  6   7   8 ; CoEHAR study group
Collaborators, Affiliations
Review

Analysis of common methodological flaws in the highest cited e-cigarette epidemiology research

Cother Hajat et al. Intern Emerg Med. 2022 Apr.

Erratum in

Abstract

The prevalence of vaping, also known as using e-cigarettes, vapes and vape pens, has prompted a demand for reliable, evidence-based research. However, published literature on the topic of vaping often raises concerns, characterized by serious flaws and a failure to adhere to accepted scientific methodologies. In this narrative review, we analyze popular vaping studies published in medical journals that purport to evaluate the association of vaping and smoking cessation, smoking initiation or health outcomes. We analyzed 24 included studies to identify the questions they claimed to address, stated methods, manner of implementation, discussions, and stated conclusions. After critical appraisal, we noted a multiplicity of flaws in these studies, and identified patterns as to the nature of such flaws. Many studies lacked a clear hypothesis statement: to the extent that a hypothesis could be inferred, the methods were not tailored to address the question of interest. Moreover, main outcome measures were poorly identified, and data analysis was further complicated by failure to control for confounding factors. The body of literature on "gateway" theory for the initiation of smoking was particularly unreliable. Overall, the results and discussion contained numerous unreliable assertions due to poor methods, including data collection that lacked relevance, and assertions that were unfounded. Many researchers claimed to find a causal association while not supporting such findings with meaningful data: the discussions and conclusions of such studies were, therefore, misleading. Herein, we identify the common flaws in the study design, methodology, and implementation found in published vaping studies. We present our summary recommendations for future vaping research. Our aim is to prompt future researchers to adhere to scientific methods to produce more reliable findings and conclusions in the field of vaping research.

Keywords: Critical Analysis; Electronic nicotine delivery systems (ENDS); Epidemiology; Vape pens; Vapes; e-cigarettes.

PubMed Disclaimer

Conflict of interest statement

CH received reimbursement from ECLAT for research conducted on tobacco harm reduction (2019–2020) including this article; she has served as a paid member of the advisory panel for the Tobacco Transformation Index (contracted by Sustainability, Sept 2019–April 2020); she is a paid consultant to TEVA pharmaceuticals on work related to multiple chronic conditions (2017 to present). ES received reimbursement from ECLAT for research conducted on tobacco harm reduction (2019–2020) including this article. AS is an employee of Pinney Associates, Inc., which provides consulting services on tobacco harm reduction to JUUL Labs, Inc. However, JUUL had no role in this study. RP is full tenured professor of Internal Medicine at the University of Catania (Italy) and Medical Director of the Institute for Internal Medicine and Clinical Immunology at the same University. In relation to his recent work in the area of respiratory diseases, clinical immunology, and tobacco control, RP has received lecture fees and research funding from Pfizer, GlaxoSmithKline, CV Therapeutics, NeuroSearch A/S, Sandoz, MSD, Boehringer Ingelheim, Novartis, Duska Therapeutics, and Forest Laboratories. Lecture fees from a number of European EC industry and trade associations (including FIVAPE in France and FIESEL in Italy) were directly donated to vaper advocacy no-profit organizations. RP has also received grants from European Commission initiatives (U-BIOPRED and AIRPROM) and from the Integral Rheumatology & Immunology Specialists Network (IRIS) initiative. He has also served as a consultant for Pfizer, Global Health Alliance for treatment of tobacco dependence, CV Therapeutics, Boehringer Ingelheim, Novartis, Duska Therapeutics, ECITA (Electronic Cigarette Industry Trade Association, in the UK), Arbi Group Srl., Health Diplomats, and Sermo Inc. RP has served on the Medical and Scientific Advisory Board of Cordex Pharma, Inc., CV Therapeutics, Duska Therapeutics Inc, Pfizer, and PharmaCielo. RP is also founder of the Center for Tobacco prevention and treatment (CPCT) at the University of Catania and of the Center of Excellence for the acceleration of HArm Reduction (CoEHAR) at the same University, which has received support from Foundation for a Smoke Free World to conduct 8 independent investigator-initiated research projects on harm reduction. RP currently involved in a patent application concerning an app tracker for smoking behavior developed for ECLAT Srl. RP is also currently involved in the following pro bono activities: scientific advisor for LIAF, Lega Italiana Anti Fumo (Italian acronym for Italian Anti-Smoking League), the Consumer Advocates for Smoke-free Alternatives (CASAA) and the International Network of Nicotine Consumers Organizations (INNCO); Chair of the European Technical Committee for standardization on “Requirements and test methods for emissions of electronic cigarettes” (CEN/TC 437; WG4).

Figures

Fig. 1
Fig. 1
Terminology related to assessing for causal associations between vaping exposures, behavior, and health outcomes
See this image and copyright information in PMC

References

    1. Centers for Disease Control. Tobacco: basic information. US Department of Health & Human Services. Bethesda, MD. https://www.cdc.gov/tobacco/basic_information/e-cigarettes/index.htm. Accessed 23 Apr 2021
    1. Hajat C, Stein E, Shantikumar S, Niaura R, Ferrara P, Polosa R. A scoping review of studies on the health impact of electronic nicotine delivery systems. Intern Emerg Med. 2021 doi: 10.1007/s11739-021-02835-4. - DOI - PMC - PubMed
    1. Bours MJL. A nontechnical explanation of the counterfactual definition of confounding. J Clin Epidemiol. 2020;121:91–100. doi: 10.1016/j.jclinepi.2020.01.021. - DOI - PubMed
    1. Alzahrani T, Pena I, Temesgen N, Glantz SA (2018) Association between electronic cigarette use and myocardial infarction [published correction appears in Am J Prev Med 2019 57(4):579–584]. Am J Prev Med. 55(4):455–461. 10.1016/j.amepre.2018.05.004. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6208321/ - PMC - PubMed
    1. Barrington-Trimis JL, Kong G, Leventhal AM et al (2018) E-cigarette use and subsequent smoking frequency among adolescents. Pediatrics. 142(6):e20180486. 10.1542/peds.2018-0486. https://pediatrics.aappublications.org/content/142/6/e20180486 - PMC - PubMed

Publication types