Is the e-cigarette research reliable?

A team of international researchers analyzed the 24 most popular studies on vaping, in order to identify the potential errors they include. And these are many.
The most popular searches scrutinized

Is scientific research on vaping shoddy? This is the question posed by a team of international scientists, led by Professors Cother Hajat, of the United Arab Emirates University, and Riccardo Polosa, founder of the Center of Excellence for the Acceleration of Harm Reduction (CoEHAR) of the University of Catania. Together, the researchers reviewed (1) the 24 most popular studies on vaping, with the aim of answering 4 questions:

• Does the study clearly describe the method of investigating causal links?
• Were the study design and research methods robust enough to control for confounders?
• Do the results support the stated conclusions, without exaggeration?
• Are researchers presenting misleading language or data, or failing to recognize important limitations?

In order to identify the most popular works, the team of scientists used the Google Scholar search engine, which brought out the most read and most cited studies for the terms e-cigarette, electronic cigarette, vaping and electronic nicotine delivery system (ENDS).

The identification was later refined to only focus on behavioral studies on human subjects regarding vaping-related causation claims. A first researcher then went through the articles in the order of ranking of the search results and identified the ten most frequently cited articles on each of the following topics:

• The effects of vaping on quitting/reducing smoking.
• The effects of vaping on smoking initiation.
• Health outcomes associated with vaping.

Afterwards, the selected works were consulted by a second researcher to confirm that they could indeed be included in the analysis. Disagreements between the 2 reviewers were resolved by consensus or by decision of a third researcher. Additional studies were then studied in the same way until a total of 10 studies were identified.

Professor Polosa's team explains that they used the Google Scholar search engine instead of another such as PubMed, for example, because it would better reflect the research methods used by policy makers, advocacy groups, service providers health care, and finally the patients.

Following the identification of this work, an initial search made it possible to obtain the titles of articles which, after manual review by a scientist from the team, were deemed not to meet the search criteria. . Specifically, articles excluded were those dealing with descriptive epidemiology, chemistry and toxicology, acute responses to exposure, and analytical articles that were not empirical.

In total, the researchers ultimately conducted a review and critical appraisal of 24 of the most popular papers on causation claims related to vaping.

A first researcher then set about grouping each of the 24 articles according to the subject he was dealing with. A second carried out their critical evaluation and then produced a report on each of them, highlighting the strengths and limitations. Then, a discussion was finally started with all the members of the team until a consensus was reached for each study analyzed.

Finally, the team of scientists explains that they contacted the authors of each of the studies reviewed to share their critical assessment, while asking them to identify potentially erroneous interpretations, and to discuss any other methodological limitations or strengths. of their work. Riccardo Polosa's team indicates that they have always received constructive advice from them, which has improved the quality of the analyzes and the final evaluation of each research.

After analyzing the 24 studies, the researchers came up with this table, which highlights common errors in study design, methodology and implementation. In the words of the team of scientists, the 24 articles studied were riddled with flaws.

Vaping and smoking cessation

On this subject, the critical evaluation of scientists would have revealed many shortcomings.

For example, researchers often assess the probability of success of a given quitting method, while incorrectly assuming that the number of quitting attempts is fixed. According to the Polosa team, education in a new quit strategy can prompt additional quit attempts. Thus, the quit method (e.g., vaping) deserves credit for prompting a new quit attempt.

In addition, several researchers allegedly failed to clearly indicate the causal link they were studying. For example, if a person has successfully attempted to quit smoking and would not have been able to do so had they not vaped, then vaping is the cause of the quit.

However, there are other potential causal pathways that the researchers would not have explored. Consider the case of a person who would not otherwise have attempted to quit smoking, but does so (and succeeds) because the option of vaping motivates them. This second pathway includes both intentional quit attempts and involuntary quits (dubbed "accidental quits"), i.e. a person who smokes tries to vape without intending to switch modes. of consumption, but finds it so attractive that she changes her mode of consumption.

The inclusion and exclusion criteria were also often incorrect. For example, a given population may consist of many former smokers who have successfully attempted to quit smoking by switching to vaping. By conducting a study on such a population, but only including current smokers, the researchers will only assess those who are already less likely to quit smoking by switching to vaping (as suggested by the fact that other did, and they didn't).

Additionally, excluding ex-smokers who successfully quit while vaping creates a biased participant population. In the team's analysis of studies about the effects of vaping on smoking cessation, many researchers overlooked these trends.

Finally, epidemiological studies assessing population trends can note the incidence and prevalence of vaping and smoking. However, these studies generally lack the specificity necessary to establish a causal association between vaping and smoking cessation. Additionally, the incidence and prevalence of vaping-related behaviors tend to fluctuate due to a confluence of variables, such as changes in technology, marketing, and media coverage, the authors explain.

The gateway theory, or the myth that vaping leads to smoking

This theory was addressed in 11 studies retained by the researchers. A theory they immediately point out is often asserted as if it were proven by data, when it is not.

Regarding the studies on this subject, the researchers point out different shortcomings, in particular concerning the fact that they did not have solid research methods and, therefore, could not reliably establish a causal link or identify a gateway effect. .

The scientists indicate that health behaviors related to vaping and smoking have been described in insufficient detail with respect to the duration, amount and frequency of vaping and smoking. This lack of precision would thus make the classification of participants uninformative and the resulting data unreliable.

Further, they also note that the propensity to start smoking in the absence of vaping is also poorly established and as such is a questionable measure. A fact that scientists find particularly troubling when claims of a gateway effect can be exploited, unsubstantiated, to raise concerns about other risky behaviors, such as illicit drug use.

The authors also point out that it is important to classify the participants according to their preference for nicotine. According to them, about half of the population likes being under the influence of nicotine and the other half does not. This variation alone would guarantee significantly higher tobacco consumption among vapers (and vice versa). Thus, people who do not vape, smoke or use any other tobacco product will inevitably start using such a product less often than users of tobacco products.

Finally, they conclude by indicating that the studies analyzed did not control for confounding factors or use methods designed to take into account the heterogeneity of participants. As such, the limitations were significant and the researchers could not make credible causal claims.

The effects of vaping on health

Here again, the authors of the analysis indicate that the 4 articles selected have many shortcomings. For example, researchers have attempted to assess the non-acute effects of vaping in a population of ex-smokers without recognizing an inherent limitation: the clinical features characteristic of this population include the consequences of past smoking, which mask the non-acute effects of vaping.

In such a population, they thus indicate that it is difficult to determine whether the morbidity and mortality results are attributable to the fact of vaping or to previous smoking. It is a major design flaw to ignore current, past, and dual use of cigarettes, ignoring that the majority of vapers do so to quit or cut down on cigarette smoking.

They add that although up to 70% of study participants reported being dual users, i.e. vapers who continued to smoke at the same time, 32 studies reportedly did not systematically consider dual use when examining the risk of vaping, thus attributing health outcomes to vaping when they could in fact be due to smoking cigarettes.

In addition, not all of the studies analyzed would have correctly classified the populations they studied, thus categorizing participants as simply smokers, former smokers, or non-smokers. Without taking into account the duration of smoking, the time elapsed since smoking cessation, or the frequency and quantity of tobacco consumed. Thus, a person smoking one cigarette per week was classified as a smoker, in the same way as another person smoking 3 packs per day. This greatly skews the results.

These flaws were found consistently in each of the four articles analyzed, making their findings misleading, the review team said.

They conclude, “Critical review of the included literature revealed many shortcomings, and the limitations clearly outweigh the strengths.”

This extensive analysis was partly funded by the Foundation for a Smoke-Free World Inc, i.e. the tobacco company Philip Morris International (PMI). Its authors indicate, however, that the content, selection and presentation of facts, as well as the opinions expressed in this document are the sole responsibility of the authors and can in no way be taken to reflect the positions of the foundation.