Good science depends on access to full data. A new study from the University of California at San Francisco highlights the point. In this instance, scientists wondered about the accuracy of peer-reviewed papers purporting to show relatively modest carcinogenic effects from tobacco additives. The earlier paper resulted from a study the tobacco industry called Project MIX.
So, how to figure out the truth about the results of Project MIX ? The UCSF scientists used research techniques to wade through 60 million pages of now-published tobacco industry documents obtained through litigation. The wading resulted in finding 500 key documents, including documents regarding Project MIX. The UCSF scientists analyzed the documents and the data. Guess what? Their conclusion is that the data had been manipulated when published. The manipulations, they say, lowered or hid the cancer-causing effects of numerous additives, including menthol.
The new paper also address a second question. How did the manipulated paper make it into peer-reviewed literature? Answer? The industry data was published in a journal for which they found myriad links to the tobacco industry. Indeed, according to the paper: "The scientist and leader of Project MIX Edward Carmines described the process of publication as "an inside job.""
The UCSF paper also is noteworthy for a third reason. The authors published in on PLoS so the full text of the article is freely available in full text. And, even better, the article is supported by online links to images of the papers they cite in their paper. So, it’s no longer a game of creating spin. Instead, the evidence is laid out for all to see.
What did the UCSF researchers conclude – here’s the short version:
"What Do These Findings Mean?
These findings show that the tobacco industry scientific research on the use of cigarette additives cannot be taken at face value: the results demonstrate that toxins in cigarette smoke increase substantially when additives are put in cigarettes. In addition, better powered studies would probably have detected a much broader range of adverse biological effects associated with the additives than identified to those identified in PM’s published papers suggesting that the published papers substantially underestimate the toxic potential combination of cigarette smoke and additives.
Regulatory authorities, including the FDA and similar agencies elsewhere who are implementing WHO FCTC, should conduct their own independent analysis of Project MIX data, which, analyzed correctly, could provide a strong evidence base for the elimination of the use of the studied additives (including menthol) in cigarettes on public health grounds."
In short, more reasons to wonder why modern studies are allowed to be published without putting full data online. One might also wonder why the tobacco industry is allowed to exist.
The full article is freely available in full text at this page of PLoS – the Public Library of Science. Set out below is a larger section of the key, summary portion of the article. But the full article is well worth reading. And, again, note that UCSF authors provide online links to actual, complete images of the underlying documents they cite in their paper. Now that’s good science !!
"What Did the Researchers Do and Find?
The researchers systematically examined tobacco industry documents in the University of California San Francisco Legacy Tobacco Documents Library (then about 60 million pages made publicly available as a result of litigation) and used an iterative process of searching, analyzing, and refining to identify and review in detail 500 relevant documents.
The researchers found that in the original Project MIX analysis, the published papers obscured findings of toxicity by adjusting the data by total particulate matter (TPM) concentration. When the researchers conducted their own analysis by studying additives per cigarette (as was specified in the original Project MIX protocol), they found that 15 carcinogenic chemicals increased by 20%. The researchers also reported that, for unexplained reasons, Philip Morris deemphasized 19 of the 51 chemicals tested in the presentation of results, including nine that were substantially increased in smoke on a per cigarette basis of additive-added cigarettes, compared to smoke of control cigarettes.
The researchers explored the possibility that the failure of Project MIX to detect statistically significant changes in the toxicity of the smoke from cigarettes containing the additives was due to underpowered experiments rather than lack of a real effect by conducting their own statistical analysis. This analysis suggests that a better powered study would have detected a much broader range of biological effects associated with the additives than was identified in Philip Morris’ published paper, suggesting that it substantially underestimated the toxic potential of cigarette smoke and additives.
The researchers also found that Food and Chemical Toxicology, the journal in which the four Project MIX papers were published, had an editor and 11 of its International Editorial Board with documented links to the tobacco industry. The scientist and leader of Project MIX Edward Carmines described the process of publication as “an inside job.”
What Do These Findings Mean?
These findings show that the tobacco industry scientific research on the use of cigarette additives cannot be taken at face value: the results demonstrate that toxins in cigarette smoke increase substantially when additives are put in cigarettes. In addition, better powered studies would probably have detected a much broader range of adverse biological effects associated with the additives than identified to those identified in PM’s published papers suggesting that the published papers substantially underestimate the toxic potential combination of cigarette smoke and additives.
Regulatory authorities, including the FDA and similar agencies elsewhere who are implementing WHO FCTC, should conduct their own independent analysis of Project MIX data, which, analyzed correctly, could provide a strong evidence base for the elimination of the use of the studied additives (including menthol) in cigarettes on public health grounds.