Validity of gene-by-environment studies questioned
In the eternal discussion between nature or nurture, as to which has greater input on health and well-being, the answer may well be… both.
Scientists generally believe that there is a physical interaction between genetics and the environment that may be identified down to the specific gene. Over the past decade, numerous studies have been done and published in leading journals to pinpoint the exact effect of these interactions.
But when Harvard School of Public Health post-doctoral fellow Laramie Duncan, Ph.D., began to look closely at the literature in preparation for initiating a gene-by-environment interaction study, she started to have questions about the validity of the studies and whether the publication of the studies could be affected by a false positive bias.
So, rather than a study of gene-environment interaction, she and coauthor Matthew C. Keller, Ph.D., of the University of Colorado, decided to undertake a review of the past decade of gene environment interaction studies. The resulting paper, published in September online ahead of print publication in the American Journal of Psychiatry, showed a widespread positive bias and a notable lack of size in the research studies.
In finding evidence of positive bias in publication of the studies, Duncan, who is also a clinical and research fellow at McLean Hospital and Massachusetts General Hospital, has opened up a can of worms. How many of these tests are correct and how do we prove it if this bias exists? Does it follow that this bias exists across the spectrum of research?
While she could not answer for other fields, Duncan spoke to New England Psychologist’s Catherine Robertson Souter about her work and the troubling results that they found.
Q: Why did your study cover 10 years?
A: That was when the first candidate gene environment interaction study was published and 10 years seemed like a comprehensible unit of time to focus on for the review.
We are relatively certain that interactions between genes and environment exist and do influence things like depression and anxiety and other psychiatric disorders. The difficulty is in identifying which genes are involved with which environmental variables.
I was originally planning on conducting one of these types of studies and when I did the literature review, I started noticing what appeared to be concerning trends. I talked about what I was seeing with colleagues and I was curious enough about it and decided to subject those trends to formal statistical analysis.
Q: What did you find?
A: First, there appears to be a high degree of publication bias in the field. In general, it also appears that these studies were too small. Lower power makes it difficult to identify real effects even if they are there. The combination of these two things made it possible that many, or most … or even all … the findings in this literature were false positives.
Q: How do you define publication bias?
A: What we were talking about was specifically positive publication bias. When researchers find positive results those are more likely to be published than when they find no results. People are more interested in positive findings. It can be more difficult to get no findings published.
I’m hopeful that we will identify gene environment interactions that everyone agrees are real. It may take large studies to do that. Some of the results that are reported may be correct. We just need to do more replication studies and we need to see all the results of the replication studies to find if it is true.
Q: What types of studies did you look at in your review?
A: The interaction we found studied most often was an interaction that was said to exist between a seratonin-related gene and stressful life events impacting depression. The results for follow-up studies of that interaction seem to indicate that the interaction is not actually real but we found evidence of publication bias among the replication studies. That was one of interest that we focused on in detail in our paper.
Q: So the problem was found even in replication studies? If what we need is more replication studies, how can we know if they are still going to be positively biased?
A: That’s where there was an even more concerning problem – the publication bias among replication attempts. In general, researchers expect there to be some publication bias among novel reports. But we’d hope that when people conducted replication attempts all the results would get published so that we could really figure out whether or not the effect was real. If there is publication bias towards positive results in the replication attempts, then that makes it harder to judge whether or not the findings are real.
Q: It seems that there must be far too many variables to track down a specific gene environment interaction.
A: That is one of the biggest issues. There are so many variables. For example, when we try to find direct effects of genetic variants on these phenotypes, there are millions of possibilities for what those genetic variants could be and potentially millions of options for gene- environment interactions as well. So it is really hard to hypothesize which would be the correct ones. On top of that, at least for direct effects of genetic variants on these disorders, we know that the effects are very, very tiny. They are not like the genetic effects that impact diseases like Huntington’s Disease. Instead there are tiny, little effects scattered throughout the entire genome.
Q: Why do we need to be so specific, to know exactly where the effect is taking place?
A: We might be able to discover which people are at greater risk by exposure to certain environmental variables. This could potentially have a lot of different benefits, including things like helping us better understand the mechanics underlying these disorders and potentially that could be used to help develop novel treatments or perhaps we could learn what types of cognitive or talk therapy interventions might be more useful for some people than for others.
Q: Then you would have to look at everyone’s genes up close to see what things would help people?
A: Yes, you would. But there is also a possibility of being able to better understand the mechanisms underlying these disorders and potentially developing a better treatment that may be generally applicable to a certain class of disorders.
Q: So we would take it from the individual and his genes and expand it back out to how it would affect everyone with this disorder?
A: Right. Maybe if we identify the specific collection of genes that are important for a certain disorder that might lead us to understanding the biological mechanism underlying the disorder.
Q: What kind of response have you gotten since your paper was published?
A: For the most part, response has been positive. A lot of people have said they suspected similar things and so the results of our study made sense to them. One journal has used the results from our paper, in addition to results from other researchers, to issue a new editorial policy about types of papers willing to publish in their journal. It was used to raise the bar in terms of this type of study for future research.