Thursday, July 12, 2012

Confirmation Bias: We have it.

If you spend a lot of time skulking around nutrition and weight loss blogs like I do you probably came across this article or various comments on this article:  Effects of Dietary Composition on Energy Expenditure During Weight-Loss Maintenance.

Now being a good SkepticRD, I am going to pause and remind all of you out in reader land to actually read the article, and not just assume SkepticRD's authority. Yes, read all of it.  I will wait.

So now that you've read it (right?) one of the things that likely popped out at you was "resulted in decreases in REE and TEE that were greatest with the low-fat diet, intermediate with the low–glycemic index diet, and least with the very low-carbohydrate diet."  In other words, people on the very low carb diet actually burned more calories sitting around existing than people on the other two diets.

If you skulked around the low carb blogs, the degree of, dare I say, smugness was overwhelming.  "Of course we already knew that" is a basic summary of of the statements circulating.  But SkepticRD is an all-around skulker, so of course I witnessed a certain amount of backlash from the more "conventional" community.  One particular article that caught my eye was this one: In Dieting, Magic Isn’t a Substitute for Science.

I was hoping that the researcher interviewed for this article would talk about the limitations of the study, because there always are limitations, things we don't know, possibilities for future research, etc.  That's what good scientists do, and there was the very positive "Dr. Hirsch, who receives no money from pharmaceutical companies or the diet industry" indicating that there would be a limit to potential bias. Instead, there were some indications of our evil nemesis--confirmation bias.

Let me take a little diversion here to review what confirmation bias is.  As defined in the The Skeptic's Dictionary:  
Confirmation bias refers to a type of selective thinking whereby one tends to notice and to look for what confirms one's beliefs, and to ignore, not look for, or undervalue the relevance of what contradicts one's beliefs.
So, in other words if you believe or accept a certain principle of weight loss, you might tend to dismiss any alternative solutions.

Where did SkepticRD see confirmation bias in this article?  My initial suspicion was when he started talking about the water weight loss on low carb diets, and then subsequently mentioned that "he didn't know how they did the calculations."  I wondered if this gentleman had actually read the study, as it was clear to me that all the participants were put on the same plan to LOSE weight, but they were put on different plans to MAINTAIN the weight. Also, the article also states that they used a process called indirect calorimetry to measure their REE, and this process should be quite familiar to someone who should have studied obesity for many years.  Despite these things, he still dismissed the weight loss benefits associated with a low carb plan. 

Then of course came the dreaded "you need to just eat less to lose weight" statement.  If you are, or have been, overweight or obese you know the dread that I'm talking about.  Especially if you have already tried that more than once only to not see any results.  Now, SkepticRD does not see results with her patients, she knows that a good scientist knows that it's time to try something different when you are not getting results, and to keep repeating the same info over and over is a definite indication of confirmation bias.

Now SkepticRD is certainly not denying the laws of physics, as there does have to be some kind of caloric deficit to lose weight.  But if SkepticRD had been asked to discuss the limitations of the study, I would have addressed the following:
1) This study used a small sample size.  I can see why they did, it costs quite a bit of money to house and feed people for any length of time.  But usually small sample sizes are great "starter" research and may promote enough to say "Hey, we need to take a second, or third, or fourth look at this issue" but you don't always have statistical significance.
2) Laboratory settings are great for losing weight and maintaining it on any type of diet since your environment is controlled.  Even though the low carb plan showed that there are some physiological factors (reduced hunger, etc) that can help people maintain weight loss, this study didn't address (once again, for factors of time) what happens when you leave the laboratory setting and you are back in whatever crazy environment that you were in before that may have contributed to your weight problem?
3) Ok, hang on low carb people, yes, there were plenty of good results, but what about that not so little problem with the elevated cortisol levels in the low carb group?  You know, that lovely little steroid hormone that can actually lead to weight gain around your midsection and a suppressed immune system?  Not everybody stays at the 10% carb level after more than a few weeks, but what of those who do?  Could you actually be setting yourself up for weight gain?  Could you be setting yourself up for increased vulnerability to illness?  Was it the type/composition of fats in this particular version that caused it?  Did they look at the participants sleep quality?  Did they look at caffeine consumption?  And also, if you have had good results with a low carb plan, can you avoid confirmation bias yourself?

The take away message here is that even if something seems to match your previous assumptions about nutrition or health information, you need to read the fine print and see if there are additional problems that you need to watch out for.  And if you are reading a study that seems to go against what you have accepted, make sure that you take some time to read it more than once before you dismiss it.  And don't take it personally, as it's about the science and the scientific process versus keeping a personal agenda.  And, I would add, it's about making sure the people you are suppossed to be helping (including yourself!) get the help they need.

No comments:

Post a Comment