How to know whether to believe a health study

Every day, new health care research findings are reported. Many of them suggest that if we do something — drink more coffee, take this drug, get that surgery or put in this policy — we will have better (or worse) health, or longer (or shorter) lives.

And every time you read such news, you are undoubtedly left asking: Should I believe this? Often the answer is no, but we may not know how to distinguish the research duds from the results we should heed...

Though a randomized trial makes two groups statistically identical to each other — apart from treatment received — it still doesn’t mean either group is identical to you. If the individuals selected to participate in the trial happen to be very similar to you — similar ages, income, living environment and so forth — that increases the chances that the results would apply to you. But if you’re, say, a 65-year-old, middle-class New Yorker, a study whose subjects were poor 30-somethings in rural China may not translate to your experience...

My colleague Aaron Carroll provided an example of just this problem. Based on the results of randomized trials that included only adults, prescriptions of drugs known as proton pump inhibitors to infants with gastroesophageal reflux disease grew sevenfold between 2000 and 2004. Only later, in 2009, a direct study of infants found that those drugs caused them harm, with no benefit...

People like you are more likely to be represented in a nonexperimental database study, so your top concern might be whether the findings are valid. After all, such a study doesn’t rely on the clean comparisons of randomized groups of people. Instead, it often compares groups of people who could have self-selected into receiving treatment or not. Maybe those who opted to receive it are systematically different — healthier, sicker, more careful, for example — and that’s what drives the findings. If so, what might appear causal isn’t, giving rise to the familiar “correlation does not imply causation.”

That concern is why researchers employ techniques to try to adjust for differences across comparison groups in nonexperimental studies. These can get complex in a hurry, and few news media reports could describe them in detail. But that doesn’t mean they’re all sketchy or all ironclad. The key fact is that they all rely on different assumptions than a randomized trial, and those assumptions can and should be probed to gain confidence in causal inferences.

Most news media reports acknowledge when a study is nonexperimental, and sometimes you can find a sentence or two about how the researchers sought to adjust for differences and tested assumptions. You should also look for statements from experts about whether those adjustments and tests were sufficient. However, these rely on judgment. There is always room for doubt.

Ultimately, no single study is perfect. Whether it’s a randomized trial or a nonexperimental one, one can never be absolutely sure study findings are valid and applicable to you. The best bet is to wait, if you can, until evidence accumulates from many studies using a range of methods and applied to different populations.

Few things are miracle cures, but when one shows up, we’ll see its signature in not just one study, but in many. Yes, that can take time. But if you want solid evidence you can count on, you cannot also be impatient.

http://www.nytimes.com/2015/08/18/upshot/how-to-know-whether-to-believe-a-health-study.html?partner=rss&emc=rss&abt=0002&abg=0&_r=0
Courtesy of:  http://www.medpagetoday.com/Psychiatry/Addictions/53111?isalert=1&uun=g906366d4438R5793688u&xid=NL_breakingnews_2015-08-18