 |
James L. Holly, M.D. |
|
 |
|
|
|
|
|
|
|
|
| Adapted by James L. Holly, MD |
| October 18, 2007 |
| Your Life Your Health - The Examiner |
|
This is the last in the four-part series, "Do we really know what makes us healthy," which has been adapted from the New York Times Magazine.
"How Doctors Confound the Science
"Another complication to what may already appear (for good reason) to be a hopelessly confusing story is what might be called the prescriber effect. The reasons a physician will prescribe one medication to one patient and another or none at all to a different patient are complex and subtle. "Doctors go through a lot of different filters when they're thinking about what kind of drug to give to what kind of person," says Avorn, whose group at Harvard has spent much of the last decade studying this effect. "Maybe they give the drug to their sickest patients; maybe they give it to the people for whom nothing else works."
"It's this prescriber effect, combined with what Avorn calls the eager-patient effect, that is one likely explanation for why people who take cholesterol-lowering drugs called statins appear to have a greatly reduced risk of dementia and death from all causes compared with people who don't take statins. The medication itself is unlikely to be the primary cause in either case, says Avorn, because the observed associations are "so much larger than the effects that have been seen in randomized-clinical trials."
"If we think like physicians, Avorn explains, then we get a plausible explanation: "A physician is not going to take somebody either dying of metastatic cancer or in a persistent vegetative state or with end-stage neurologic disease and say, ‘Let's get that cholesterol down, Mrs. Jones.' The consequence of that, multiplied over tens of thousands of physicians, is that many people who end up on statins are a lot healthier than the people to whom these doctors do not give statins. Then add into that the people who come to the doctor and say, ‘My brother-in-law is on this drug,' or, ‘I saw it in a commercial,' or, ‘I want to do everything I can to prevent heart disease, can I now have a statin, please?' Those kinds of patients are very different from the patients who don't come in. The coup de grâce then comes from the patients who consistently take their medications on an ongoing basis, and who are still taking them two or three years later. Those people are special and unusual and, as we know from clinical trials, even if they're taking a sugar pill they will have better outcomes."
"The trick to successfully understanding what any association might really mean, Avorn adds, is "being clever." "The whole point of science is self-doubt," he says, "and asking could there be another explanation for what we're seeing."
"H.R.T. and the Plausibility Problem
"Until the HERS and W.H.I. trials tested and refuted the hypothesis that hormone-replacement therapy protected women against heart disease, Stampfer, Willett and their colleagues argued that these alternative explanations could not account for what they observed. They had gathered so much information about their nurses, they said, that it allowed them to compare nurses who took H.R.T. and engaged in health-conscious behaviors against women who didn't take H.R.T. and appeared to be equally health-conscious. Because this kind of comparison didn't substantially change the size of the association observed, it seemed reasonable to conclude that the association reflected the causal effect of H.R.T. After the W.H.I. results were published, says Stampfer, their faith was shaken, but only temporarily. Clinical trials, after all, also have limitations, and so the refutation of what was originally a simple hypothesis - that H.R.T. wards off heart disease - spurred new hypotheses, not quite so simple, to explain it.
"At the moment, at least three plausible explanations exist for the discrepancy between the clinical trial results and those of the Nurses' Health Study and other observational studies. One is that the associations perceived by the epidemiologic studies were due to healthy-user and prescriber effects and not H.R.T. itself. Women who took H.R.T. had less heart disease than women who didn't, because women who took H.R.T. are different from women who didn't take H.R.T. And maybe their physicians are also different. In this case, the trials got the right answer; the observational studies got the wrong answer.
A second explanation is that the observational studies got the wrong answer, but only partly. Here, healthy-user and prescriber effects are viewed as minor issues; the question is whether observational studies can accurately determine if women were really taking H.R.T. before their heart attacks. This is a measurement problem, and one conspicuous limitation of all epidemiology is the difficulty of reliably assessing whatever it is the investigators are studying: not only determining whether or not subjects have really taken a medication or consumed the diet that they reported, but whether their subsequent diseases were correctly diagnosed. "The wonder and horror of epidemiology," Avorn says, "is that it's not enough to just measure one thing very accurately. To get the right answer, you may have to measure a great many things very accurately."
"The most meaningful associations are those in which all the relevant factors can be ascertained reliably. Smoking and lung cancer, for instance. Lung Cancer is an easy diagnosis to make, at least compared with heart disease. And "people sort of know whether they smoke a full pack a day or half or what have you," says Graham Colditz, who recently left the Nurses' study and is now at Washington University School of Medicine in St. Louis. "That's one of the easier measures you can get." Epidemiologists will also say they believe in the associations between LDL cholesterol, blood pressure and heart disease, because these biological variables are measured directly. The measurements don't require that the study subjects fill out a questionnaire or accurately recall what their doctors may have told them.
"Even the way epidemiologists frame the questions they ask can bias a measurement and produce an association that may be particularly misleading. If researchers believe that physical activity protects against chronic disease and they ask their subjects how much leisure-time physical activity they do each week, those who do more will tend to be wealthier and healthier, and so the result the researchers get will support their preconceptions. If the questionnaire asks how much physical activity a subject's job entails, the researchers might discover that the poor tend to be more physically active, because their jobs entail more manual labor, and they tend to have more chronic diseases. That would appear to refute the hypothesis.
"The simpler the question or the more objective the measurement the more likely it is that an association may stand in the causal pathway, as these researchers put it. This is why the question of whether hormone-replacement therapy effects heart-disease risk, for instance, should be significantly easier to nail down than whether any aspect of diet does. "For a measurement "as easy as this," says Jamie Robins, a Harvard epidemiologist, "where maybe the confounding is not horrible, maybe you can get it right." It's simply easier to imagine that women who have taken estrogen therapy will remember and report that correctly - it's yes or no, after all - than that they will recall and report accurately what they ate and how much of it over the last week or the last year.
"But as the H.R.T. experience demonstrates, even the timing of a yes-or-no question can introduce problems. The subjects of the Nurses' Health Study were asked if they were taking H.R.T. every two years, which is how often the nurses were mailed new questionnaires about their diets, prescription drug use and whatever other factors the investigators deemed potentially relevant to health. If a nurse fills out her questionnaire a few months before she begins taking H.R.T., as Colditz explains, and she then has a heart attack, say, six months later, the Nurses' study will classify that nurse as "not using" H.R.T. when she had the heart attack.
"As it turns out, 40 percent of women who try H.R.T. stay on it for less than a year, and most of the heart attacks recorded in the W.H.I. and HERS trials occurred during the first few years that the women were prescribed the therapy. So it's a reasonable possibility that the Nurses' Health Study and other observational studies misclassified many of the heart attacks that occurred among users of hormone therapy as occurring among nonusers. This is the second plausible explanation for why these epidemiologic studies may have erroneously perceived a beneficial association of hormone use with heart disease and the clinical trials did not.
"In the third explanation, the clinical trials and the observational studies both got the right answer, but they asked different questions. Here the relevant facts are that the women who took H.R.T. in the observational studies were mostly younger women going through menopause. Most of the women enrolled in the clinical trials were far beyond menopause. The average age of the women in the W.H.I. trial was 63 and in HERS it was 67. The primary goal of these clinical trials was to test the hypothesis that H.R.T. prevented heart disease. Older women have a higher risk of heart disease, and so by enrolling women in their 60s and 70s, the researchers didn't have to wait nearly as long to see if estrogen protected against heart disease as they would have if they only enrolled women in their 50s.
"This means the clinical trials were asking what happens when older women were given H.R.T. years after menopause. The observational studies asked whether H.R.T. prevented heart disease when taken by younger women near the onset of menopause. A different question. The answer, according to Stampfer, Willett and their colleagues, is that estrogen protects those younger women - perhaps because their arteries are still healthy - while it induces heart attacks in the older women whose arteries are not. "It does seem clear now," Willett says, "that the observational studies got it all right. The W.H.I. also got it right for the question they asked: what happens if you start taking hormones many years after menopause? But that is not the question that most women have cared about."
"This last explanation is now known as the "timing" hypothesis, and it certainly seems plausible. It has received some support from analyses of small subsets of the women enrolled in the W.H.I. trial, like the study published in June in The New England Journal of Medicine. The dilemma at the moment is that the first two explanations are also plausible. If the compliance effect can explain why anyone faithfully following her doctor's orders will be 50 percent less likely to die over the next few years than someone who's not so inclined, then it's certainly possible that what the Nurses' Health Study and other observational studies did is observe a compliance effect and mistake it for a beneficial effect of H.R.T. itself. This would also explain why the Nurses' Health Study observed a 40 percent reduction in the yearly risk of death from all causes among women taking H.R.T. And it would explain why the Nurses' Health Study reported very similar seemingly beneficial effects for antioxidants, vitamins, low-dose aspirin and folic acid, and why these, too, were refuted by clinical trials. It's not necessarily true, but it certainly could be.
"While Willett, Stampfer and their colleagues will argue confidently that they can reasonably rule out these other explanations based on everything they now know about their nurses - that they can correct or adjust for compliance and prescriber effects and still see a substantial effect of H.R.T. on heart disease - the skeptics argue that such confidence can never be justified without a clinical trial, at least not when the associations being studied are so small. "You can correct for what you can measure," says Rory Collins, an epidemiologist at Oxford University, "but you can't measure these things with precision so you will tend to under-correct for them. And you can't correct for things that you can't measure."
"The investigators for the Nurses' Health Study "tend to believe everything they find," says Barrett-Connor of the University of California, San Diego. Barrett-Connor also studied hormone use and heart disease among a large group of women and observed and published the same association that the Nurses' Health Study did. She simply does not find the causal explanation as easy to accept, considering the plausibility of the alternatives. The latest variation on the therapeutic wisdom on H.R.T. is plausible, she says, but it remains untested. "Now we're back to the place where we're stuck with observational epidemiology," she adds. "I'm back to the place where I doubt everything."
"What to Believe?
"So how should we respond the next time we're asked to believe that an association implies a cause and effect, that some medication or some facet of our diet or lifestyle is either killing us or making us healthier? We can fall back on several guiding principles, these skeptical epidemiologists say. One is to assume that the first report of an association is incorrect or meaningless, no matter how big that association might be. After all, it's the first claim in any scientific endeavor that is most likely to be wrong. Only after that report is made public will the authors have the opportunity to be informed by their peers of all the many ways that they might have simply misinterpreted what they saw. The regrettable reality, of course, is that it's this first report that is most newsworthy. So be skeptical.
"If the association appears consistently in study after study, population after population, but is small - in the range of tens of percent - then doubt it. For the individual, such small associations, even if real, will have only minor effects or no effect on overall health or risk of disease. They can have enormous public-health implications, but they're also small enough to be treated with suspicion until a clinical trial demonstrates their validity.
"If the association involves some aspect of human behavior, which is, of course, the case with the great majority of the epidemiology that attracts our attention, then question its validity. If taking a pill, eating a diet or living in proximity to some potentially noxious aspect of the environment is associated with a particular risk of disease, then other factors of socioeconomic status, education, medical care and the whole gamut of healthy-user effects are as well. These will make the association, for all practical purposes, impossible to interpret reliably.
"The exception to this rule is unexpected harm, what Avorn calls "bolt from the blue events," that no one, not the epidemiologists, the subjects or their physicians, could possibly have seen coming - higher rates of vaginal cancer, for example, among the children of women taking the drug DES to prevent miscarriage, or mesothelioma among workers exposed to asbestos. If the subjects are exposing themselves to a particular pill or a vitamin or eating a diet with the goal of promoting health, and, lo and behold, it has no effect or a negative effect - it's associated with an increased risk of some disorder, rather than a decreased risk - then that's a bad sign and worthy of our consideration, if not some anxiety. Since healthy-user effects in these cases work toward reducing the association with disease, their failure to do so implies something unexpected is at work.
"All of this suggests that the best advice is to keep in mind the law of unintended consequences. The reason clinicians test drugs with randomized trials is to establish whether the hoped-for benefits are real and, if so, whether there are unforeseen side effects that may outweigh the benefits. If the implication of an epidemiologist's study is that some drug or diet will bring us improved prosperity and health, then wonder about the unforeseen consequences. In these cases, it's never a bad idea to remain skeptical until somebody spends the time and the money to do a randomized trial and, contrary to much of the history of the endeavor to date, fails to refute it.
"Gary Taubes is the author of the forthcoming book "Good Calories, Bad Calories: Challenging the Conventional Wisdom on Diet, Weight Control and Disease."
|
|
|
|
|
|
|
|
|
|
|
