Once again, the flu season is well underway and there is no shortage of commercials, posters, billboards, and doctors telling us to get our flu shot. But is the flu shot
really your best shot at preventing the flu? In part 1 of this series on flu prevention I hope to prove to you that the flu vaccine isn't as impressive as we have been led to believe. In the next post(s) I will discuss what has been proven in the research to naturally prevent the flu.
I know you're not all scientists, but a quick overview of the different forms of scientific research is necessary to fully appreciate the second half of this article. There are many types of research articles out there, and they all have varying degrees of abundance and credibility in the scientific community. For a more detailed description and explanation please see
this website. For now all you need to know is that not all research is created equal. The most credible is the meta-analysis, followed by systemic reviews, randomized double-blind placebo studies, then cohort studies. Most of the research on the flu vaccine tends to be cohort studies, which are decent, but can still have a lot of flaws. Randomized control trials eliminate a lot of bias, but may still have issues as we will discuss below. Meta-analyses are considered to be the best because of their rigorous inclusion criteria. These studies compile information from many different articles, and more importantly, only the best articles make it into a meta-analysis.
As you will soon see, there are many different issues surrounding the debate of Influenza Vaccine Effectiveness. Issues like cohort (study group) selection bias, non-specific end points, and proper diagnosis of the flu can all stand in the way of an otherwise decent study, and will undoubtedly alter the reported numbers. Because it seems to be the most commonly recognized confounding variable in cohort studies, I will focus briefly on selection bias.
The term "
selection bias" is a little bit of a misnomer in this case, because it is seen in cohort studies where the groups are not actually "selected" by the scientists. Rather, these studies follow two groups of people (those who got the vaccine and those who did not) and use the information to estimate how effective the vaccine is. The problem is that it's nearly impossible to account for other differences between the two groups of people. Factors such as health status, belief or disbelief in vaccine effectiveness, vaccine availability and possibly other unknown variables may all lead to the difference in
behavior studied (i.e. the behavior to get the vaccine or to choose to not be vaccinated).
A great deal of skeptisism comes from what scientists are calling the "healthy user" effect, which means that the people who are getting vaccinated tend to be healthier, and therefore less likely to get the flu or die from it, than those who do not get the vaccine. This is the image of the ideal patient- someone who listens to their doctors advice; weather it be to eat more fiber, exercise, or get a vaccine [4]. One study examined this more in depth and compared several variables between the two groups: age, sex, the log of the insurance risk score, self-assessed health status, and presence of chronic diseases such as diabetes and heart disease. This study found a strong correlation between insurance risk score and self-assessed health status with risk of death, but more importantly they found a strong negative correlation between risk of death (poorer health prior to the flu season) and vaccine coverage. In other words, the more frail people, those who were the most likely to die with or without the flu, were among the least likely to get the flu shot, which undoubtedly skews the results of cohort studies in favor of higher vaccine effectiveness [5].
Aside from the patient's conscious choice there are also other factors that may create this effect. For example, vaccination is generally not recommended for the extremely frail or immuno-comprimised, a group that would be expected to have naturally higher hospitalization and death rates, who then get lumped into the "unvaccinated" group in a simple cohort study [4]. This is called a "frailty selection" [1]. In placebo controlled studies, it is considered unethical to include high-risk patients in such studies (it's not nice to give someone a placebo when there is a higher likelihood that the vaccine can save their life), so there is absolutely no high-quality (double blind, placebo controlled) data on the group that is theorized to benefit from vaccination the most [2].
In their paper, Simonsen et al [1] outlined criteria that can be used to detect such bias. The authors note, however, that it is easier to detect bias than to account for it and calculate exactly how much of the results are due to bias.
Seasonality. Think about it, the vaccine's ability to prevent death should be seasonal- only during the flu season when the virus is actually circulating. If there is a benefit outside of the flu season when the virus is not
circulating, this is a strong indicator of a difference between the two
groups. Several studies have identified a similar effect on all-cause mortality and hospitalization both during flu season and the off season, indicating an intrinsic difference between the ultimately immunized and non-immunized cohorts [1,6,7]. In 2008 Eurich et al demonstrated a 51% reduction in all cause mortality in pneumonia patients outside the flu season in those who would eventually get the flu shot- a number that is strikingly similar to what other groups report during the flu season [6]. The authors go on to say that this indicates that some
or all of the benefit in mortality that is seen in cohort studies is likely due to confounding variables such as the healthy user effect [1].
Vaccine match. Each year the flu virus mutates, creating the supposed need to get the updated yearly flu shot. Each summer and fall, scientists have to try to predict which flu virus will be the predominant strain to make a vaccine against it. Some seasons end up being better matches than others. Such information can be found on the CDC website [8]. Logically, the effectiveness of the vaccine should fluctuate with how well the vaccine is matched to the circulating strain. However, this does not appear to be the case [1,4]. Below is a chart from a 2012 meta-analysis (remember, this means that the studies they use should be the best of the best) that shows how well the vaccine is matched and effectiveness [1]. As you can see, most of the studies from the years that were reported as being mismatched were reported to have the same
or better effectiveness than other years. How can that possibly be?
The use of specific end points is important for obtaining accurate, clean results in a study. The more specific your end point (the thing you're measuring), the better the data you can obtain. Three things that are commonly used to describe flu vaccine effectiveness are all cause mortality, Influenza-like illness (someone says they have the flu), and laboratory confirmed influenza. Since people die from a lot of different things, even during the flu season, we would expect the vaccine's effectiveness at preventing all cause mortality to be relatively low. On the flip side, since the specific goal of the vaccine is to prevent the real flu (as opposed to ILI which may be the flu virus or something else), we would expect the vaccine's effectiveness here to be highest. On the contrary, current cohort evidence shows vaccine effectiveness estimates are highest (and implausibly large) for all cause mortality and lowest for laboratory confirmed influenza [1].
Cohort studies are most prone to bias because of the healthy-user effect, but even placebo controlled trials have their problems. Not only is the specificity of the end point being measured a concern, but the way those results are obtained may be an issue even in placebo controlled studies. Serology confirmed or laboratory confirmed influenza has been reported to skew results in favor of exaggerating vaccine effectiveness [2,4]. Culture confirmed influenza or better, vaccine-matched culture confirmed influenza seem to be the best in this regard, but the research in this area is sparse. Most studies look at all cause mortality, and many of the others look at laboratory confirmed infection, neither of which will give entirely accurate results.
Several authors have noted that the repeatedly reported 50% effectiveness against all cause mortality is simply not realistic. The flu only accounts for about 5% of all deaths in the winter, so "that the influenza vaccine can prevent ten times as many deaths as the disease itself causes is not plausible"[1]. Because influenza only contributes a small percentage to all cause mortality, it cannot be reasonably expected to do more than eliminate this excess. As I will show below, the real flaw lies in how that number, 50% is used, but there is another point to be made here. Several studies in the United States have tried to evaluate weather there has been a decrease in all cause mortality with increasing vaccination rates. In the last twenty years, since the flu vaccination rate among the elderly has soared from 15% to 65% there has not been a single study to has documented any change in hospital admission rates nor all cause mortality [6]. Similarly, studies failed to show an increase in mortality during the 1997-98 season, during which the vaccine was completely mismatched with the circulating strain [1].
Okay, here comes the really crazy part of this whole mess. For the time being, let's put aside all the stuff I just talked about and focus on the numbers- we'll assume that the data the studies are getting is actually reflective of what the vaccine is doing and that none of it is from confounding variables. We're not all statisticians here, but I think we can all do some basic 8th grade math. Let's solve this word problem:
100 people get the flu vaccine. Two of them get the flu. 100 other people do not get the vaccine. Four of them get the flu. How effective is the vaccine?
Okay, pencils down. Most of us would think through this problem like so, "We need to compare what percentage of each group got the flu. 2 percent versus 4 percent of each group got the flu. The vaccine was therefore 2% effective at preventing the flu. In other words, for every 100 people who got the flu vaccine, two less got the flu." What we just did was we calculated the absolute difference between the two groups- the percent difference. This is how people generally perceive percentages- we like to think of the number of people who's life would have been changed with the intervention. However, this is not how vaccine effectiveness is calculated. These studies calculate (and then report to us, the lay people) the relative difference between the two numbers. This is what it would be like for the example above, "We need to compare what percentage of each group got the flu. 2 percent versus 4 percent got the flu. 4 is twice as big as 2. The vaccine is therefore 50% effective." Let that logic soak in for a minute or so. Yes, this is a perfectly legitimate way to compare these two numbers, two and four, but exactly what that percentage is comparing is rarely, if ever explained on flu vaccine posters. No wonder they're (and we) all are getting so confused! I think this is incredibly misleading. Let's look at the numbers from the studies cited by that 2012 meta-analysis again and compare the absolute values to their inflated relative values:
The way I see it, there has been an unintentional miscommunication between the science (research) world and the consumer world. We tend to see things how they relate to ourselves and we like to apply numbers to real life. I think when people see that the flu vaccine is 50% effective, they imagine it as "if I was 100% at risk, now that risk is only 50% which is pretty good". The reality is that most people's odds of getting the flu are already pretty low (2.73% odds without the vaccine if we use the numbers cited above), especially if you take care of yourself. What the vaccine does is it lowers your risk of getting the flu from 2.73% to 1.18%- which to you is only an overall difference in risk of 1.55%. Another way of looking at it is that you need to vaccinate 100 people to prevent ONE set of influenza symptoms [10]. As if that's not impressive enough, Fireman et al estimated that you would need to vaccinate 4,000 people to prevent one death from the flu in their elderly Kaiser Permanente cohort [5]. Again, this is the group of people theorized to need the vaccine the most and whom make up approximately 90% of influenza related deaths. Oh, and let's not forget that this doesn't even take into consideration the numerous confounding variables we talked about in the first half of the article, some of which have been shown to either partially or completely explain any benefit the vaccine has been shown to have. Call me crazy, but that doesn't sound nearly as impressive as the 50-70% effectiveness rates we typically hear about...
Not only is this misleading to the lay person, but researchers seem to be confused by their own statistics. Recall if you will, the authors who said that the 50% reduction in all cause mortality reported with the vaccine was unrealistic because the flu itself only accounts for about 5% of all winter deaths. They are trying to compare apples to oranges- you would have to back track and look at the absolute differences in order to compare these results to all cause mortality. Perhaps if they used the numbers from the Fireman et al study, 1 death prevented for every 4,000 vaccinated, that would make more sense. The kicker is that this wording was only used once toward the end of the article. If you read their abstract or their conclusion, however, even this group reported the vaccine effectiveness rate as 47% [5]!
It is worth noting that numerous studies have demonstrated that the vaccine is least effective in the two groups that are presumed to need it the most- children and the elderly [1,4]. In one trial the researchers estimate that the vaccine effectiveness drops from 75% to 23% between the ages of 65 and 70- a huge and important drop considering that 90% of all influenza deaths are in folks over the age of 65. There were
no randomized clinical control trials that were good enough to meet inclusion criteria for the 2012 meta-analysis that showed TIV (trivalent influenza vaccine) effectiveness in people aged 2-17 or over 65 [4]. Only the live attenuated vaccine (LAIV) was shown to have any effect in the 65 years old plus crowd, but that vaccine is not approved for use in adults over 50 in the United States [4]. Perhaps most importantly for seniors, the influenza vaccine has been shown to do nothing to prevent community acquired pneumonia. Pneumonia is responsible for the vast majority of influenza-related deaths (approximately 34,000 of the 36,000 deaths) each year [9], so this certainly doesn't lend much credibility to those estimates on decreasing all cause mortality.
I think it's safe to say at this point that there are a LOT of things to consider when it comes to this body of research. There are numerous confounding variables that may either partially or completely account for the reported effectiveness of the vaccine, "effectiveness" can mean anything from preventing death to preventing the flu, there is a lack of solid evidence that the vaccine works at all in the populations that supposedly need it the most, and the method of reporting the vaccine effectiveness is misleading at best. There are other issues that simply could not make it into this post such as the ingredients in the vaccine, side effects, politics, marketing, and conflicts of interest with funding sources. But all things said, I can not legally tell you whether or not you should get the flu shot. Whether you choose to get the vaccine or not is your personal choice, and I understand that there are a lot of factors that do, and should, go into this decision. All I can hope to do is educate you so that you can make the best decision for you and your family. I personally think I have a better shot (pun!) at preventing the flu if I keep myself healthy... especially when you take the unimpressive results from the research into consideration.
Even after all that, we still haven't gotten to talk about what actually
causes the flu! I know what you're thinking, the influenza virus causes the flu. But the reality is that it is predominantly your health as host that determines weather or not a virus will successfully make you sick. Surely we have all lived with someone who has gotten sick, yet did not get sick ourselves. There had to have been a reason why you didn't get sick, and it surely wasn't from lack of exposure. Next blog post we're going to talk about what actually
causes the flu, as well as what natural things you can do to prevent it. Spoiler alert-
there is a vitamin out there that kicks flu vaccine butt at preventing the flu!
Stay healthy,
Nikki Cyr
References:
[1] Simonsen, L. Mortality benefits of influenza vaccination in elderly people: an ongoing controversy. Lancet Infectious Dis 2007;7:658-66
[2] Jackson, L. Safety, efficacy and immunogenicity of an inactivated influenza vaccine in healthy adults: a randomized, placebo-controlled trial over two influenza seasons. British Medical Journal Infectious Diseases 2010, 10:71
[3] Lang, P. Effectiveness of influenza vaccine in aging and older adults: comprehensive analysis of the evidence. Clinical Interventions in Aging 2012:7; 55-64
[4] Osterholm, M. Efficacy and effectiveness of influenza vaccines: a systemic review and meta-analysis. Lancet Infectious Disease 2012; 12: 36-44
[5] Fireman, B. Influenza vaccination and mortality: Differentiating vaccine effects from bias. American Journal of Epidemiology 2009;170:650-656
[6] Eurich D. Mortality reduction with influenza vaccine in patients with pneumonia outside "flu" season. American Journal of Respiratory Care Medicine 2008;178:527-533
[7] Hottes, T. Influenza vaccine effectiveness in the elderly based on administrative databases: change in immunization habit as a marker for bias. PLoS ONE 6970: e22618
[8] http://www.cdc.gov/flu/pastseasons/1112season.htm
[9] Jackson. Influenza vaccination and risk of community acquired pneumonia in immunocompetent elderly people: a population-based nested case-control study. 2008;372:398-405
[10] Jefferson, T. Vaccination for preventing influenza in healthy adults. Cochrane database of systemic reviews; 8 article CDOO4879
