Comprehensive Meta-Analysis of COVID-19 Vaccine Effectiveness--Raises Lots of Questions
How effective are the COVID-19 vaccines at preventing serious illness and death? This data is more uncertain than mainstream media may let on based on an actual death of clinical trials data. In reality, the ability to predict the efficacy of the COVID-19 vaccines via the use of antibody concentrations is perplex. Thus, a group of prominent Chinese epidemiologists, health data analysts, and physician-scientists affiliated with academic medical centers in Shenzhen China conducted a systematic review and meta-analysis of randomized controlled trials probing for the efficacy of SARS-CoV-2 vaccines. Employing use of the Cochrane tool for bias assessment, the study team used a frequentist, random-effects model to combine efficacy for common outcomes (symptomatic and asymptomatic infections) and Bayesian-effects model for rare outcomes (e.g., hospital admission, severe infection, and death). The investigators also used meta-regression to study source of heterogeneity as well as dose-response relationships of neutralizing, spike-specific IgG, and receptor binding domain-specific IgG antibody titers with efficacy in preventing symptomatic and severe COVID-19. These findings reinforce what TrialSite has reported for over two years: the COVID-19 vaccines were never able to control the pandemic because the efficacy of vaccination in preventing infections wasn’t great—under 50% according to these findings. Thus, the authors of this inquiry point out, “That vaccines alone are unlikely to be able to stop the pandemic.” Also, this systematic review and meta-analysis revealed a lower vaccine efficacy against symptomatic infection than comparable studies.
Recently published in The Lancet, Microbe, the study funded by the Shenzhen Science and Technology Programs raises interesting questions that need to be taken seriously in the West. For example, the group of epidemiologists, health data quants, and physician-scientists write:
“Our findings suggested that SARS-CoV-2 vaccines are insufficiently efficacious in preventing infections, and therefore cannot stop the pandemic alone. However, all the vaccines can be used to effectively prevent severe infection and death. Waning of vaccine efficacy makes it important to time vaccination in relation to an anticipated outbreak. Moreover, the antibody–efficacy relationship is complex and should be used with caution to predict efficacy in uncertain situations. Since the vaccines can no longer be assessed in previous predominant variants that are now extinct, this systematic review provides an irreplaceable reference for comparisons with and interpretation of future studies assessing vaccine efficacy in new scenarios, such as booster vaccine regimens, new SARS-CoV-2 vaccines, and future SARS-CoV-2 variants.”
Findings
The study team’s search ultimately led to 28 randomized controlled trials involving 286,915 vaccination groups and 233,236 placebo groups with a median follow up of 1-6 months post vaccine administration across 32 publications. What follows are key findings:
Combined Efficacy
Findings
Full vaccination—preventing infections
44.5% (95% CI 27·8–57·4)
Full vaccination—preventing asymptomatic infections
76.5% (69·8–81·7)
Preventing hospitalization
95·4% (95% credible interval 88·0–98·7)
Preventing severe infection
90·8% (85·5–95·1)
Preventing Death
85.8% (68·7–94·6 )
The study authors reported, “Insufficient evidence to suggest whether the efficacy could differ according to the type of vaccine, age of the vaccinated individual, and between-dose interval (p>0·05 for all).”
Not surprisingly, durability challenges persist in this systematic review: effectiveness wanes post full vaccination. On average, the decrease equals 13.6% ( 95% CI 5·5–22·3; p=0·0007) per month, however, the booster can improve this performance. While the study team reports a “significant non-linear relationship between each type of antibody and efficacy against symptomatic and severe infections (p<0·0001 for all),” considerable variability in efficacy cannot be explained by antibody concentrations. The authors report that the risk of bias across most studies was low.
What follows is a TrialSite breakdown of the systematic review and meta-analysis:
What’s the overall quality of evidence that COVID-19 vaccines can reduce risk of SARS-CoV-2 infections and effectively prevent most severe infections and deaths?
Moderate quality
Can the magnitude of vaccine efficacy vary across populations?
Yes. The authors base this assessment on the moderate-to-high heterogeneity across randomized controlled trials.
Does efficacy wane over time post full series vaccine administration?
Yes. But this can be enhanced via booster.
What’s the main source of SARS-CoV-2 transmission?
Mild infection
Is the evolution of SARS-CoV-2 the likely cause for compromised efficacy?
Yes
Is confirmation of these findings based on indirect comparisons from different randomized controlled trials challenging?
Yes. This is due to what the authors write is “the large unexplained heterogeneity and incomplete coverage of people who are infected for whole-genome sequencing in randomized controlled trials.”
So, are the COVID-19 vaccines used to eradicate SARS-CoV-2?
Absolutely not. This first wave of infections can only help reduce severe infection and death.
Do the authors still consider vaccination against COVID-19 as “crucial”?
Yes. Because based on this meta-analysis these products prevented about 90% of severe COVID-19 illness and deaths.
But what about the durability challenge, how problematic is this?
Considerably problematic. Vaccine efficacy based on the findings lowers in the first six months post full vaccination was completed in the randomized controlled trials. But the authors note that the time the vaccine impacts peak, and wane tend to vary by vaccine, and that booster doses “might compensate for the waning of efficacy.”
Did this meta-analysis find sufficient data to establish changes in efficacy over time to anticipate severe infection performance?
No. The authors report insufficient data from the randomized controlled trials... However, the authors point to observational data pointing to robust protection for six months.
Is the timing of the vaccination and booster key?
Yes. This is due to waning effectiveness. Time the vaccination in relation to an anticipated outbreak.
What about SARS-CoV-2 antibodies elicited by vaccines—is this a sound way to predict or compare the impact of vaccine efficacy?
No. the study team writes that the “validity of such predictions” hasn’t been established sufficiently yet. And, while the study team did find generally that the higher the concentration of neutralizing, anti-spike, or anti-RBC antibody titers, the higher the efficacy, especially against more severe infection. But this efficacy isn’t proportional to concentrations of antibodies.
Are efforts to use antibody concentration to predict vaccine efficacy full proof?
Absolutely not. The study team suggests proceeding with caution, especially when “antibodies and efficacy are measured from different populations.” T-cell immunity could help augment in a more comprehensive efficacy model along with other factors that need consideration for use of antibody concentrations alone as a predictor for COVID-19 vaccine efficacy.
What are some challenges with T-cell concentrations as a predictor?
The dose relationship involving T-cell concentration and efficacy hasn’t been established. Plus, the relationship represents a quandary based on current available clinical trial data (different measures of T -cell immunity, etc.).
What are some study limitations?
The authors acknowledged severe limitations including:
Notable heterogeneity in the efficacy of vaccination for preventing symptomatic infections; moderate heterogeneity for preventing severe infection—this is likely the result of trial design factors and makes comparing the different types of vaccines under evaluation in separate rerandomized controlled trials difficult.
Limitations in this meta-analysis based on the limitations in underlying randomized controlled trials.
“Ecological bias” could likely exist in the meta-regression using aggregated study-level data—other factors may influence the antibody concentration and efficacy (types of vaccines, variants of SARS-CoV-2, etc.).
What are key implications for this research?
The current COVID-19 vaccines are not well suited to stop or prevent SARS-CoV-2 transmission, thus they are not meant to control the pandemic. However, they do help prevent more serious morbidity and mortality. A mutating RNA virus assures waning effectiveness of vaccines. Individual immunogenicity with booster likely cannot consistently predict vaccine efficacy.
With the omicron variant vaccine efficacy wants, past 6 months actually effectiveness isn’t certain and other attributes of vaccination, such as age, between-dose interval and variant lead to unsatisfactory efficacy predictions—there just isn’t enough clinical trials data.
Want to enable valid comparisons of study results from different studies? Standardize the relationships of vaccine efficacy with humoral and cellular immune responses.
Surveillance testing for SARS-Cov-2 covering an entire population can reduce detection bas
Seek better comparative efficacy between vaccine regimens or between variant participant subgroups? Increase the power of new studies.
Real-world studies are important alternative to randomized controlled trials (on large scale) to address important decision-making questions.
