G&H How does a network meta-analysis differ from a traditional meta-analysis?
SS A traditional meta-analysis, also known as a pairwise or direct meta-analysis, compares treatments that have been directly compared to each other. However, given the paucity of head-to-head trials in the field of inflammatory bowel disease (IBD), the value of a traditional meta-analysis in informing comparative efficacy of therapies is limited. In this situation, a network meta-analysis may be helpful. A network meta-analysis involves the simultaneous analysis of both direct and indirect evidence to calculate a mixed effect estimate as the weighted average of both types of evidence. Direct evidence comes from randomized, controlled trials (RCTs) directly comparing treatments of interest, in which treatment A is compared to treatment B. Indirect evidence comes from RCTs comparing treatments of interest with a common comparator. For example, if treatments A and B have been compared with a common treatment (C) in 2 different sets of trials (A vs C and B vs C), then the relative effectiveness between A and B can be estimated indirectly via the common comparator C.
G&H Could you expand on the rationale behind performing a network meta-analysis?
SS Over the last several years, multiple pharmacologic options have been approved for the treatment of IBD. However, for regulatory approval, these agents are required to demonstrate superiority over placebo, and the number of head-to-head trials of active interventions is negligible. Patients and providers, however, are more interested in understanding the comparative efficacy of different therapies to inform relative positioning in the treatment paradigm. In this situation, a network meta-analysis can inform stakeholders regarding the comparative efficacy and safety of these interventions while head-to-head trials are awaited. In some occasions, especially when head-to-head trials are small and underpowered, a network meta-analysis can improve the precision of comparison.
G&H What are the most important parts of conducting a network meta-analysis?
SS The most important part is determining whether the question and evidence base lend themselves to a network meta-analysis. In the field of IBD, because head-to-head trials are limited, network meta-analyses rely heavily on indirect comparisons. In these instances, it is very important to ascertain whether the trials are very similar to each other; that is, the trials should be very similar in terms of key factors that determine treatment efficacy, including patients (similar disease characteristics and severity, prior failure of therapies), interventions (standard dose and schedule), cointerventions (which can influence treatment efficacy), and outcome assessment (similar reporting indices and definitions for outcome, assessed in a standard manner). This concept is known as the transitivity or comparability of trials, and is a matter of judgment.
Also important, and a major assumption for network meta-analyses that should be measured statistically, is consistency or homogeneity of effect between direct head-to-head evidence and indirect evidence, based on common comparators.
G&H Are there any guidelines for conducting a network meta-analysis?
SS Similar to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) reporting guidelines for traditional meta-analyses, there is a PRISMA extension statement that applies to network meta-analyses that details what is important to report. The International Society for Pharmacoeconomics and Outcomes Research has also published a conceptual framework for good research practices when performing a network meta-analysis that I find helpful. For consumers of network meta-analyses, there are user guides on how to interpret the results and judge credibility.
G&H How should bias and quality be assessed in a network meta-analysis?
SS Bias within a network meta-analysis can be studied at 2 levels. One is the bias in individual studies. This primarily pertains to the methodologic quality of included RCTs and involves assessment of the approach to the blinding of patients, providers, and outcome assessors; sequence generation; allocation concealment; and other potential biases.
Besides assessing bias within the context of each individual study, a larger assessment of the quality of evidence coming from a network meta-analysis can be studied using different approaches. One is the GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) approach, which can be applied to network meta-analyses to determine confidence in the effect estimates. This involves an overall judgment of the risk of bias across the entire body of evidence, imprecision of estimates derived from a network meta-analysis, indirectness of evidence (which captures whether evidence being generated in the network meta-analysis is being applied to the right patient population), heterogeneity or inconsistency, and publication bias. Additionally, in the context of a network meta-analysis, it is important to judge transitivity or comparability of different trials with regard to patients included in the trials (clinical and disease characteristics), interventions, cointerventions/concomitant therapies, and outcome assessment (similar definitions of treatment success, use of similar disease activity indices, and measurement at comparable time points).
G&H Could you further explain why network meta-analyses are needed specifically in IBD?
SS As previously discussed, the field of IBD is expanding rapidly as the number of treatment options is increasing. The US Food and Drug Administration (FDA) currently requires a new medication to demonstrate superiority over placebo (rather than an established active intervention) for regulatory approval. Infliximab, adalimumab, golimumab (Simponi, Janssen), vedolizumab (Entyvio, Takeda), ustekinumab (Stelara, Janssen), and tofacitinib (Xeljanz, Pfizer) have all been approved by the FDA based on placebo-controlled trials. Although such trials are useful in establishing the efficacy of an intervention, they do not help patients and physicians understand the comparative efficacy of the different interventions that are available. To date, there has been only 1 head-to-head trial comparing different biologic agents in IBD—the VARSITY trial, which compared the use of vedolizumab to adalimumab in patients with moderate to severe ulcerative colitis. Additional head-to-head trials are currently underway and/or being planned. While results of these trials are eagerly being awaited, patients, physicians, and other stakeholders can look to network meta-analyses, which combine direct and indirect evidence, to help inform them of the comparative efficacy and safety of different interventions.
G&H As an example, could you outline the steps that were taken to conduct a recent network meta-analysis in IBD?
SS My colleagues and I recently conducted a network meta-analysis to inform the comparative efficacy of FDA-approved interventions for inducing remission in patients with moderate to severe active ulcerative colitis when used in biologic-naive patients and in patients with prior exposure to tumor necrosis factor–α antagonists. The steps performed for this network meta-analysis were similar to those normally taken for a systematic review. We defined the question in terms of patients, interventions, comparator, and outcome format; conducted a systematic literature search of multiple databases; and identified studies, abstracted the data, and conducted a risk of bias assessment in duplicate at the level of each individual trial. Most importantly, we performed a qualitative assessment of the transitivity or comparability of the trials. Because prior exposure to tumor necrosis factor–α antagonists is one of the strongest and most consistent effect modifiers, we opted to conduct separate network meta-analyses in biologic-naive patients and patients with prior exposure to tumor necrosis factor–α antagonists. Otherwise, all of the trials were reasonably comparable in terms of inclusion criteria and patients’ clinical characteristics, relied only on FDA-approved agents and corresponding doses, and used a very similar outcome definition.
G&H How should the findings of this network meta-analysis be interpreted?
SS Based on a total of 15 randomized trials with 3747 biologic-naive patients, my colleagues and I observed that all agents were more effective than placebo for inducing clinical remission, and, on indirect comparison, infliximab was more effective than adalimumab in inducing clinical remission. Infliximab was ranked highest in efficacy for inducing remission among all agents. Similarly, based on 7 trials with close to 1600 patients with prior exposure to tumor necrosis factor–α antagonists, we observed that both tofacitinib and ustekinumab were more effective than adalimumab and vedolizumab in inducing remission.
G&H What are the limitations of these findings?
SS We relied heavily on indirect treatment comparisons due to the paucity of head-to-head trials in the field. Additionally, while trials of induction therapy were very similar in design, trials of maintenance therapy had
different designs (trials of infliximab and adalimumab were designed as treat-straight-through trials, whereas trials of vedolizumab, golimumab, ustekinumab, and tofacitinib rerandomized responders to induction therapy during the maintenance trial). Otherwise, there were subtle differences in how the trials in our network meta-analysis were conducted. For example, trials of tofacitinib and ustekinumab used blinded center readers for endoscopic assessment, whereas the other trials relied on local readers. There were also subtle differences in the timing of assessment of induction of remission. These slight differences should be acknowledged, and our findings should be interpreted with caution when being applied to clinical practice.
G&H How well-accepted are network meta-analyses by practicing gastroenterologists?
SS Network meta-analyses have been published for 20 years, but over the last 5 to 7 years, their frequency has risen across all fields, including gastroenterology, with the improvement of statistical methodology and the increase in acceptability of findings. In a simulation study in the field of glaucoma published in the Annals of Internal Medicine, investigators observed that if guidelines had used network meta-analyses to inform comparative efficacy and treatment recommendations, they would have reached the same conclusions approximately 10 years sooner than they eventually did with reliance on only traditional meta-analyses. Because of the paucity of head-to-head trials within the field of IBD, network meta-analyses are gradually gaining more attention and traction in informing the comparative efficacy of different interventions as more treatment options are becoming available. The aforementioned ulcerative colitis network meta-analysis was used to inform the recent American Gastroenterological Association’s guidelines on management of moderate to severe ulcerative colitis in terms of the positioning of different biologic agents and small molecules in this patient population.
However, as previously mentioned, it is important for physicians to recognize the basic assumptions of a network meta-analysis and judge whether they believe that the included trials are sufficiently comparable to allow indirect comparison and whether they can believe the results from the network meta-analysis.
G&H What are some of the other network meta-analyses that have been recently conducted in IBD?
SS In 2015, a network meta-analysis that was conducted by Dr Gilaad Kaplan’s group found that adalimumab monotherapy and the combination of infliximab and azathioprine were likely the best treatment options for induction of clinical remission in patients with Crohn’s disease. My colleagues and I conducted a network meta-analysis looking at the comparative efficacy of different interventions for the management of Crohn’s disease after surgical resection to decrease the risk of clinical and endoscopic recurrence. Other groups have compared different endoscopic techniques for dysplasia detection in patients with longstanding chronic colitis. These are just a few examples of recent network meta-analyses that have been conducted in IBD.
G&H What issues would you like to see being examined in IBD network meta-analyses in the future?
SS I believe that the best evidence comes from a connected network that includes multiple head-to-head trials; hence, as more head-to-head trials are conducted, updated network meta-analyses will be more reliable and informative. A network meta-analysis may also be useful to study comparative efficacy of various interventions in different phenotypes, such as patients with only moderate ulcerative colitis or patients with small bowel vs colon-dominant Crohn’s disease, if relevant patient-level data accounting for key effect modifiers are available.
Dr Singh receives research support from AbbVie and Janssen and personal fees from Pfizer for ad-hoc grant review.
Suggested Reading
Bessissow T, Dulai PS, Restellini S, et al. Comparison of endoscopic dysplasia detection techniques in patients with ulcerative colitis: a systematic review and network meta-analysis. Inflamm Bowel Dis. 2018;24(12):2518-2526.
Feuerstein JD, Isaacs KL, Schneider Y, Siddique SM, Falck-Ytter Y, Singh S; AGA Institute Clinical Guidelines Committee. AGA clinical practice guidelines on the management of moderate to severe ulcerative colitis. Gastroenterology. 2020;158(5):1450-1461.
Hazlewood GS, Rezaie A, Borman M, et al. Comparative effectiveness of immunosuppressants and biologics for inducing and maintaining remission in Crohn’s disease: a network meta-analysis. Gastroenterology. 2015;148(2):344-354.e5; quiz e14-e15.
Hoaglin DC, Hawkins N, Jansen JP, et al. Conducting indirect-treatment-
comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. Value Health. 2011;14(4):429-437.
Hutton B, Salanti G, Caldwell DM, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162(11):777-784.
Jansen JP, Fleurence R, Devine B, et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value Health. 2011;14(4):417-428.
Li T, Puhan MA, Vedula SS, Singh S, Dickersin K; Ad Hoc Network Meta-
Analysis Methods Meeting Working Group. Network meta-analysis—highly attractive but more methodological research is needed. BMC Med. 2011;9:79.
Mills EJ, Ioannidis JP, Thorlund K, Schünemann HJ, Puhan MA, Guyatt GH. How to use an article reporting a multiple treatment comparison meta-analysis. JAMA. 2012;308(12):1246-1253.
Nguyen NH, Fumery M, Dulai PS, et al. Comparative efficacy and tolerability of pharmacological agents for management of mild to moderate ulcerative colitis: a systematic review and network meta-analyses. Lancet Gastroenterol Hepatol. 2018;3(11):742-753.
Singh S. Network meta-analysis to inform positioning of biologics in patients with Crohn’s disease: promise and perils. Best Pract Res Clin Gastroenterol. 2019;38-39:101614.
Singh S, Allegretti JR, Siddique SM, Terdiman JP. AGA technical review on the management of moderate to severe ulcerative colitis. Gastroenterology. 2020;158(5):1465-1496.e17.
Singh S, Fumery M, Sandborn WJ, Murad MH. Systematic review and network meta-analysis: first- and second-line biologic therapies for moderate-severe Crohn’s disease. Aliment Pharmacol Ther. 2018;48(4):394-409.
Singh S, Garg SK, Pardi DS, Wang Z, Murad MH, Loftus EV Jr. Comparative efficacy of pharmacologic interventions in preventing relapse of Crohn’s disease after surgery: a systematic review and network meta-analysis. Gastroenterology. 2015;148(1):64-76.e2; quiz e14.
Singh S, Murad MH, Fumery M, Dulai PS, Sandborn WJ. First- and second-line pharmacotherapies for patients with moderate to severely active ulcerative colitis: an updated network meta-analysis [published online January 13, 2020]. Clin Gastroenterol Hepatol. doi:10.1016/j.cgh.2020.01.008.