Intended for healthcare professionals

CCBYNC Open access
Research

Comparative safety of immune checkpoint inhibitors in cancer: systematic review and network meta-analysis

BMJ 2018; 363 doi: https://doi.org/10.1136/bmj.k4226 (Published 08 November 2018) Cite this as: BMJ 2018;363:k4226
  1. Cheng Xu, resident physician1,
  2. Yu-Pei Chen, resident physician1,
  3. Xiao-Jing Du, associate professor1,
  4. Jin-Qi Liu, postgraduate1,
  5. Cheng-Long Huang, postgraduate1,
  6. Lei Chen, associate professor1,
  7. Guan-Qun Zhou, fellow1,
  8. Wen-Fei Li, associate professor1,
  9. Yan-Ping Mao, associate professor1,
  10. Chiun Hsu, professor2,
  11. Qing Liu, statistician3,
  12. Ai-Hua Lin, statistician3,
  13. Ling-Long Tang, associate professor1,
  14. Ying Sun, professor1,
  15. Jun Mahttp://orcid.org/1
  1. 1Department of Radiation Oncology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, 651 Dongfeng Road East, Guangzhou 510060, China
  2. 2Department of Oncology, National Taiwan University Hospital, Taipei, Taiwan
  3. 3Department of Medical Statistics and Epidemiology, School of Public Health, Sun Yat-sen University, Guangzhou, China
  1. Correspondence to: J Ma majun2{at}mail.sysu.edu.cn
  • Accepted 4 October 2018

Abstract

Objective To provide a complete toxicity profile, toxicity spectrum, and a safety ranking of immune checkpoint inhibitor (ICI) drugs for treatment of cancer.

Design Systematic review and network meta-analysis.

Data sources Electronic databases (PubMed, Embase, Cochrane Library, and Web of Science) were systematically searched to include relevant studies published in English between January 2007 and February 2018.

Review methods Only head-to-head phase II and III randomised controlled trials comparing any two or three of the following treatments or different doses of the same ICI drug were included: nivolumab, pembrolizumab, ipilimumab, tremelimumab, atezolizumab, conventional therapy (chemotherapy, targeted therapy, and their combinations), two ICI drugs, or one ICI drug with conventional therapy. Eligible studies must have reported site, organ, or system level data on treatment related adverse events. High quality, single arm trials and placebo controlled trials on ICI drugs were selected to establish a validation group.

Results 36 head-to-head phase II and III randomised trials (n=15 370) were included. The general safety of ICI drugs ranked from high to low for all adverse events was as follows: atezolizumab (probability 76%, pooled incidence 66.4%), nivolumab (56%, 71.8%), pembrolizumab (55%, 75.1%), ipilimumab (55%, 86.8%), and tremelimumab (54%, not applicable). The general safety of ICI drugs ranked from high to low for severe or life threatening adverse events was as follows: atezolizumab (49%, 15.1%), nivolumab (46%, 14.1%), pembrolizumab (72%, 19.8%), ipilimumab (51%, 28.6%), and tremelimumab (28%, not applicable). Compared with conventional therapy, treatment-related adverse events for ICI drugs occurred mainly in the skin, endocrine, hepatic, and pulmonary systems. Taking one ICI drug was generally safer than taking two ICI drugs or one ICI drug with conventional therapy. Among the five ICI drugs, atezolizumab had the highest risk of hypothyroidism, nausea, and vomiting. The predominant treatment-related adverse events for pembrolizumab were arthralgia, pneumonitis, and hepatic toxicities. The main treatment-related adverse events for ipilimumab were skin, gastrointestinal, and renal toxicities. Nivolumab had a narrow and mild toxicity spectrum, mainly causing endocrine toxicities. Integrated evidence from the pooled incidences, subgroup, and sensitivity analyses implied that nivolumab is the best option in terms of safety, especially for the treatment of lung cancer.

Conclusions Compared with other ICI drugs used to treat cancer, atezolizumab had the best safety profile in general, and nivolumab had the best safety profile in lung cancer when taking an integrated approach. The safety ranking of treatments based on ICI drugs is modulated by specific treatment-related adverse events.

Systematic review registration PROSPERO CRD42017082553.

Introduction

Over recent decades, immunotherapy has become one of the most important breakthroughs in cancer treatment, especially, but not only, for patients with advanced, recurrent, and metastatic malignancy.1 Immune checkpoint inhibitor (ICI) drugs enhance self immune functions against cancer cells through a unique mechanism that blocks negative regulators expressed on immune or tumour cells. These regulators include cytotoxic T lymphocyte associated antigen 4, programmed cell death 1, and programmed cell death ligand 1.2 Although tumour cells invalidate immune surveillance and develop malignancies by taking advantage of programmed cell death 1, programmed cell death ligand 1, or cytotoxic T lymphocyte associated antigen 4, these immune checkpoints are normally essential in preventing over activation of the immune system and maintaining immunologic homoeostasis.13 Therefore, although monoclonal antibodies against programmed cell death 1 (nivolumab and pembrolizumab), programmed cell death ligand 1 (atezolizumab), and cytotoxic T lymphocyte associated antigen 4 (ipilimumab and tremelimumab) provide impressive anti-tumour activity in many solid tumours, they also have associated adverse events, known as immune-related adverse events.4

Chemotherapy has long been the first line treatment for advanced cancers. Doctors often use multiple chemotherapy drugs and different combinations (that is, investigator’s choice chemotherapy) for patients as their condition changes. Chemotherapy can have serious adverse events that lead to low compliance, dose reductions, treatment delays, and treatment refusals. The toxicities of many chemotherapy drugs have been clearly shown, owing to long term clinical use. The National Comprehensive Cancer Network has published several guidelines addressing adverse events associated with chemotherapy.5 Compared with the toxicities caused by conventional therapy (chemotherapy, targeted therapy, and their combinations), immune-related adverse events from ICI drugs are unique in terms of the organs involved, onset patterns, and severity.6 Numerous randomised controlled trials have outlined a crude profile of immune-related adverse events, including skin, gastrointestinal, pulmonary, hepatic, and endocrine toxicities.789101112131415161718192021222324252627282930313233343536373839404142 Although most immune-related adverse events can be well controlled by supportive treatment and glucocorticoids, fatal immune-related adverse events are an increasing concern regarding the safety of ICI drugs and patients’ tolerability.4344 The mechanism of immune-related adverse events remains unclear. Different ICI drugs and different doses of the same ICI drug may result in different adverse events. The United States Food and Drug Administration has approved several ICI drug products for the treatment of cancers with comparatively poor prognosis (eg, melanoma, lung cancer, urothelial cancer, renal cell carcinoma, and gastric cancer), and there has been an increase in combined treatment (that is, use of two ICI drugs, or use of one ICI drug with conventional therapy).8915161826272829383940 Robust evidence indicates that ICI drugs are superior to traditional standard chemotherapy in first line treatment for some types of cancer (eg, lung cancer and melanoma) and in second line and subsequent anti-cancer treatment for others, and confer significantly higher survival benefits.171922253032333435 In particular, owing to its noticeably improved efficacy in lung cancer, pembrolizumab was promoted to first line therapy in October 2016 and now has expanded indications.45 However, it is difficult to acquire a complete picture of immune-related adverse events from randomised controlled trials due to limitations in the study design and realistic practicalities. For example, nivolumab and pembrolizumab have never been head-to-head compared, because clinical trials about them are initiated or sponsored by different and competitive pharmaceutical companies. Past meta-analyses on this issue focused on a specific immune-related adverse event or an ICI drug (or both), failing to cover all possible comparisons.4647 In addition, these meta-analyses included phase I, single arm, or placebo control studies, which also diminished the reliability and validity of their results.

High quality studies focusing on adverse events are required to aid clinicians in improving early management and recognition of immune-related adverse events. Researchers also need to be able to optimise the design of future trials with enhanced anti-tumour efficacy and safety as two major criteria of success. Here, we report a comprehensive dose and drug based network meta-analysis of the comparative safety of ICI drugs, with the aim of providing a complete toxicity profile, toxcicity spectrum, and a safety ranking of ICI drugs alone or in combination with conventional therapy.

Methods

This network meta-analysis was reported in line with the preferred reporting items for systematic reviews and meta-analysis (PRISMA) and the PRISMA extension statement for network meta-analysis (supplementary materials, table S1).4849 A prospective protocol was created in advance and uploaded to the PROSPERO online platform, with the registration number CRD42017082553.

Data sources and searches

We searched the PubMed, Embase, Cochrane Library, and Web of Science databases to include relevant studies published in English between January 2007 and February 2018. Search terms and their combinations used in the search strategy included cancer, carcinoma, neoplasm, immune checkpoint inhibitors (anti-programmed cell death 1, anti-programmed cell death ligand 1, and anti-cytotoxic T lymphocyte associated antigen 4), and specific ICI drug names (nivolumab, pembrolizumab, ipilimumab, tremelimumab, and atezolizumab). The search strategy is detailed in the supplementary materials. After the preliminary screening of titles or abstracts, two independent reviewers (CX and X-JD) assessed the full text and reference lists of relevant publications for final inclusion; articles cited as references that were considered to be potentially relevant were also retrieved and assessed in full.

Study selection

Only head-to-head phase II and III randomised controlled trials comparing two or more treatments including an ICI drug (that is, one ICI drug, two ICI drugs, and one ICI drug with conventional therapy), or comparing an ICI drug with conventional therapy in patients with cancer were eligible. Studies that compared different doses or usages (such as pharmaceutical forms and administration sites) of one ICI drug were also included. We excluded conference abstracts, posters, and presentations of ongoing randomised controlled trials because these brief reports did not contain detailed safety data. Studies providing a summary but no site, organ, or system level toxicity data were also excluded. In addition, high quality, single arm trials and placebo controlled trials meeting certain criteria were selected to establish a validation group (supplementary materials and tables S2 and S3).

Data extraction

Although immune-related adverse events were thought to be the outcome of interest, this term was inappropriate to describe the toxicity from conventional therapy. We selected treatment-related adverse events as the primary outcome, because these were compatible with most studies and deemed to be a suitable alternative to immune-related adverse events. If treatment-related adverse events were not available in a study, all adverse events were used. We evaluated the main text, supplementary materials, and all possible information available at ClinicalTrials.gov to implement an extensive and detailed data extraction. Although both original and updated studies derived from one trial were included in this network meta-analysis, we extracted treatment-related adverse events data from the study with the most detailed report; the other data were used to supplement basic information. Two reviewers (CX and X-JD) used a standardised form independently to extract and summarise the following data: first author, year of publication, study ID, region, cancer type, study design, total number of patients, number of patients in safety analysis, arms and treatment regimens, version of Common Terminology Criteria for Adverse Events, follow-up time, treatment-related adverse event reporting rate, and the frequency of each specific grade 1-5 and grade 3 or 4 adverse event. The Common Terminology Criteria for Adverse Events is the most commonly used tool for evaluating adverse event type and severity in clinical practice with a grading scale and clear definitions. All grades, grade 3, and grade 4 adverse events indicate complete, severe, and life threatening toxicity, respectively.50

Quality assessment

Two reviewers (CX and X-JD) assessed the risk of bias based on the original study, possible updated study, and supplementary materials by using the tool recommended by the Cochrane Collaboration Handbook. The adequacy of five aspects was assessed: random sequence generation, allocation concealment, blinding method, assessment of outcomes, and reporting of results.51 Each aspect was assigned an assessment index associated with the risk of bias classified as yes, no, or unclear. We scored the method quality of studies quantitatively by using the modified Jadad scale (high quality, four points and over; or low quality, three points and under).52 All disagreements in the study selection, data extraction, and quality assessment were resolved by discussion to achieve a consensus.

Data synthesis and statistical analysis

Treatment-related adverse events were assessed from two different perspectives: overview and detail, based on the total number of all treatment-related adverse events and the number of each specific treatment-related adverse event, respectively. Regardless of the adverse event grading, general safety was used to indicate the overview of treatment-related adverse events without distinguishing between their specific classifications. We used odds ratios and 95% credibility intervals as summary statistics to quantify the effect of dose (of ICI drug) or drug on the risk of grade 1-5 and grade 3 or 4 adverse events in the network meta-analysis. Odds ratios greater than one represented a safety benefit favouring the control group. Two sided P<0.05 was considered significant. If a study reported zero adverse events in any arm, the classic half integer continuity correction (adding a 0.5 to each cell) was applied for data preparation.

We generated plots depicting the network geometry using Stata version 12.0 (StataCorp).53 Both dose and drug based network meta-analysis were performed in a Bayesian framework using Markov chain Monte Carlo methods in WinBUGS version 1.4.3 (MRC Biostatistics Unit, Cambridge, UK).54 After completion of the dose based network meta-analysis, we re-prepared the data for drug based network meta-analysis for each study by combining multiple treatment arms that represent different doses of an ICI drug into one arm that represents one ICI drug; we excluded studies with only one treatment arm after the combination. We used random effects and consistency models to calculate odds ratios and 95% credibility intervals; these models are thought to be the most conservative approach to dealing with between-study heterogeneity.55 We used non-informative prior distributions and over dispersed initial values (with a scaling of 5) in four chains to fit the model,54 yielding 100 000 iterations (including 20 000 tuning iterations) and a thinning interval of 10 for each chain; this method was used to generate posterior distributions of model parameters. Convergence of iterations was assessed with the Gelman-Rubin-Brooks statistic.56 According to odds ratios and the posterior probabilities, we ranked the probability of each treatment being the safest, followed by the second, and the third, and so on.

Outstanding consistency, represented by the agreement between the direct and indirect results, is the key to robust results. Appraisal of inconsistency was achieved by using a three step method. Firstly, we made a rough comparison of the fit of the consistency model with that of the inconsistency model.57 Secondly, we tested inconsistency in the entire network on particular comparisons (nodes) by node splitting analysis; P<0.05 indicated significant inconsistency.5758 Thirdly, pairwise meta-analysis was conducted in a Frequentist framework by use of Review Manager version 5.3 (Cochrane Collaboration) to provide direct estimates (that is, odds ratios and 95% confidence intervals). Between-study heterogeneity was assessed using the χ2 and Ι2 tests, with significance set at P<0.10; Ι2 more than 25%, 50%, or 75% indicated low, moderate, or high heterogeneity, respectively.59 We used the random effects model if significant heterogeneity existed; otherwise, we used the fixed effects model. Network meta-analysis results (indirect results) were compared with pairwise meta-analysis results (direct results) to explore the source of inconsistency.

We generated the pooled incidence of grade 1-5 and grade 3 or 4 adverse events for treatments by using Meta-Analyst.60 Subgroup analyses were performed based on each specific treatment-related adverse event and cancer type. We conducted sensitivity analyses to assess the stability of results by restricting the original network meta-analysis to phase III randomised controlled trials, high quality studies, studies explicitly reporting treatment-related adverse events, studies on programmed cell death 1 and programmed cell death ligand 1 ICI drugs, studies using the current recommended dose of ICI drugs, and studies that included patients who were previously treated or untreated.

Patient and public involvement

No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for design or implementation of the study. No patients were asked to advise on interpretation or writing up of results. There are no plans to disseminate the results of the research to study participants or the relevant patient community. It was not evaluated whether the studies included in the review had any patient involvement.

Results

Systematic review and characteristics

Figure 1 shows a flowchart of the study selection procedure. The electronic searches yielded 1794 potentially relevant studies, of which 92 potentially eligible articles were assessed. Figure 2A shows that 36 randomised controlled trials (n=15 370) evaluating 15 treatments with different doses (nivolumab 2-3 mg/kg every two weeks, 0.3 mg/kg every three weeks, and 10 mg/kg every three weeks; ipilimumab 3 mg/kg or 10 mg/kg every three weeks and 3 mg/kg every four weeks; tremelimumab 10 mg/kg every 90 days; pembrolizumab 10 mg/kg every two weeks, 2 mg/kg or 10 mg/kg every three weeks, and 200 mg every three weeks; atezolizumab 1200 mg every three weeks; two ICI drugs; one ICI drug with conventional therapy; and conventional therapy) were included in the dose based network meta-analysis.789101112131415161718192021222324252627282930313233343536373839404142Figure 2B shows that 32 randomised controlled trials (n=14 252) evaluating eight treatments with different drugs (nivolumab, ipilimumab, tremelimumab, pembrolizumab, atezolizumab, two ICI drugs, one ICI drug with conventional therapy, and conventional therapy) were included in the drug based network meta-analysis.7891011121314151617181921222325262728293031323334353637383940 Both network meta-analyses included five updated randomised controlled trials without duplicate counting of the sample sizes.911141629 Ipilimumab 0.3 mg/kg every three weeks was reported to cause disease stabilisation only, which is obviously different from the enhanced effect of the objective response at the dose of 3 mg/kg and 10 mg/kg every three weeks; therefore, we discarded the group receiving ipilimumab 0.3 mg/kg every three weeks in the study by Wolchok and colleagues.42Figure 2A shows that among the 15 370 patients included in the dose based network meta-analysis, 2733 received nivolumab, 2113 received ipilimumab, 325 received tremelimumab, 2014 received pembrolizumab, 1210 received atezolizumab, 407 received two ICI drugs, 1429 received one ICI drug with conventional therapy, and 5139 received conventional therapy.

Fig 1
Fig 1

Flowchart of study selection and design

Fig 2
Fig 2

Network plots of comparisons for dose (A) and drug (B) based network meta-analyses. Each circular node represents a type of treatment. The circle size is proportional to the total number of patients (in parentheses). The width of lines is proportional to the number of studies performing head-to-head comparisons in the same study. ICI=immune checkpoint inhibitor

Table 1 shows the baseline characteristics of the 31 studies (36 randomised controlled trials). Nineteen studies (61.3%) were phase III trials. Twenty eight (90.3%) were multinational trials. Cancer types tested in these studies included lung cancer (n=12), melanoma (13), urinary system cancer (4), head and neck cancer (1), and digestive system cancer (1). Two arm studies predominated (n=23, 74.2%) with only 25.8% (n=8) having three arms; the most common treatment arm was conventional therapy (n=22, 71.0%). Modified Jadad scores indicated that 25 studies were high quality and six studies were relatively low quality (supplementary materials, table S4).123031343536

Table 1

Baseline characteristics of 31 studies (36 randomised controlled trials) for Bayesian network meta-analysis by cancer type

View this table:

Dose based network meta-analysis

The upfront dose based network meta-analysis was conducted to investigate the comparability of doses of the same drug. In the consistency model, we found non-significant differences in the risk of grade 1-5 or grade 3 or 4 adverse events between any two different doses of the same ICI drug, although ipilimumab 10 mg/kg every three weeks had a higher risk of grade 3 or 4 adverse events than 3 mg/kg every three weeks (odds ratio 3.08, 95% credibility interval 1.52 to 6.32). The results obtained using the consistency model fit well with the results using the inconsistency model; node splitting analyses showed no significant inconsistency (all P>0.05; supplementary materials, figs S1 and S2 and table S5). These results indicated that the safety profile for each ICI drug, except for ipilimumab, was the same for each dose; this is a prerequisite for the drug based network meta-analysis.

Drug based network meta-analysis and inconsistency

The safety profile in the consistency model (fig 3A) showed a noticeably lower risk of grade 1-5 adverse events favouring atezolizumab or nivolumab over ipilimumab, tremelimumab, two ICI drugs, one ICI drug with conventional therapy, and conventional therapy; ipilimumab or conventional therapy over one ICI drug with conventional therapy; and pembrolizumab over tremelimumab, one ICI drug with conventional therapy, and conventional therapy. Figure 3A shows that for grade 3 or 4 adverse events, atezolizumab or nivolumab was significantly safer than ipilimumab, tremelimumab, two ICI drugs, one ICI drug with conventional therapy, and conventional therapy; ipilimumab was safer than two ICI drugs and one ICI drug with conventional therapy; pembrolizumab was safer than two ICI drugs, one ICI drug with conventional therapy, and conventional therapy; conventional therapy was safer than one ICI drug with conventional therapy. Figure 3A shows that after combining the validation group (two high quality placebo controlled trials and six high quality single arm trials6162636465666768; supplementary materials, table S2 and S3), the pooled incidence of grade 1-5 adverse events for atezolizumab, nivolumab, pembrolizumab, and ipilimumab was 66.4%, 71.8%, 75.1%, and 86.8%, respectively. The pooled incidence of grade 3 or 4 adverse events for atezolizumab, nivolumab, pembrolizumab, and ipilimumab was 15.1%, 14.1%, 19.8%, and 28.6%, respectively. It was not applicable to pool the incidence of treatment-related adverse events for tremelimumab, as only one study was available.37Figure 3B shows that atezolizumab was associated with the best safety ranking for grade 1-5 adverse events (probability=76%), followed by nivolumab (56%), pembrolizumab (55%), ipilimumab (55%), conventional therapy (51%), two ICI drugs (30%), one ICI drug with conventional therapy (48%), and finally tremelimumab (54%). The safety ranking for grade 3 or 4 adverse events from high to low was: atezolizumab (49%), nivolumab (46%), pembrolizumab (72%), ipilimumab (51%), conventional therapy (54%), tremelimumab (28%), one ICI drug with conventional therapy (45%), and two ICI drugs (47%). The probabilities are detailed in the supplementary materials, table S6.

Fig 3
Fig 3

Safety profile (A) and ranking curves (B) according to the drug based network meta-analysis (NMA) in the consistency model. Each cell of the safety profile contains the pooled odds ratios and 95% credibility intervals for grade 1-5 adverse events and grade 3 or 4 adverse events; significant results are in bold. The pooled odds ratios and 95% credibility intervals indicate the result of the top treatment compared with the bottom treatment. Ranking curves indicate the probability of the highest risk of grade 1-5 adverse events and grade 3 or 4 adverse events, the second highest, the third highest, and so on. ICI=immune checkpoint inhibitor

The results in the inconsistency model had a generally satisfactory fit compared with that of the results in the consistency model, except for some comparisons based on ipilimumab; node splitting analysis indicated no significant inconsistency (supplementary materials, table S7 and fig S3). Figure 4 shows the direct and indirect results for 11 head-to-head comparisons. Pairwise meta-analysis was feasible for six comparisons. Four comparisons (nivolumab, pembrolizumab, atezolizumab, and one ICI drug with conventional therapy) were versus conventional therapy. Two comparisons (nivolumab and two ICI drugs) were versus ipilimumab. The pairwise meta-analysis results showed outstanding consistency in both significance and tendency with respect to the corresponding network meta-analysis results. Significant between-study heterogeneity was found in almost all pairwise meta-analysis comparisons (Ι2=73% to 96%; all P<0.05), but not atezolizumab versus conventional therapy (supplementary materials, fig S4 and S5). Among the remaining five comparisons based on one study, only ipilimumab versus conventional therapy showed obvious inconsistency between the direct (safety favoured conventional therapy) and indirect (safety favoured ipilimumab) results, representing the major source of inconsistency.

Fig 4
Fig 4

Forest plots depicting the direct and indirect results of head-to-head comparisons. ICI=immune checkpoint inhibitor; NA=not applicable. *Values in brackets are 95% CrIs.

Subgroup analysis based on specific treatment related adverse events

Ten to 27 studies (4585 to 14 139 patients) were included in the subgroup analysis based on each specific grade 1-5 adverse event. Table 2 shows that nivolumab was safer than tremelimumab for diarrhoea, and safer than ipilimumab for diarrhoea and colitis. Ipilimumab was safer than nivolumab for hyperthyroidism, and safer than pembrolizumab for arthralgia. Pembrolizumab and atezolizumab were safer than ipilimumab and tremelimumab for diarrhoea. All ICI drugs except for tremelimumab had a wider safety profile than two ICI drugs or one ICI drug with conventional therapy. Compared with conventional therapy, ICI drugs were associated predominantly with pruritus, rash, diarrhoea, colitis, hypothyroidism, hyperthyroidism, and pneumonitis; two ICI drugs or one ICI drug with conventional therapy was associated specifically with hepatic toxicities. Figure 5 shows that the ranking profile was generally consistent with the original network meta-analysis ranking with some new findings. Among the five ICI drugs, atezolizumab had the highest risk for causing hypothyroidism, nausea, and vomiting. Tremelimumab had the highest risk for causing rash, diarrhoea, and fatigue. The main adverse events caused by pembrolizumab were arthralgia, pneumonitis, and hepatic toxicities. Nivolumab and ipilimumab had non-overlapping toxicity results. Nivolumab mainly caused endocrine toxicities. Ipilimumab mainly caused skin, gastrointestinal, and renal toxicities. Integrated evidence implied that nivolumab was the best option for safety, relying on a narrow and mild toxicity spectrum.

Table 2

Effect of treatment on each specific grade 1-5 adverse event. Values are odds ratios* unless stated otherwise

View this table:
Fig 5
Fig 5

Toxicity spectra and rankings in the subgroup analysis based on each specific grade 1-5 adverse event and cancer type. Immune checkpoint inhibitor (ICI) drugs are shown with a dark background. The number in each parenthesis represents the probability of risk to rank; values with the same rank are underlined.

Subgroup analysis based on cancer type

The subgroup analysis of lung cancer involved 12 studies and 6259 patients. Figure 6A shows that a lower risk of grade 1-5 or grade 3 or 4 adverse events was favoured by nivolumab, pembrolizumab, and atezolizumab over one ICI drug with conventional therapy and conventional therapy; conventional therapy over one ICI drug with conventional therapy; and nivolumab over pembrolizumab. The subgroup analysis of melanoma involved 10 studies and 5318 patients. Figure 6B shows that a lower risk of grade 1-5 or grade 3 and 4 adverse events was favoured by nivolumab and pembrolizumab over two ICI drugs and one ICI drug with conventional therapy; and conventional therapy over one ICI drug with conventional therapy. Figure 5 shows that all rankings were identical to those of the original network meta-analysis, with the exception of two substantial changes. Firstly, for lung cancer, nivolumab was safer than atezolizumab with an obviously higher probability in terms of grade 1-5 (70% v 60%) and grade 3 or 4 (93% v 74%) adverse events. Secondly, for melanoma, ipilimumab was less safe than conventional therapy, with an increased risk of grade 1-5 and grade 3 or 4 adverse events.

Fig 6
Fig 6

Safety profiles in the subgroup analysis based on lung cancer (A) and melanoma (B). Each cell contains the pooled odds ratios and 95% credibility intervals for grade 1-5 adverse events or grade 3 or 4 adverse events; significant results are in bold. ICI=immune checkpoint inhibitor

Sensitivity analysis

Individual sensitivity analysis of 21 high quality studies,7891011131415161718192122232526272829323337383940 26 studies explicitly reporting treatment-related adverse events,789101112131417181921222325262728293031323334353637383940 and 13 studies including patients who were previously untreated gave stable outcomes without any changes compared with the original network meta-analysis.891516182125262728293237383940 The ranking order of pembrolizumab was slightly different in the sensitivity analysis based on 21 studies that used the current recommended dosage of ICI drugs,891213141516171819212225262728293132343536373839 and this alteration only occurred between pembrolizumab and nivolumab. Individual sensitivity analysis based on 13 studies that included patients who were previously treated,712131417192223303133343536 18 phase III studies,78910111213141516171819212225263234353637 and 15 studies on programmed cell death 1 and programmed cell death ligand 1 ICI drugs suggested that nivolumab was safer than atezolizumab with a lower risk of grade 3 or 4 adverse events (supplementary materials, table S8).12131417192122252730313233343536

Discussion

ICI drugs act through distinct immunologic mechanisms and should not be regarded as one entity. Given that a comprehensive toxicity profile for ICI drugs remains to be clearly defined, we included 36 head-to-head phase II and III randomised controlled trials (15 370 patients) in the first network meta-analysis of the comparative safety of ICI drugs. The general safety of ICI drugs ranked from high to low is: atezolizumab, nivolumab, pembrolizumab, ipilimumab, and tremelimumab. There were differences in the toxicity spectra among atezolizumab (hypothyroidism, nausea, and vomiting), nivolumab (endocrine toxicities), pembrolizumab (arthralgia, pneumonitis, and hepatic toxicities), ipilimumab (skin, gastrointestinal, and renal toxicities), and tremelimumab (rash, diarrhoea, and fatigue). Integrated evidence based on the pooled incidences, subgroup, and sensitivity analyses implied that nivolumab is the safest ICI drug, especially for lung cancer.

Strengths and implications

A good exchangeability assumes that the estimates of the study effects, characteristics of participants, and the common treatment in different trials are similar and exchangeable.69 Lack of exchangeability is associated with the development of inconsistency and the disregard of similarity and transitivity. Some previous meta-analyses on this topic included placebo controlled trials and phase I trials.4647707172 Given that patients with mild disease tend to be recruited in placebo controlled trials for ethical and safety reasons,73 and phase I trials have small sample sizes focusing mainly on pharmacokinetics and pharmacometrics, we included head-to-head phase II and III trials exclusively to avoid potential bias and confounding effects. Furthermore, all these previous meta-analyses ignored the influence of different drug doses.4647707172 Combining the estimates of different doses of the same drug in a meta-analysis without knowing the individual doses can seriously damage the consistency and quality of the results. With stringent inclusion criteria and a tailored study design (dose and drug based network meta-analysis), our network meta-analysis has an excellent transitivity and the results have high validity and reliability. This information will aid clinicians to manage life threatening treatment-related adverse events, optimise future trial design, and modify the prescribing of ICI drugs.

Similar to observations from previous studies investigating the safety of ICI drugs, we confirmed that anti-cytotoxic T lymphocyte associated antigen 4 ICI drugs have generally higher toxicities than anti-programmed cell death ligand 1 ICI drugs, and one ICI drug with conventional therapy is less safe than ICI drugs on their own.7475 Of note, we have obtained some unique findings. Firstly, some treatment-related adverse events (eg, pneumonitis and hepatic toxicity) are fatal, warranting a thorough investigation of the toxicity profiles for all ICI drugs.4370 However, previous studies were limited to a superficial level, failing to resolve this issue satisfactorily. Khoja and colleagues suggested that pneumonitis, hypothyroidism, and arthralgia were more commonly associated with anti-programmed cell death 1 ICI drugs than with anti-cytotoxic T lymphocyte associated antigen 4 ICI drugs.71 Focusing specifically on each ICI drug, we found that pembrolizumab had a higher risk of pneumonitis and arthralgia; nivolumab had a higher risk of hypothyroidism. Hypotheses that can account for this finding include tumour microenvironment, expression of immune related genes, and individual response to ICI drugs, although the underlying mechanisms are not yet clear.76 A recent network meta-analysis showed non-significant differences in anti-tumour efficacy for lung cancer between nivolumab, pembrolizumab, and atezolizumab,77 particularly highlighting the importance of investigating the safety discrepancy between ICI drugs in that safety is the critical factor for drug evaluation. Secondly, this study indicated atezolizumab had the highest risk of hypothyroidism, nausea, and vomiting among the five ICI drugs, and nivolumab was the best choice based on both general (ranking) and specific (toxicity spectrum) evidence. This is somewhat surprising given that the anti-programmed cell death ligand 1 drug atezolizumab is most commonly considered to be the safest ICI drug owing to its programmed cell death ligand 2 sparing ability that preserves normal immunological homoeostasis. These findings will be helpful in selecting the most appropriate ICI drug for patients with cancer with different clinical backgrounds.

As mentioned previously, the unequal doses of ipilimumab was a major source of inconsistency, especially reflected in the direct results of ipilimumab versus conventional therapy based on one study. Since this study compared ipilimumab at 10 mg/kg every three weeks with the best supportive care (chemotherapy or no active maintenance treatment),23 the direct results cannot be representative of all doses of ipilimumab. Furthermore, the control group containing no active maintenance treatment is weaker and therefore, has fewer toxicities than full chemotherapy used as a control, which also led to inconsistency. Nonetheless, results of other ICI drugs showed satisfactory consistency and the sensitivity analysis further confirmed the stability of our results, providing assurance that this network meta-analysis has led to a better understanding of treatment-related adverse events of ICI drugs.

Limitations

Several limitations of this study should be stated. Firstly, some treatment-related adverse events are late onset diseases, at which point, patients may have already been discharged from hospital. Pneumonitis is reported to occur between 7.4 and 24.3 months after taking ICI drugs.4 Additionally, the median follow-up time was different for all included studies, ranging from 5.1 to 38.0 months, and the updated studies provide long term results. As such, the frequency of treatment-related adverse events may not only be caused by ICI drugs, but also by the confounding effect of time. Secondly, studies only reported the treatment-related adverse event with an incidence above a certain percentage ranging between ≥1% and ≥15%. One possible reason is that the method for recognising treatment-related adverse events has not been standardised, usually depending on investigators’ evaluation or self reporting by patients, or both. Thirdly, heterogeneity between the included studies—the weakness of meta-analysis—was generally present in this network meta-analysis, manifesting in the difference of follow-up time, cancer type, programmed cell death ligand 1 expression level, and so on. The studies included in this network meta-analysis used different cut-off values to determine programmed cell death ligand 1 positive patients, while a positive association has been identified between tumour programmed cell death ligand 1 expression levels and survival benefits.77 In addition, the disparity in the common comparator (eg, chemotherapy) among the included studies could have undermined the exchangeability of the network meta-analysis to some extent. Supplementary materials, table S9 details the severity of toxicity and the safety profile of different chemotherapy strategies in 24 studies.121314151617181921222325262730313233343536373839 Single agent dacarbazine and the best supportive care was associated with obviously lower incidence of grade 1-5 and grade 3 or 4 adverse events compared with other strategies, in line with a previous meta-analysis that compared the toxicity of five commonly used drugs to treat patients with melanoma.78 The study indicated that dacarbazine has the lowest incidence rates of adverse events, ranging from 0.00008 to 0.0001 cases per 100 person years.78 As most studies adopted investigator’s choice chemotherapy comprising several drugs and combinations, the chemotherapy strategies had varied and irregular safety profiles. All between-study discrepancies listed here could subject the pairwise meta-analysis results to high heterogeneity. Fourthly, the anti-programmed cell death 1 ICI drug trials were generally started later than the anti-cytotoxic T lymphocyte associated antigen 4 ICI drug trials. As the understanding and management of treatment-related adverse events have improved over time, it could lead to superior safety of anti-programmed cell death 1 ICI drugs over anti-cytotoxic T lymphocyte associated antigen 4 ICI drugs. Another caveat that is a cause for concern is the relation of cancer types and treatment-related adverse events. Patients with lung cancer are reported to have a higher risk of treatment-related pneumonitis from an ICI drug than patients with melanoma.72 Given that our findings are based entirely on clinical trials without available comorbidity data, future studies are needed for validation in the real world, in which patients with cancer are usually in a less favourable condition owing to high rates of comorbidity and competing risks.

Conclusions

In this dose and drug based network meta-analysis, clinically important differences in safety exist among ICI drugs for patients with cancer. We found that atezolizumab had the best safety profile in general, and nivolumab had the best safety profile in lung cancer when taking and integrated approach. The safety ranking of treatments based on ICI drugs is modulated by specific treatment-related adverse events.

What is already known on this topic

  • Adverse events are a cause for increasing concern regarding the comparative safety of immune checkpoint inhibitor (ICI) drugs in the treatment of patients with cancer

  • Different ICI drugs and doses can have different toxicity features

  • Pairwise meta-analyses are limited to comparisons of two drugs, failing to generate a complete picture of the toxicity profile of ICI drugs

What this study adds

  • Based on 36 eligible studies and 15 370 patients, the rank of ICI drugs from high to low with respect to general safety assessed by grade 1-5 or grade 3 or 4 adverse events is: atezolizumab (pooled incidence of 66.4% and 15.1%, respectively), nivolumab (71.8% and 14.1%), pembrolizumab (75.1% and 19.8%), ipilimumab (86.8% and 28.6%), and tremelimumab (not applicable)

  • Nivolumab had the best safety profile in lung cancer when taking an integrated approach

Acknowledgments

We thank Brigette BY Ma, from the Chinese University of Hong Kong, for her helpful comments on this study.

Footnotes

  • Contributors: The first four authors (CX, Y-PC, X-JD, and J-QL) contributed equally to this work. JM conceived, designed, and supervised the project. JM performed the quality assessment, review, and approval of the manuscript. CX, X-JD, Y-PC, and J-QL contributed to the design of the study, writing the protocol, data preparation, analysis and interpretation, and writing and final approval of the report. CX and X-JD identified eligible trials, extracted the data, and assessed the quality of included trials. CX, J-QL, C-LH, L-LT, LC, G-QZ, W-FL, and Y-PM processed the data and generated the tables and figures. CX, A-HL, and QL were responsible for statistical analysis and interpretation. CX, Y-PC, L-LT, and LC contributed to the interpretation of the data. CX, X-JD, and J-QL drafted the manuscript. CX, Y-PC, YS, and CH revised the manuscript before submission. CX, Y-PC, and J-QL participated in the formal revision, including statistical analysis, data processing, production of figures and tables, and text modification. All authors had full access to the data in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. JM is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding: This work was supported by grants from the Natural Science Foundation of Guang Dong Province (No 2017A030312003), Health and Medical Collaborative Innovation Project of Guangzhou City (No 201803040003), Innovation Team Development Plan of the Ministry of Education (No IRT_17R110), and Overseas Expertise Introduction Project for Discipline Innovation (111 Project, No B14035). The sponsors or funders had no involvement in any parts of this study. All authors confirmed the independence of researchers from funding sources.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from the Natural Science Foundation of Guang Dong Province, Health and Medical Collaborative Innovation Project of Guangzhou City, Innovation Team Development Plan of the Ministry of Education, and Overseas Expertise Introduction Project for Discipline Innovation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

  • Ethical approval: Not required.

  • Data sharing: No additional data available.

  • The manuscript’s guarantor (JM) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned and registered have been explained.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

References