Abstract
The magnitude of treatment effect can be assessed by a number of methods. One method of collectively analysing data is that used by the Cochrane Collaboration. Their systematic reviews identify, analyse and present research-based evidence in an accessible format. These reviews may contain meta-analyses combining data from multiple studies to provide robust evaluations of overall treatment effects.
In 2003, Cochrane reviews of data for treatment with corticosteroids in idiopathic pulmonary fibrosis (IPF) found no evidence supporting their use; similarly, reviews of immunomodulatory agents found very little evidence to support their use.
A recent update of these Cochrane reviews failed to identify any evidence supporting the use of corticosteroids in IPF; however, a review of non-steroid agents in the treatment of IPF identified 15 clinical trials suitable for analysis. Two trials of interferon-γ-1b were combined, and no treatment effect was observed in terms of survival. Two Japanese trials of treatment with pirfenidone were combined, and a positive effect of pirfenidone on pulmonary function decline was observed. Meta-analysis of three phase III studies suggested that pirfenidone significantly increased progression-free survival by 30%.
The findings of this systematic review, although not presenting new original data, together with an acceptable safety profile, suggest that pirfenidone may have a role in IPF treatment.
Until approximately 40 yrs ago, clinical practice was not based on evidence derived from the combined results of randomised clinical trials. A seminal book published in 1972 by the British professor Archibald Cochrane [1] highlighted this fact, and stressed the importance of using evidence from randomised controlled trials (RCTs) to guide daily clinical practice. He suggested that RCTs were likely to provide more highly reliable information than other potential sources of evidence. This book became the basis for the way in which the UK National Health Service provided treatment. Clinical practice today takes into account the results published from RCTs, as well as recommendations issued by expert committees; however, the main issue in obtaining accurate treatment guidelines is how to combine data from various clinical trials in order to assess the real benefits of any given therapy.
HOW TO ACCURATELY ASSESS THE EFFECT OF A THERAPY
Combining data from different clinical trials for one given therapy or disease area can often be difficult, even when trials are similar in design. Some trials may not be large enough to provide significant data, and others may have non-comparable end-points. One approach is to pool data from similarly designed studies. For example, two phase III trials of pirfenidone in patients with idiopathic pulmonary fibrosis (IPF) [2] were similarly designed in order to enable pooling of data. Analysis of pooled data from both trials was conducted to derive precise estimates of the magnitude of the treatment effect [2]. The number of clinical trials carried out in IPF is now accumulating rapidly, and a decision should soon be made on the recommended treatment strategy based on evidence emerging from multiple trials. However, the question remaining is: what is the ideal method to assess the overall effect of a therapy based on multiple different trials?
THE COCHRANE COLLABORATION
Following the publication in 1972 of the book Effectiveness and Efficiency by Cochrane [1], a non-profit network known as the Cochrane Collaboration was established with the aim of analysing existing data on various treatment effects. The Cochrane Collaboration publishes systematic reviews, which identify, analyse and present research-based evidence in an accessible format [3]. The Cochrane reviews differ from classical scientific reviews (also known as non-systematic reviews) as they do not simply reflect the opinions of the authors; they aim to reduce bias possibly generated by individual opinions or practices. Cochrane reviews are based on a systematic analysis of all published data for a given treatment, which is based on a pre-defined analysis and may contain a meta-analysis. Meta-analysis is a statistical approach to aggregating data from relevant clinical trials and, thus, providing more precise estimates of the effects of a given treatment. There are pre-defined criteria for the inclusion of trials in a Cochrane meta-analysis. Meta-analyses are typically conducted on a study end-point and graphically depicted as “forest plots” [4]. The forest plot shows the effect of different studies on a given end-point. The vertical line in the middle of the forest plot represents the line of “no effect”. The areas to either side of the no-effect line are representative of the individual studies being either favourable to the treatment or to placebo. Each study is evaluated and represented on the plot by a square (which shows the effect of the individual study, with the square varying in size to account for the weight any one study has on the overall analysis, i.e. the larger the square the more weight the study has) and line (representing the 95% confidence interval of a study). The diamond at the bottom of the plot represents the pooled result (overall effect) of all studies, with the outer edges representing the 95% confidence intervals.
A classic example of the importance of meta-analysis is the study published in 1992 by Antman et al. [5] on the effects of oral β-blockers for the secondary prevention of mortality in patients surviving a myocardial infarction (fig. 1). This study showed that a number of trials on this specific topic were unnecessarily carried out, because a meta-analysis of the previous studies wasn't performed. Such an analysis, if timely, would have identified the use of β-blockers as the standard of care for heart failure patients, thus avoiding the performance of many other placebo-controlled trials.
Effects of oral β-blockers for a secondary prevention of mortality in patients surviving a myocardial infarction Individual randomised clinical trials and overall meta-analysis result. z=-4.47, p<0.00001. Reproduced from [5] with permission from the publisher.
PHARMACOLOGICAL TREATMENT OF IPF
In 2000, the American Thoracic Society and European Respiratory Society published an international consensus statement with guidelines for the diagnosis and treatment of IPF [6]. The guidelines recommended that until adequate studies were conducted to define the best treatment for patients with IPF, they should be treated with a combined therapy of corticosteroids and an immunosuppressive agent (e.g. azathioprine or cyclophosphamide). In 2003, a Cochrane review was published on the use of corticosteroids in IPF [7]. This review highlighted that there were no placebo-controlled clinical trials analysing the efficacy of corticosteroid therapy in IPF patients and, therefore, there was no existing evidence to support the efficacy of corticosteroids alone in the treatment of IPF.
Moreover, Davies et al. [8] published a Cochrane review in 2003 on the use of immunomodulatory agents in IPF. The authors found four randomised controlled clinical studies suitable for a meta-analysis [9–12]; however, these four small studies used four different immunosuppressive agents (cyclophosphamide, azathioprine, colchicine and interferon (IFN)-γ-1b) and thus it was not actually possible to conduct a meta-analysis. The authors concluded that there was little evidence to justify the routine use of any immunosuppressive agent (or in fact any non-corticosteroid agent) in the management of IPF, either as sole therapy or as steroid-sparing agents.
In 2010, the Cochrane meta-analysis on the use of corticosteroids for IPF was updated [13]. However, 7 yrs after the publication of the first Cochrane review on the use of corticosteroids in IPF, there was still no evidence supporting the routine use of corticosteroids alone in the management of IPF [13]. Since 2003, no placebo-controlled trials of corticosteroids in IPF have been undertaken, probably due to the fact that there was no previous clear evidence to show that they were or were not an effective treatment option. Therefore, clinicians were left with continued uncertainty over the use of these agents in IPF.
In 2010, a systematic search was conducted to identify RCTs carried out using non-steroid agents for the treatment of IPF. This then became the subject of an updated Cochrane review [14]. The search for relevant clinical trials involved the use of various trial registries and databases, as well as direct contact with pharmaceutical companies and researchers in this field. The end-points of trials identified as potentially suitable for inclusion were reviewed, and their methodological quality assessed. This systematic search found that up to 2003, four clinical studies were eligible for a meta-analysis, while in 2010, 13 studies could be included. Contact with pharmaceutical companies and researchers led to identification of two further clinical trials which were suitable for meta-analyses (fig. 2) and were to be published soon after. The analysis of end-points and quality of the methodology of these 15 trials reduced the number of eligible trials to seven (fig. 3): only the anti-fibrotic agents IFN-γ-1b and pirfenidone were evaluated in more than one trial and, as such, were potentially eligible for two meta-analyses.
Two systematic searches for randomised clinical trials of non-steroid agents in idiopathic pulmonary fibrosis (IPF) up to 2003 and up to 2010. IFN: interferon.
Randomised clinical trials of non-steroid agents in idiopathic pulmonary fibrosis eligible for meta-analyses. IFN: interferon.
Combining the data from the two RCTs on IFN-γ-1b [15, 16] in a meta-analysis of the clinical end-point of overall survival (both trials reported hazard ratio estimates) showed that there were no statistically significant differences in mortality between IFN-γ-1b and placebo (fig. 4). Interestingly, the larger of the two trials, published by King et al. [15], was negative for efficacy in terms of the overall survival end-point, whereas the smaller trial almost demonstrated statistical significance [16].
Meta-analysis of overall survival in idiopathic pulmonary fibrosis with interferon (IFN)-γ-1b. HR: hazard ratio; df: degree of freedom. Reproduced from [14] with permission from the publisher.
Meta-analyses were performed to determine the treatment effect of pirfenidone. The two Japanese clinical trials [18] could be combined in an analysis on the end-point of absolute change in vital capacity (fig. 5) [16]. Statistically significant differences were observed in terms of decline in vital capacity between the pirfenidone and placebo groups. This meta-analysis confirmed the beneficial effect of pirfenidone on the change of vital capacity compared to baseline. The CAPACITY 1 and 2 studies [2] measured the end-point of change in per cent predicted forced vital capacity (FVC), with the results being analysed using a rank analysis, and thus the heterogeneity of these studies did not allow for their inclusion in the same meta-analysis.
Meta-analysis of absolute change in vital capacity for pirfenidone in idiopathic pulmonary fibrosis. Reproduced from [14] with permission from the publisher.
For the combined end-point of progression-free survival (PFS), progression being defined as either death or 10% decrease in FVC, data from the CAPACITY 1 and 2 studies and the Japanese study published by Taniguchi et al. [18] could be combined in a meta-analysis (fig. 6). The CAPACITY 1 and 2 studies reported the hazard ratio (HR) estimate and 95% confidence intervals (CIs). The Japanese study did not report the HR; however, this was estimated indirectly using the information from the survival curves reported by the authors. In CAPACITY 2 and the study by Taniguchi et al. [18], similar and significant differences were observed in PFS between the pirfenidone and placebo arms. There was, however, a non-significant effect in terms of PFS observed in CAPACITY 1. The overall result of this meta-analysis suggests that treatment with pirfenidone reduced the risk of disease progression by 30% (HR 0.70, 95% CI 0.56–0.88).
Meta-analysis of progression-free survival with pirfenidone in idiopathic pulmonary fibrosis. Reproduced with modificaton from [14] with permission from the publisher.
It is noteworthy that a recent publication by Vancheri et al. [24] observed that IPF has similarities and links to the biology of neoplastic disorders. In light of the biological and prognostic similarities between IPF and lung cancer, there may be a rationale for adopting PFS as a primary end-point in therapeutic clinical trials in IPF, especially as many commonly used drugs for the treatment of nonsmall cell lung cancer were approved on the basis of an effect on PFS [25–27]. Indeed, the magnitude of the treatment effect of pirfenidone on PFS in the previously referenced meta-analysis was generally consistent with the magnitude of effect of adjuvant platinum-based chemotherapy in a meta-analysis of 10 RCTs in patients with nonsmall cell lung cancer (HR 0.87, 95% CI 0.79–0.94) [28].
CONCLUSIONS
With the number of good-quality randomised controlled clinical trials in IPF increasing, systematic reviews containing meta-analyses recently became feasible. The advantages and limitations of meta-analysis do, however, need to be appreciated. For example, it is not always possible to combine the results from all trials that have investigated the treatment of a particular disease if they differ in terms of end-points and analysis. Also, it is important to note that the results of meta-analyses do not constitute new clinical data, since they are based on the combination of previously performed studies: as such, the results of meta-analyses should be considered carefully before being directly applied to clinical practice. Nonetheless, systematic reviews are helpful to identify previously unrecognised missing gaps or to confirm results from smaller studies. For example, meta-analysis of the efficacy of corticosteroids in IPF has shown that there is no evidence to support their use. In fact, the most recently published 2011 American Thoracic Society/European Respiratory Society/Japanese Respiratory Society/Latin-American Thoracic Society guidelines for the diagnosis and management of IPF gave a strong recommendation against corticosteroid monotherapy [29]. Indeed, the available combined data point to a role for pirfenidone in treating IPF patients. Bearing in mind that the ultimate goal is successfully treating IPF, evidence from systematic reviews should play a role in designing future clinical trials and in fostering international networking, which is key to understanding and improving the treatment of IPF.
Footnotes
Statement of Interest
L. Richeldi has received consultancy fees from Boehringer Ingelheim, InterMune, Celgene and Gilead, and lecture fees from InterMune.
Provenance
Publication of this peer-reviewed article was sponsored by InterMune International AG, Muttenz, Switzerland (article sponsor, European Respiratory Review issue 124).
- Received February 3, 2012.
- Accepted February 21, 2012.
- ©ERS 2012
REFERENCES
