Randomised clinical trials (RCTs) and real-world (RW) studies: pros and cons
| Key element | RCTs | RW studies |
| Learning curve | Not considered Impacts can be biased if the protocol enhances the standardisation of the intervention | Can be controlled if the initial experience of the physician is known Could be useful to start data collection from the first intervention |
| Selection bias | Not applicable by design Nevertheless, some open-label studies allow patients to self-select In this case, it is possible to generate selection bias However, in the latter case, selection bias is seldom discussed in the study | The most relevant issue Appropriate data collection and statistical procedures can significantly reduce it Bias reduction can be measured and reported |
| Adherence to the real clinical practice | No: if the RCT's protocol serves to determine if a causal relationship between the technological innovation use and outcomes exists, it is often too distant from actual practice, especially in terms of patients' selection criteria and treatment protocol | Yes |
| Comparator | Head-to-head trials are not mandatory and placebo is often used This has increased the use of ex-post indirect comparisons (i.e. network meta-analysis) that often lead to opposite recommendations [23] | Always a direct comparator, used in real clinical practice (normally standard of care) |
| Costs | Generally high It may increase when replications are needed due to versioning | Low if the study is retrospective Comparable to an RCT if the study is prospective |
| Ethical issues | Requires approval by ethical committees because it alters clinical practice | Clinical practice is not altered and ethical issues mostly regard patients' privacy |
| Economic evaluation | Feasible ex-post Seldom, costs are also collected during the experimental phase but often irrelevant since protocol driven | Feasible in parallel for prospective studies For retrospective studies, costs are collected ex-post (however, the costs reflect clinical practice closer than in the case of RCTs) |
| Sample | Minimum required by the expected magnitude of the effect | Large Depending on the statistical requirements, a RW analysis may need many observations (because many observations might be lost if common support is imposed by PSM) or data regarding the period prior to the intervention (difference-in-differences) |
| Time to results | It depends on the protocol | Depends on the protocol (prospective studies) Short for retrospective studies Ideal for follow-up studies |
| Internal validity | High | Always inferior to RCTs However, appropriate study design and statistical methods can substantially increase RW study internal validity |
| External validity | Low The conditions of the RCT do not reflect those where the policy decision would apply | High The conditions of RW studies reflect routine practice |
PSM: propensity score matching.