TABLEĀ 1

Randomised clinical trials (RCTs) and real-world (RW) studies: pros and cons

Key elementRCTsRW studies
Learning curveNot considered
Impacts can be biased if the protocol enhances the standardisation of the intervention
Can be controlled if the initial experience of the physician is known
Could be useful to start data collection from the first intervention
Selection biasNot applicable by design
Nevertheless, some open-label studies allow patients to self-select
In this case, it is possible to generate selection bias
However, in the latter case, selection bias is seldom discussed in the study
The most relevant issue
Appropriate data collection and statistical procedures can significantly reduce it
Bias reduction can be measured and reported
Adherence to the real clinical practiceNo: if the RCT's protocol serves to determine if a causal relationship between the technological innovation use and outcomes exists, it is often too distant from actual practice, especially in terms of patients' selection criteria and treatment protocolYes
ComparatorHead-to-head trials are not mandatory and placebo is often used
This has increased the use of ex-post indirect comparisons (i.e. network meta-analysis) that often lead to opposite recommendations [23]
Always a direct comparator, used in real clinical practice (normally standard of care)
CostsGenerally high
It may increase when replications are needed due to versioning
Low if the study is retrospective
Comparable to an RCT if the study is prospective
Ethical issuesRequires approval by ethical committees because it alters clinical practiceClinical practice is not altered and ethical issues mostly regard patients' privacy
Economic evaluationFeasible ex-post
Seldom, costs are also collected during the experimental phase but often irrelevant since protocol driven
Feasible in parallel for prospective studies
For retrospective studies, costs are collected ex-post (however, the costs reflect clinical practice closer than in the case of RCTs)
SampleMinimum required by the expected magnitude of the effectLarge
Depending on the statistical requirements, a RW analysis may need many observations (because many observations might be lost if common support is imposed by PSM) or data regarding the period prior to the intervention (difference-in-differences)
Time to resultsIt depends on the protocolDepends on the protocol (prospective studies)
Short for retrospective studies
Ideal for follow-up studies
Internal validityHighAlways inferior to RCTs
However, appropriate study design and statistical methods can substantially increase RW study internal validity
External validityLow
The conditions of the RCT do not reflect those where the policy decision would apply
High
The conditions of RW studies reflect routine practice
  • PSM: propensity score matching.