What type of clinical evidence is needed to assess medical devices?

Rosanna Tarricone; Paola Roberta Boscolo; Patrizio Armeni

doi:10.1183/16000617.0016-2016

Tables

Additional Files

TABLE 1

Randomised clinical trials (RCTs) and real-world (RW) studies: pros and cons

Key element	RCTs	RW studies
Learning curve	Not considered Impacts can be biased if the protocol enhances the standardisation of the intervention	Can be controlled if the initial experience of the physician is known Could be useful to start data collection from the first intervention
Selection bias	Not applicable by design Nevertheless, some open-label studies allow patients to self-select In this case, it is possible to generate selection bias However, in the latter case, selection bias is seldom discussed in the study	The most relevant issue Appropriate data collection and statistical procedures can significantly reduce it Bias reduction can be measured and reported
Adherence to the real clinical practice	No: if the RCT's protocol serves to determine if a causal relationship between the technological innovation use and outcomes exists, it is often too distant from actual practice, especially in terms of patients' selection criteria and treatment protocol	Yes
Comparator	Head-to-head trials are not mandatory and placebo is often used This has increased the use of ex-post indirect comparisons (i.e. network meta-analysis) that often lead to opposite recommendations [23]	Always a direct comparator, used in real clinical practice (normally standard of care)
Costs	Generally high It may increase when replications are needed due to versioning	Low if the study is retrospective Comparable to an RCT if the study is prospective
Ethical issues	Requires approval by ethical committees because it alters clinical practice	Clinical practice is not altered and ethical issues mostly regard patients' privacy
Economic evaluation	Feasible ex-post Seldom, costs are also collected during the experimental phase but often irrelevant since protocol driven	Feasible in parallel for prospective studies For retrospective studies, costs are collected ex-post (however, the costs reflect clinical practice closer than in the case of RCTs)
Sample	Minimum required by the expected magnitude of the effect	Large Depending on the statistical requirements, a RW analysis may need many observations (because many observations might be lost if common support is imposed by PSM) or data regarding the period prior to the intervention (difference-in-differences)
Time to results	It depends on the protocol	Depends on the protocol (prospective studies) Short for retrospective studies Ideal for follow-up studies
Internal validity	High	Always inferior to RCTs However, appropriate study design and statistical methods can substantially increase RW study internal validity
External validity	Low The conditions of the RCT do not reflect those where the policy decision would apply	High The conditions of RW studies reflect routine practice