Real-world evidence, (RWE), also known as observational data, was once the poor relation of experimental research. The clinical community often shunned RWE based on its lack of scientific rigor, and relegated it to a lowly position in research hierarchies. But in this digital era, RWE has become increasingly useful, and available, during clinical trials and beyond. Those working in the HTA domain have increasingly embraced this new source of evidence for manifold reasons.

What is Real-World Evidence?

The short answer is: any form of data that isn’t from a Randomized Clinical Trial (RCT) or trial evidence synthesis. This type of evidence is broad and includes anything from pragmatic or uncontrolled trials, registries, administrative data, health surveys, and electronic health records.

Real-World Evidence and Health Technology Assessment

Health Technology Assessment, (HTA), is heavily dependent on RWE because it usually involves constructing decision models. Populating decision models is complex and it requires an increasingly diverse range of data sources. It is fair to say that RWE has always underpinned HTA processes to an extent, because HTA often relies on outcomes that are not measured or available from a systematic review or clinical trial. Specifically, costs, patient-related outcomes, disease progression rates, and rare adverse events.

For example, RWE can play a role in designing the research question to enhance the effectiveness of clinical trials. It can be used to, (1) identify the best population for the trial; (2) identify the most valuable place in therapy; (3) identify the appropriate comparator to represent standard care; (4) define the way in which the intervention is delivered in practice and; (5) understand which outcomes are most important to stakeholders. 

Real-World Evidence and Cost Data in Health Technology Assessment

Several jurisdictions consider cost in HTA. Clinical trials seldom yield generalisable cost data. Even if costs (resource use) are measured, they are applicable only to the jurisdiction they are gathered in. Local surveys, administrative data, prescription databases, and expert opinion are often RWE that is essential in estimating cost data for HTAs.

Patient-Related Outcomes

Similarly, gathering outcomes data in the correct form for international HTA submissions is challenging when relying on RCT data alone. Jurisdictions vary in their approaches to valuing patient outcomes – some focus on clinical effectiveness, whilst others prefer health-related quality of life (HRQoL) measures. Trials generally measure clinical effectiveness, but sometimes apply an interim or surrogate marker, such as blood pressure, rather than longer-term mortality data. Even when quality of life is measured, the data may not meet every HTA requirement as multiple measures exist. The National Institute of Health and Care Excellence (NICE) insists on the use of quality-adjusted life years (QALY) to measure both the quantity and quality of life, with the EuroQol-5D questionnaire (EQ-5D) being the preferred instrument. This presents problems because whilst the EQ-5D is generic, it is insensitive to disease-specific outcomes that are considered important to patients with certain conditions. Additionally, the EQ-5D does not resonate with the clinicians who are instrumental in gathering trial data. Consequently, EQ-5D data gathering in Phase 3 clinical trials is often suboptimal. Those gathering the data may be reluctant to prioritise this measure because they don’t always understand it and are hesitant to overburden patients, particularly when they are nearing the end of life. Therefore, RWE is often used to fill the evidence gaps surrounding outcomes.

Why are Clinical Trials Considered the Holy Grail?

Many consider RCTs to be the most valid approach to evaluating therapeutic efficacy and trials have traditionally been the cornerstones of securing regulatory approval. RCTs are prospectively designed to compare a well-specified intervention in a clearly specified and restricted population, with pre-defined endpoints often over a short period. Importantly, the trial population is randomized between the intervention and the comparator arms to ensure that the factors that can influence the outcome of interest are evenly distributed. Consequently, the study outcome can be attributed to the intervention. Real-world evidence can include pragmatic clinical trials, which are clinical trials set in more realistic settings. This type of RWE is said to measure effectiveness, that is how a treatment works in the real world.

Is Real-World Evidence Reaching the Evidence Randomized Clinical Trials (RCTs) Cannot or Do Not?

Not only is RWE invaluable in trial development, the HTA process, and for providing cost and outcome data, but it is also useful in providing longer term data beyond the truncated RCT period. Randomized clinical trials are time and resource consuming and are consequently often short-lived. Frequently, longer term outcomes are captured using registers or other sources of cohort data. Similarly, the incidence of adverse event data and safety outcomes are often sourced in this way as these outcomes can be rare and occur over a protracted period after an RCT has finished.

One of the key difficulties that evidence hierarchies ignore is that different types of questions require different types of evidence. The RCT is designed to answer certain research questions because RCTs are conducted in an idealised environment, for short periods of time, and in specified populations. Decision makers are aware that this type of evidence does not reflect all aspects of clinical practice because patients, healthcare professionals, and healthcare systems are heterogeneous and seldom ideal. The hierarchy of evidence that has long been in place is beginning to shift. The Scottish Intercollegiate Guidelines Network encapsulates this sentiment in the statement: The accepted grading system was designed for application of efficacy; however, for medical practice settings, the RCT may not be practical, nor may it provide the best evidence. Guideline users may misinterpret the grade of recommendation or they may fail to properly weigh lower grade recommendations.

Examples of Real-World Evidence Informing Health Technology Assessment

The National Institute for Health and Care Excellence (NICE) guidance TA279 provides multiple examples of the diversity of evidence sources that can be used to inform HTAs. In this example, the technology appraisal made an optimised recommendation for percutaneous vertebroplasty and percutaneous balloon kyphoplasty without stenting, for treating osteoporotic vertebral compression fracture. The company and Evidence Review Group (ERG) used U.S. Medicare claims data to obtain estimates of the relative risk of death. Additionally, costs were derived using hospital episode statistics and National Health Service (NHS) reference costs. Furthermore, expert clinical opinion informed the ERG economic model. Specifically, a clinical specialist advised that approximately 15% of procedures would use high-viscosity cement or other, more expensive cement types.

The recent NICE guidance TA474, issued for Sorafenib for treating advanced hepatocellular carcinoma, used data from an observational study to validate survival extrapolations from the company’s original submission. The Sorafenib Hepatocellular Carcinoma Assessment Randomized Protocol (SHARP) study, which formed the basis of the company’s submission, was stopped early at the second interim analysis, potentially underestimating the survival benefit of Sorafenib.

NICE guidance TA431 for Mepolizumab, for treating severe refractory eosinophilic asthma, recognised that the starting age for treatment was an important driver of the model. The committee noted that the company presented a scenario with a starting age of 30 years, which increased the company’s base-case incremental Cost-Effectiveness Ratio (ICER). The clinical experts stated that 30 years was younger than the people that they saw in clinical practice in the U.K. The company used U.K. registry data from the British Thoracic Society, a cross-sectional registry data, and a historical cohort study to validate that the patient age at onset was likely to be 34.5 years.


Real-world evidence is increasingly viewed as an important part of the HTA process, especially in this digital era. It is proving to be an invaluable source of data to fill evidence gaps and parameterize models. Real-world evidence is not without its limitations, however, and we are only just beginning to understand how to deal with the complexities it introduces. But the message is loud and clear; RWE is here to stay and will have considerable implications for HTA.

DRG experts like Wendy routinely help our clients build real-world evidence that serves their business needs. Visit our real-world evidence webpage to download a snapshot of the healthcare data landscapes in three countries!

How Glympse Bio oversubscribed their Series B funding amidst the pandemic

View Now