Standard methods for indirect comparisons and network meta-analysis are based on aggregate data obtained from clinical studies and the key assumption behind such methods is that there is no difference between the trials in terms of the distribution of treatment effect-modifying variables. However, in practice high levels of heterogeneity are often noted within evidence networks. Particularly within sparse networks, the validity of indirect comparisons must be carefully considered as they can be vulnerable to bias due to imbalances in effect modifier distributions across trials.

Population-adjusted methods for indirect comparisons which adjust for between-trial differences in the distribution of variables which may influence outcomes are therefore highly desirable. The National Institute for Health and Care Excellence (NICE) decision support unit (DSU) have recently released technical support document 18 (TSD18) which examines these methods (1).

In population-adjusted indirect comparisons, individual patient level data (IPD) in one or more trials are used to adjust for between trial differences in the distribution of variables that influence outcomes. Matching adjusted indirect comparison (MAIC) and simulated treatment comparison (STC) are two examples of methods of population-adjusted indirect comparisons which are reviewed within TSD18 (1). According to the authors of TSD18 there are currently 10 published peer reviewed applications of MAIC and one of STC, and the use of these analyses to support submissions to NICE is increasing (1). However, the authors also note that little is known about the reliability or the general properties of these methods, particularly in the context of NICE technology appraisals (1).

Based on general principles and empirical findings, the document outlines some provisional recommendations as to the role of population-adjusted estimates of treatment effects in submissions to NICE (1). Briefly, these cover:

  • Rationale for the use of population adjustment in submissions
  • Justifying the use of population adjustment in both anchored and unanchored scenarios
  • Variables for which population adjustment is required
  • Generation of indirect comparisons for the appropriate target population
  • Reporting guidelines for analyses involving population adjustment

Whilst there is a potential role for population-adjusted comparisons in health technology assessment there remains a lack of clarity around how and when these methods should be applied in practice.  Further research is needed to develop these methods further; to assess the comparative vulnerability to failures of any assumptions; to find ways of estimating systematic error due to unaccounted covariates; to ensure appropriate uncertainty propagation in population-adjusted estimates and to prepare suitable software tools with a range of worked examples (1).

A recent research placed DRG Abacus as one of the leading consultancies in the area of Systematic Review and Network Meta-Analysis. Get in touch with our expert SR&NMA consultants at

1. Phillippo D, Ades, T., Dias, S., Palmer, S., Abrams, K. &,Welton, N.J.,. NICE DSU Technical Support Document 18:Methods for population-adjusted indirect comparisons in submissions to NICE. NICE Decision Support Unit, 2016 (Technical Support Documents). 2016.

DRG becomes Clarivate

View Now