Health Economics

Health Economic evaluations commonly use observational studies, either alongside or instead of data from randomised controlled trials (RCTs). A major concern is that the results suffer from treatment selection bias due to confounding variables that influence both treatment and outcomes. Commonly used analytical methods for dealing with confounding such as regression analysis or propensity score matching can be highly sensitive to model specification.

We are undertaking research on improving methods in economic evaluations for dealing with confounding. Our recent and ongoing research programme in this area, covers the following topics:

– An assessment of the relative performance of a multivariate matching approach, Genetic Matching in contrast to more conventional propensity score matching estimators [1-4]

– An evaluation of double-robust methods for addressing confounding in evaluations of effectiveness and cost-effectiveness [5-6]

– An investigation of a machine learning approach for evaluating the effectiveness of continuous treatments [7]

– A critical examination of the relative merits of the synthetic control method [8] for estimating treatment effects in longitudinal settings [8-9]

– Comparing approaches for addressing time-varying confounding when informing sequential decisions

– Considering mediation approaches in the context of decision-modelling and cost-effectiveness analysis?

Key people: Richard Grieve, Noemi Kreif, Stephen O’Neill, Neil Hawkins.

Recommendations encourage cost-effectiveness analyses (CEAs) to report intention to treat (ITT) estimates, but this may be insufficient for policy making as new decision problems may arise subsequent to the trial design. Clinical decision-makers also require cost-effectiveness estimates for patients who meet particular treatment thresholds, or for lower levels of compliance, which better reflect routine practice. Per protocol (PP) analyses are common but will provide biased estimates.

We aim to improve methods for estimating causal treatment effects after deviation from protocol. In particular, the instrumental variables (IV) approach we propose can deal with non-compliance in settings where there are multiple endpoints.

Key people: Richard Grieve

Evaluations that use RCTs lack external validity if the RCT and the target population differ according to patient and provider characteristics that modify relative cost-effectiveness. We lead NIHR-funded evaluations where clinical investigators suggest the potential lack of external validity could stop them applying the results in practice. We have developed methods that make plain the assumptions required to generalise effectiveness and cost-effectiveness estimates from an RCT, to target populations of prime interest [10].

Key people: Richard Grieve

In economic evaluations a common problem is that there are missing data, for example, because patients are lost to follow-up or fail to respond to quality-of-life or resource use questionnaires. Missing data may be problematic because individuals with missing information tend to be systematically different from those with complete data. Most published studies fail to address this issue and report cost-effectiveness inferences based solely on the complete cases. Inappropriate methods will lead to biased results, and ultimately can affect the decision of whether an intervention should be prioritised.

While standard multiple imputation methods have been proposed for handling missing data in cost-effectiveness analysis, these may be insufficient in many settings. For example, they assume that individual observations are independent (which may be implausible in multicentre studies or meta-analysis of individual-participant data) or that the imputation model is correctly specified. In addition, the methods proposed assumed that data are missing at random, i.e. the probability of missingness is only conditional on the observed data. However, the probability of missing costs or outcomes may depend on unobserved values, i.e. data may be missing not at random.

We focus on the following areas:

– Comparing different approaches for dealing with clustering when addressing missing data in CEA [11].

– Assessing novel ‘doubly robust’ approaches that minimise the reliance on model specification when handling missing data in CEA that use non-randomised studies.

– Delineating strategies for dealing with cost-effectiveness data missing not at random.

– Developing tools to elicit expert opinion to inform sensitivity analysis to assumptions about the missing data in CEA studies.

– Investigating how untestable assumptions about data missing not at random link multiple imputation and Heckman selection models.

Key people: Alexina Mason, Manuel Gomes, Richard Grieve

CEAs often use data from cluster randomised trials (CRTs), where randomisation is at the level of the cluster (for example the hospital) rather than the individual. Here, statistical methods are required that recognise within-cluster costs and health outcomes may be correlated. However, most CEA alongside CRTs use methods that assume observations are independent, which may lead to incorrect inferences.

Our research interests include:

– Developing appropriate methods for CEA that use CRTs and provide guidance on their use [12-13]

– Investigating approaches for handling missing values that recognise the hierarchical structure of the data such as random-effects and fixed-effects multiple imputation [14]

Key people: Manuel Gomes, Richard Grieve

1. Sekhon, Jasjeet. Multivariate and propensity score matching with automated balance search. Journal of Statistical Software 2011

2. Sekhon, Jasjeet, and Richard D. Grieve. “A matching method for improving covariate balance in cost‐effectiveness analyses.”Health economics 21, no. 6 (2012): 695-714.

3. Radice, Rosalba, Roland Ramsahai, Richard Grieve, Noemi Kreif, Zia Sadique, and Jasjeet S. Sekhon. “Evaluating treatment effectiveness in patient subgroups: a comparison of propensity score methods with an automated matching approach.”The international journal of biostatistics8, no. 1 (2012).

4. Kreif, Noemi, Richard Grieve, Rosalba Radice, Zia Sadique, Roland Ramsahai, and Jasjeet S. Sekhon. “Methods for estimating subgroup effects in cost-effectiveness analyses that use observational data.”Medical Decision Making 32, no. 6 (2012): 750-763.

5. Kreif, Noémi, Richard Grieve, Rosalba Radice, and Jasjeet S. Sekhon. “Regression-adjusted matching and double-robust methods for estimating average treatment effects in health economic evaluation.”Health Services and Outcomes Research Methodology13, no. 2-4 (2013): 174-202.

6. Kreif, Noémi, Susan Gruber, Rosalba Radice, Richard Grieve, and Jasjeet S. Sekhon. “Evaluating treatment effectiveness under model misspecification: a comparison of targeted maximum likelihood estimation with bias-corrected matching.”Statistical methods in medical research(2014): 0962280214521341.

7. Kreif, Noémi, Richard Grieve, Iván Díaz, and David Harrison. “Evaluation of the Effect of a Continuous Treatment: A Machine Learning Approach with an Application to Treatment for Traumatic Brain Injury.”Health economics24, no. 9 (2015): 1213-1228.

8. Abadie, Alberto, Alexis Diamond, and Jens Hainmueller. “Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program.”Journal of the American Statistical Association105, no. 490 (2010).

9. Kreif, Noémi, Richard Grieve, Dominik Hangartner, Alex James Turner, Silviya Nikolova, and Matt Sutton. “Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units.”Health economics (2015). doi: 10.1002/hec.3258.

10. Hartman, Erin, Richard Grieve, Roland Ramsahai, and Jasjeet S. Sekhon. “From sample average treatment effect to population average treatment effect on the treated: combining experimental with observational studies to estimate population treatment effects.”Journal of the Royal Statistical Society: Series A (Statistics in Society)178, no. 3 (2015): 757-778.

11. Díaz‐Ordaz, Karla Michael G. Kenward, and Richard Grieve. “Handling missing values in cost effectiveness analyses that use data from cluster randomized trials.”Journal of the Royal Statistical Society: Series A (Statistics in Society)177, no. 2 (2014): 457-474.

12. Gomes, Manuel, Edmond S-W. Ng, Richard Grieve, Richard Nixon, James Carpenter, and Simon G. Thompson. “Developing appropriate methods for cost-effectiveness analysis of cluster randomized trials.”Medical Decision Making 32, no. 2 (2012): 350-361.

13. Gomes, Manuel, Richard Grieve, Richard Nixon, Edmond S‐W. Ng, James Carpenter, and Simon G. Thompson. “Methods For Covariate Adjustment In Cost‐Effectiveness Analysis That Use Cluster Randomised Trials.”Health economics21, no. 9 (2012): 1101-1118.

14. Ng, Edmond SW, Karla Diaz-Ordaz, Richard Grieve, Richard M. Nixon, Simon G. Thompson, and James R. Carpenter. “Multilevel models for cost-effectiveness analyses that use cluster randomised trial data: an approach to model choice.”Statistical methods in medical research(2013): 0962280213511719.