Background: Comparative effectiveness research in non-randomized studies is often prone to various sources of confounding. Recently, prognostic score analysis has been proposed to address this issue, which aims to achieve prognostic balance across the different treatment groups. Although it is common to use the non-randomized data at hand to develop the necessary prognostic scores, this strategy is problematic when sample sizes are relatively small. It has previously been demonstrated that prognostic scores from historical cohorts may actually outperform internally developed prognostic scores for causal inference, and that their accuracy can further be improved through evidence synthesis.

Objectives: To present new meta-analysis methods for causal inference in non-randomized data sources. Hereto, we consider the aggregation of multiple prognostic scores derived from historical cohorts.

Methods: We extend existing methodology for causal inference and meta-analysis of prediction models, and propose new methods to derive comparative treatment effects from non-randomized studies. We conducted an extensive simulation study based on a real clinical dataset comparing different treatment strategies for asthma control. We aggregated previously identified prognostic scores for predicting exacerbations of asthma, and used the resulting model to estimate the average treatment effect in the overall (ATE) and in the treated (ATT) population of various simulated datasets. We compared various implementation strategies by assessing the bias and mean squared error of the estimated ATE and ATT, and the ratio of the estimated standard errors to the empirical standard deviations.

Conclusions: Initial simulation study results suggest that aggregation of historical prognostic scores may substantially improve the estimation of comparative treatment effects in non-randomized data sources.

Patient or healthcare consumer involvement: A clinician was involved in the provision of relevant patient-level data, and the interpretation of comparative treatment effect estimates.