Predicting future outcomes of patients is essential to clinical practice, with many prediction models published each year. Empirical evidence suggests that published studies often have severe methodological limitations, which undermine their usefulness. This article presents a step-by-step guide to help researchers develop and evaluate a clinical prediction model. The guide covers best practices in defining the aim and users, selecting data sources, addressing missing data, exploring alternative modelling options, and assessing model performance. The steps are illustrated using an example from relapsing-remitting multiple sclerosis. Comprehensive R code is also provided.

Background: Heart failure (HF) is a chronic and common condition with a rising prevalence, especially in the elderly. Morbidity and mortality rates in people with HF are similar to those with common forms of cancer. Clinical guidelines highlight the need for more detailed prognostic information to optimise treatment and care planning for people with HF. Besides proven prognostic biomarkers and numerous newly developed prognostic models for HF clinical outcomes, no risk stratification models have been adequately established. Through a number of linked systematic reviews, we aim to assess the quality of the existing models with biomarkers in HF and summarise the evidence they present.

Methods: We will search MEDLINE, EMBASE, Web of Science Core Collection, and the prognostic studies database maintained by the Cochrane Prognosis Methods Group combining sensitive published search filters, with no language restriction, from 1990 onwards. Independent pairs of reviewers will screen and extract data. Eligible studies will be those developing, validating, or updating any prognostic model with biomarkers for clinical outcomes in adults with any type of HF. Data will be extracted using a piloted form that combines published good practice guidelines for critical appraisal, data extraction, and risk of bias assessment of prediction modelling studies. Missing information on predictive performance measures will be sought by contacting authors or estimated from available information when possible. If sufficient high quality and homogeneous data are available, we will meta-analyse the predictive performance of identified models. Sources of between-study heterogeneity will be explored through meta-regression using pre-defined study-level covariates. Results will be reported narratively if study quality is deemed to be low or if the between-study heterogeneity is high. Sensitivity analyses for risk of bias impact will be performed.

Discussion: This project aims to appraise and summarise the methodological conduct and predictive performance of existing clinically homogeneous HF prognostic models in separate systematic reviews.Registration: PROSPERO registration number CRD42019086990.

Background: There is an unmet need for precise methods estimating disease prognosis in multiple sclerosis (MS).

Objective: Using advanced statistical modeling, we assessed the prognostic value of various clinical measures for disability progression.

Methods: Advanced models to assess baseline prognostic factors for disability progression over 2 years were applied to a pooled sample of patients from placebo arms in four different phase III clinical trials. Least absolute shrinkage and selection operator (LASSO) and ridge regression, elastic nets, support vector machines, and unconditional and conditional random forests were applied to model time to clinical disability progression confirmed at 24 weeks. Sensitivity analyses for different definitions of a combined endpoint were carried out, and bootstrap was used to assess prediction model performance.

Results: A total of 1582 patients were included, of which 434 (27.4%) had disability progression in a combined endpoint over 2 years. Overall model discrimination performance was relatively poor (all C-indices ≤ 0.65) across all models and across different definitions of progression.

Conclusion: Inconsistency of prognostic factor importance ranking confirmed the relatively poor prediction ability of baseline factors in modeling disease progression in MS. Our findings underline the importance to explore alternative predictors as well as alternative definitions of commonly used endpoints.

Over the past few years, evidence synthesis has become essential to investigate and improve the generalizability of medical research findings. This strategy often involves a meta-analysis to formally summarize quantities of interest, such as relative treatment effect estimates. The use of meta-analysis methods is, however, less straightforward in prognosis research because substantial variation exists in research objectives, analysis methods and the level of reported evidence.

We present a gentle overview of statistical methods that can be used to summarize data of prognostic factor and prognostic model studies. We discuss how aggregate data, individual participant data, or a combination thereof can be combined through meta-analysis methods. Recent examples are provided throughout to illustrate the various methods.

It is widely recommended that any developed - diagnostic or prognostic - prediction model is externally validated in terms of its predictive performance measured by calibration and discrimination. When multiple validations have been performed, a systematic review followed by a formal meta-analysis helps to summarize overall performance across multiple settings, and reveals under which circumstances the model performs suboptimal (alternative poorer) and may need adjustment. We discuss how to undertake meta-analysis of the performance of prediction models with either a binary or a time-to-event outcome. We address how to deal with incomplete availability of study-specific results (performance estimates and their precision), and how to produce summary estimates of the c-statistic, the observed:expected ratio and the calibration slope. Furthermore, we discuss the implementation of frequentist and Bayesian meta-analysis methods, and propose novel empirically-based prior distributions to improve estimation of between-study heterogeneity in small samples. Finally, we illustrate all methods using two examples: meta-analysis of the predictive performance of EuroSCORE II and of the Framingham Risk Score. All examples and meta-analysis models have been implemented in our newly developed R package "metamisc".

OBJECTIVES: To compare different methods to handle treatment when developing a prognostic model that aims to produce accurate probabilities of the outcome of individuals if left untreated.

STUDY DESIGN AND SETTING: Simulations were performed based on two normally distributed predictors, a binary outcome, and a binary treatment, mimicking a randomized trial or an observational study. Comparison was made between simply ignoring treatment (SIT), restricting the analytical data set to untreated individuals (AUT), inverse probability weighting (IPW), and explicit modeling of treatment (MT). Methods were compared in terms of predictive performance of the model and the proportion of incorrect treatment decisions.

RESULTS: Omitting a genuine predictor of the outcome from the prognostic model decreased model performance, in both an observational study and a randomized trial. In randomized trials, the proportion of incorrect treatment decisions was smaller when applying AUT or MT, compared to SIT and IPW. In observational studies, MT was superior to all other methods regarding the proportion of incorrect treatment decisions.

CONCLUSION: If a prognostic model aims to produce correct probabilities of the outcome in the absence of treatment, ignoring treatments that affect that outcome can lead to suboptimal model performance and incorrect treatment decisions. Explicitly, modeling treatment is recommended.