Workshop 2023
Workshop on the 23th and 24th of November at the Medical University of Vienna
We cordially invite you to the workshop Statistical Model Building Strategies for Cardiologic Applications (SAMBA) - New results and future challenges on November 23th and 24th in Vienna. The workshop is organized by the PIs of the international joint project SAMBA in cooperation with the Wiener Biometrische Sektion (WBS) and the Center for Medical Data Science (CeDAS) at the Medical University of Vienna. We are delighted to welcome Prof. Dr. Tobias Pischon from Berlin, Germany, and Prof. Dr. Maarten van Smeden from Utrecht, Netherlands, as keynote speakers. We acknowledge the support by the Austrian Science Funds (FWF) and the Deutsche Forschungsgemeinschaft (DFG) through the joint project SAMBA, grants I4739-B and RA-2347/8-1, respectively.
The workshop will cover two half days. On the afternoon of November 23th our focus will be on the intersection of epidemiology and biostatistics. Prof. Pischon will talk on the Interaction of epidemiology with biostatistics within the large prospective cohort study NAKO with emphasis on cardiovascular diseases followed by a panel discussion on Complex longitudinal studies: Needs and challenges in the interaction epidemiology – biostatistics. The morning of November 24th is dedicated to The power and risks of algorithmic model building, where Prof. van Smeden will give his talk on Rage against the machine learning. The participants of the SAMBA project will also present issues and opportunities of statistical model building strategies with a focus on cardiological applications.
The participation in the workshop is free of charge and open to any national and international participants of any scientific discipline interested in the interaction of biostatistics, epidemiology, machine learning and cardiology.
We are looking forward to welcoming you in Vienna in November!
Daniela Dunkler, Georg Heinze, Heiko Becher & Geraldine Rauch (organizing committee)
Registration
Registration has closed!
Programme
Thursday, 23.11.2023
Topic of the day Epidemiology and biostatistics: we need each other
Venue: Medical University Vienna, Hörsaalzentrum (building 4 on the attached plan), Hörsaal 3
| Start | End | Topic | Presenter |
| 13:00 | 13:05 | Introduction | Georg Heinze & Heiko Becher |
| 13:05 | 13:25 | Causal model building in the context of cardiac rehabilitation: A systematic review and a case study | Nilufar Akbari |
| 13:25 | 14:15 | Invited talk: The German National Cohort (NAKO): Opportunity, challenges, and interactions with biostatistics (see Section 4.0.1) | Tobias Pischon |
| 14:15 | 14:45 | Panel discussion: Complex longitudinal studies: Needs and challenges in the interaction between biostatistics and epidemiology | Panelists: Tobias Pischon, Heiko Becher, Ulrike Grittner, Dörte Huscher, Maarten van Smeden, Georg Heinze |
| 14:45 | 15:15 | BREAK | |
| 15:15 | 15:35 | Phase IV study of joint models for longitudinal and time to event data | Jil Kollmus-Heege |
| 15:35 | 15:55 | Using background knowledge from previous studies in model building: the good, the bad and the ugly | Lorena Hafermann |
| 15:55 | 16:15 | Correlated predictors: is variable omission a good default solution? | Mariella Gregorich |
| 16:15 | 16:20 | Closing | Heiko Becher |
Evening: Social programme
Friday, 24.11.2023
Topic of the day The power and risks of algorithmic model building
Venue: Medical University Vienna, Spitalgasse 23, Jugendstilhörsaal
| Start | End | Topic | Presenter |
| 9:00 | 9:10 | Introduction | Georg Heinze |
| 9:10 | 9:30 | Model building using statistical and machine learning approaches: Comparison of performance and explainability | Georg Heinze |
| 9:30 | 10:30 | Invited talk: Rage against the machine learning (see Section 4.0.2) | Maarten van Smeden |
| 10:30 | 11:00 | BREAK | |
| 11:00 | 11:20 | Variable selection methods for linear and logistic regression: A comparison of approaches | Theresa Ullmann |
| 11:20 | 11:40 | Reliable confidence intervals after variable selection: does it work in practice? | Michael Kammer |
| 11:40 | 12:00 | Variable selection: Improving on the bad habit to ignore selection decisions in model inference | Nilufar Akbari |
| 12:00 | 12:05 | Closing | Georg Heinze |
Abstracts of invited talks
The German National Cohort (NAKO): Opportunity, challenges, and interactions with biostatistics
Tobias Pischon
The German National Cohort (NAKO) is a large, multidisciplinary, population-based cohort study that aims to investigate the development and etiology of major chronic diseases, identify risk factors and enhance early detection and prevention of diseases. Between 2014 and 2019, a total of 205,217 persons aged 20-74 years was recruited and examined at 18 study centers across Germany. Whole-body 3T magnetic resonance imaging (MRI) readings were performed on 30,861 participants at 5 study centers. Participants are followed-up for incident diseases via questionnaires, health records and disease registries. In addition, a mortality follow-up has been established. In addition, all study participants are being invited for a first re-examination, similar to the baseline program, between 2019 and 2024, and a second re-examination between 2024 and 2028. Interested scientists can use the web-based NAKO TransferHub (transfer.nako.de) to obtain insight into data dictionary and study population, and to apply for data or biological samples, which are made available on the basis of a Use and Access Policy. In my presentation I will provide an overview of the German National Cohort and give examples of opportunity and challenges with a focus on interactions with biostatistics.
Rage against the machine learning
Maarten van Smeden
Medicine appears to be at the start of a new era due to the many promising developments in AI and machine learning (ML). Part of that promise is that ML will improve upon traditional statistical modelling. Another promise is that ML will outperform medical doctors. In this talk I will highlight a few medical settings where ML seems to have lived up to its promises, and some where it has not. A couple of challenges for fair comparisons between machine learning, traditional statistical modelling and doctors will be identified and discussed. Finally, I will speculate on the future role of machine learning, traditional statistical modelling and medical doctors.
Abstracts of contributed talks
Causal model building in the context of cardiac rehabilitation: A systematic review and a case study
Nilufar Akbari, Georg Heinze, Geraldine Rauch, Ben Sander, Heiko Becher & Daniela Dunkler
Randomization is an effective design option to prevent bias from confounding in the evaluation of the causal effect of interventions on outcomes. However, sometimes randomization is not possible, making subsequent adjustment for confounders essential to obtain valid results. Several methods exist to adjust for confounding, with multivariable modeling being among the most widely used. The main challenge is to determine which variables should be included in the causal model and to specify appropriate functional relations for continuous variables in the model. While the statistical literature gives a variety of recommendations on how to build multivariable regression models in practice, this guidance is often unknown to applied researchers. We set out to investigate the current practice of explanatory regression modeling to control confounding in the field of cardiac rehabilitation, for which mainly non-randomized observational studies are available. In particular, we conducted a systematic methods review to identify and compare statistical methodology with respect to statistical model building in the context of the existing recent systematic review CROS-II, which evaluated the prognostic effect of cardiac rehabilitation.
Phase IV study of joint models for longitudinal and time to event data
Jil Kollmus-Heege, Annette Aigner, Sonja Greven & Ulrike Grittner
In recent years, there has been an increasing use of joint models to describe the relationship between a longitudinally measured biomarker and the time to an event. The current literature suggests that joint models are preferable to conventional methods as they were shown to be unbiased in such settings. However, it is important to conduct further investigation into this relatively new and complex approach to fully understand its pitfalls and to identify what to watch out for when using it. Therefore, we compare joint models to the time-varying Cox model and the two-stage approach in multiple simulation studies. These include basic simulations as well as more complex ones based on actual observational data.
Using background knowledge from previous studies in model building: the good, the bad and the ugly
by Lorena Hafermann, Heiko Becher, Carolin Herrmann, Nadja Klein, Michael Kammer, Georg Heinze & Geraldine Rauch
The question of which variables to include in a statistical model is of key importance. Thereby, the integration of background knowledge into the modelling process is an issue that is gaining increasing interest. Often background knowledge is based on results from previous studies. Its quality can vary substantially, particular with regard to the methods used to select variables, raising concerns about its utility for model building in a new study. We found that in the context of descriptive and explanatory linear regression models, background knowledge from previous studies that employed inappropriate model building strategies such as univariable selection distorted the specification of an appropriate predictor set. We also investigated if prediction models estimated by Random Forests benefitted from the incorporation of appropriate background knowledge on the set of predictors. Surprisingly, there was no benefit in terms of discrimination, but the calibration of the model improved by well-informed restriction of the candidate predictor set. Therefore, we recommend to use background knowledge with care and to critically appraise the methodological quality of any previous studies from which it was derived.
Model building using statistical and machine learning approaches: Comparison of performance and explainability
Christine Schilhart-Wallisch, Asan Agibetov, Daniela Dunkler, Maria Haller, Matthias Samwald, Georg Dorffner & Georg Heinze
While boundaries between classical statistical model building (SMB) and machine learning (ML) are fluid, both modeling cultures have been used to develop cardiovascular prediction models based on typical predictors such as total cholesterol, blood pressure, sex or age. We reanalyzed data from a large registry of 1.5 million participants in a national health screening program. We developed analytical strategies based on SMB (including nonlinear functional forms and interactions) or on ML to predict cardiovascular events within 1 year from health screening. This was done for the full data set and with gradually reduced sample sizes. Predictive performance was similar if the models were derived on the full data set, whereas smaller sample sizes favored simpler models. Visualized predictor-risk relations differed between SMB and ML, even with large sample sizes. Hence, the supposed role of cardiovascular predictors in prognosis can depend on how a risk prediction model was developed.
Reliable confidence intervals after variable selection: does it work in practice?
by Michael Kammer, Daniela Dunkler, Stefan Michiels & Georg Heinze
Biomedical data analysis relies heavily on variable selection for regression models. However, classical statistical frequentist theory, which assumes a fixed set of model covariates, does not cover inference post-selection. We compared selective inference methods for Lasso and adaptive Lasso submodel parameters: sample splitting, selective inference conditional on Lasso selection (SI), and universally valid post-selection inference (PoSI). Using R software packages, we evaluated their selective confidence intervals through simulations mirroring typical biomedical data. We also applied these methods to a real dataset. SI method showed generally acceptable frequentist properties but failed to achieve claimed selective coverage levels, especially with adaptive Lasso. Sample splitting is an alternative if simplicity is favored over efficiency. PoSI was extremely conservative and is suitable only when few predictors undergo selection or avoiding false positive claims is crucial. In summary, selective inference aids in assessing uncertainties in the importance of selected predictors for future applications.
Variable selection methods for linear and logistic regression: A comparison of approaches
by Theresa Ullmann, Christine Schilhart-Wallisch, Lorena Hafermann, Georg Heinze & Daniela Dunkler
Data-driven variable selection is frequently performed in statistical modeling, i.e., when modeling the associations between an outcome and multiple independent variables. Variable selection may improve the interpretability, parsimony and/or predictive accuracy of a model. Yet variable selection can also have negative consequences, such as false exclusion of important variables or inclusion of noise variables, biased estimation of regression coefficients, underestimated standard errors and invalid confidence intervals, as well as model instability. Few large-scale simulation studies have neutrally compared data-driven variable selection methods with respect to their consequences for the resulting models. We present results from a simulation study that compares different variable selection methods: forward selection, (augmented) backward elimination, univariable selection, and penalized likelihood approaches (Lasso, relaxed Lasso, adaptive Lasso). In the interpretation of the results, we put a specific focus on which estimands and performance criteria are relevant for which modeling goals (descriptive, explanatory or predictive modeling).
Variable selection: Improving on the bad habit to ignore selection decisions in model inference
by Nilufar Akbari & Georg Heinze
For regression models, methods for estimating the variance for the regression coefficients a set of pre-specified predictors are well-established. However, when the set of predictors is based on data-driven variable selection, there is no widely accepted method to account for selection uncertainty. Since it must be assumed that not selecting a variable into a model is associated with uncertainty, variance estimates of the coefficients of the unselected variables are desirable. The idea of this study is to use a modified sandwich variance estimator for this task. Hereby, the robust variance (sandwich) formula is applied to all candidate variables, including variables that were not selected into the final model. We conducted a simulation study to investigate the method and compare its performance to two model-based variance methods and two bootstrap based methods.
Biosketch of invited speakers
Tobias Pischon
Prof. Dr. med. Tobias Pischon, MPH
Molecular Epidemiology Research Group
| 2010- | Head, Molecular Epidemiology Research Group, Max-Delbrück-Centrum für Molekulare Medizin (MDC) Berlin-Buch, Germany |
| 2010- | Full Professor in Molecular Epidemiology, Charité – Universitätsmedizin Berlin, Germany |
| 2018- | Member, Board of Directors, German National Cohort (NAKO Gesundheitsstudie), NAKO e.V., Heidelberg, Germany |
| 2015- | Head, Biobank Technology Platform, Max-Delbrück Centrum für Molekulare Medizin in der Helmholtz-Gemeinschaft (MDC), Berlin, Germany |
| 2015- | Member, Board of Directors, Biobank Core Facility, Berlin Institute of Health (BIH), Berlin, Germany | |
| 2011- | Member of the PIs, German National Cohort (NAKO Gesundheitsstudie), NAKO e.V., Heidelberg, Germany Education and Qualification | |
| 2007 | Venia docendi (Habilitation) and Venia legendi (Lehrbefugnis), Charité – Universitätsmedizin Berlin, Germany |
| 2007 | Venia docendi (Habilitation) and Venia legendi (Lehrbefugnis), Charité – Universitätsmedizin Berlin, Germany |
| 2005 | Certificate Clinical Nutrition, Med. Assoc. Berlin and Brandenburg, Germany |
| 2005 | Certificate Epidemiology, German Society for Epidemiology |
| 1999-2001 | Study of Public Health; Degree: MPH, Technische Universität Berlin, Germany |
| 1996-1999 | Dissertation; Degree: Dr. med., Freie Universität Berlin, Germany |
| 1991-1998 | Study of Medicine; Degree: MD, Freie Universität Berlin, Germany |
| 2008-2010 | Head, Biomarker Research Group, Dept. of Epidemiology, Deutsches Institut für Ernährungsforschung (DIfE) Potsdam-Rehbrücke, Germany |
| 2005-2007 | Senior Investigator (Co-Appointment), Institut für Sozialmedizin, Epidemiologie und Gesundheitsökonomie, Charité – Universitätsmedizin Berlin, Germany |
| 2004-2008 | Senior Investigator, Dept. of Epidemiology, Deutsches Institut für Ernährungsforschung (DIfE) Potsdam-Rehbrücke, Germany |
| 2001-2004 | Postdoc/Research Associate, Depts. of Epidemiology and Nutrition, Harvard School of Public Health, Harvard University, Boston, MA, USA |
| 1998-2001 | Medical Residency/Postdoc, Dept. of Nephrology and Hypertensiology, Charité, and Dept. of Endocrinology and Nephrology, Freie Universität Berlin, Germany |
Publications:
Maarten van Smeden
Maarten van Smeden is a medical statistician and associate professor at the Julius Center for Health Sciences and Primary Care, UMC Utrecht, the Netherlands. He is coordinating the methodology research program, member of the leadership team of the Julius Center and head of the biostatistics team at the Julius Center. His research is focused on the development and evaluation of statistical methodology, with a particular interest in methodology related to the development and validation of prediction models based on machine learning and artificial intelligence. He is coordinator of the AI methods lab in the UMC Utrecht.
Van Smeden’s work has resulted in publications in well-renowned journals in the field of medical statistics (e.g. Statistics in Medicine), epidemiology (e.g. American journal of Epidemiology), and general medicine (e.g. BMJ). As a collaborator, he has also contributed to implementing advanced methodology in a variety of disciplines (e.g. cardiology, infectious diseases, general practice). He is also associate editor at the European Heart Journal, Statistics in Medicine and Diagnostic and Prognostic Research (BMC journals) and has been involved in the development and validation of numerous diagnostic and prognostic prediction models. He is an experienced teacher and invited speaker on topics related to epidemiology, methodology and statistics.
Van Smeden is member and chair of the datasets and communication panels of the STRATOS initiative, a collaboration of methods experts worldwide who develop guidance on design and analysis of biomedical research. He is the first author on the Dutch guideline for high-quality diagnostic and prognostic applications of AI in healthcare, commissioned by the Dutch Ministry of Health, Welfare and Sport.
SAMBA participants & presenters
Akbari, Nilufar (Charité Universitätsmedizin Berlin)
Becher, Heiko (Universitätsklinikum Heidelberg )
Dunkler, Daniela (Medical University of Vienna)
Gregorich, Mariella (Medical University of Vienna)
Hafermann, Lorena (Charité Universitätsmedizin Berlin)
Heinze, Georg (Medical University of Vienna)
Herrmann, Carolin (Charité Universitätsmedizin Berlin)
Kammer, Michael (Medical University of Vienna)
Kollmus-Heege, Jil (Charité Universtitätsmedizin Berlin)
Rauch, Geraldine Technische Universität Berlin
Schilhart-Wallisch, Christine (Agentur für Gesundheit und Ernährungssicherheit, Wien)
Ullmann, Theresa (Medical University of Vienna)
Selected output of the SAMBA project
The project Statistical Model Building Strategies for Cardiologic Applications (SAMBA) is funded by the Austrian Science Fund (FWF), grant number I4739-B to Daniela Dunkler, and the Deutsche Forschungsgemeinschaft (DFG), grant number RA-2347/8-1 to Geraldine Rauch and Heiko Becher.
Wallisch C, Dunkler D, Rauch G, de Bin R, Heinze G: Selection of variables for multivariable models: Opportunities and limitations in quantifying model stability by resampling. Statistics in Medicine (2021) 40:369-381.
Abstract: Statistical models are often fitted to obtain a concise description of the associ- ation of an outcome variable with some covariates. Even if background knowledge is available to guide preselection of covariates, stepwise variable selection is commonly applied to remove irrelevant ones. This practice may introduce additional variability and selection is rarely certain. However, these issues are often ignored and model stability is not questioned. Several resampling-based measures were proposed to describe model stability,including variable inclusion frequencies (VIFs), model selection frequencies, relative conditional bias (RCB), and root mean squared difference ratio (RMSDR). The latter two were recently proposed to assess bias and variance inflation induced by variable selection. Here, we study the consistency and accuracy of resampling estimates of these measures and the optimal choice of the resampling technique. In particular, we compare subsampling and bootstrapping for assessing stability of linear, logistic, and Cox models obtained by backward elimination in a simulation study. Moreover, we exemplify the estimation and interpretation of all suggested measures in a study on cardiovascular risk. The VIF and the model selection frequency are only consistently estimated in the subsampling approach. By contrast, the bootstrap is advantageous in terms of bias and precision for estimating the RCB as well as the RMSDR. Though, unbiased estimation of the latter quantity requires independence of covariates, which is rarely encountered in practice. Our study stresses the importance of addressing model stability after variable selection and shows how to cope with it.Gregorich M, Strohmaier S, Dunkler D, Heinze G: Regression with Highly Correlated Predictors: Variable Omission Is Not the Solution. International Journal of Environmental Research and Public Health (2021) 18(8).
Abstract: Regression models have been in use for decades to explore and quantify the association between a dependent response and several independent variables in environmental sciences, epidemiology and public health. However, researchers often encounter situations in which some independent variables exhibit high bivariate correlation, or may even be collinear. Improper statistical handling of this situation will most certainly generate models of little or no practical use and misleading interpretations. By means of two example studies, we demonstrate how diagnostic tools for collinearity or near-collinearity may fail in guiding the analyst. Instead, the most appropriate way of handling collinearity should be driven by the research question at hand and, in particular, by the distinction between predictive or explanatory aims.Hafermann L, Becher H, Herrmann C, Klein N, Heinze G, Rauch G: Statistical Model Building: Background “Knowledge” Based on Inappropriate Preselection Causes Misspecification. BMC Med Res Methodol (2021) 21(1):196.
Abstract:
Background: Statistical model building requires selection of variables for a model depending on the model’s aim. In descriptive and explanatory models, a common recommendation often met in the literature is to include all variables in the model which are assumed or known to be associated with the outcome independent of their identification with data driven selection procedures. An open question is, how reliable this assumed “background knowledge” truly is. In fact, “known” predictors might be findings from preceding studies which may also have employed inappropriate model building strategies.
Methods: We conducted a simulation study assessing the influence of treating variables as “known predictors” in model building when in fact this knowledge resulting from preceding studies might be insufficient. Within randomly generated preceding study data sets, model building with variable selection was conducted. A variable was subsequently considered as a “known” predictor if a predefined number of preceding studies identified it as relevant.
Results: Even if several preceding studies identified a variable as a “true” predictor, this classification is often false positive. Moreover, variables not identified might still be truly predictive. This especially holds true if the preceding studies employed inappropriate selection methods such as univariable selection.
Conclusions: The source of “background knowledge” should be evaluated with care. Knowledge generated on preceding studies can cause misspecification.Wallisch C, Agibetov A, Dunkler D, Haller M, Samwald M, Dorffner G, Heinze G: The Roles of Predictors in Cardiovascular Risk Models - a Question of Modeling Culture? BMC Medical Research Methodology (2021) 21(1):284.
Abstract:
Background: While machine learning (ML) algorithms may predict cardiovascular outcomes more accurately than statistical models, their result is usually not representable by a transparent formula. Hence, it is often unclear how specific values of predictors lead to the predictions. We aimed to demonstrate with graphical tools how predictor-risk relations in cardiovascular risk prediction models fitted by ML algorithms and by statistical approaches may differ, and how sample size affects the stability of the estimated relations.
Methods: We reanalyzed data from a large registry of 1.5 million participants in a national health screening program. Three data analysts developed analytical strategies to predict cardiovascular events within 1 year from health screening. This was done for the full data set and with gradually reduced sample sizes, and each data analyst followed their favorite modeling approach. Predictor-risk relations were visualized by partial dependence and individual conditional expectation plots.
Results: When comparing the modeling algorithms, we found some similarities between these visualizations but also occasional divergence. The smaller the sample size, the more the predictor-risk relation depended on the modeling algorithm used, and also sampling variability played an increased role. Predictive performance was similar if the models were derived on the full data set, whereas smaller sample sizes favored simpler models.
Conclusion: Predictor-risk relations from ML models may differ from those obtained by statistical models, even with large sample sizes. Hence, predictors may assume different roles in risk prediction models. As long as sample size is sufficient, predictive accuracy is not largely affected by the choice of algorithm.Hafermann L, Klein N, Rauch G, Kammer M, Heinze G: Using Background Knowledge from Preceding Studies for Building a Random Forest Prediction Model: A Plasmode Simulation Study. Entropy (Basel) (2022) 24(6).
Abstract: There is an increasing interest in machine learning (ML) algorithms for predicting patient outcomes, as these methods are designed to automatically discover complex data patterns. For example, the random forest (RF) algorithm is designed to identify relevant predictor variables out of a large set of candidates. In addition, researchers may also use external information for variable selection to improve model interpretability and variable selection accuracy, thereby prediction quality. However, it is unclear to which extent, if at all, RF and ML methods may benefit from external information. In this paper, we examine the usefulness of external information from prior variable selection studies that used traditional statistical modeling approaches such as the Lasso, or suboptimal methods such as univariate selection. We conducted a plasmode simulation study based on subsampling a data set from a pharmacoepidemiologic study with nearly 200,000 individuals, two binary outcomes and 1152 candidate predictor (mainly sparse binary) variables. When the scope of candidate predictors was reduced based on external knowledge RF models achieved better calibration, that is, better agreement of predictions and observed outcome rates. However, prediction quality measured by cross-entropy, AUROC or the Brier score did not improve. We recommend appraising the methodological quality of studies that serve as an external information source for future prediction model development.Kammer M, Dunkler D, Michiels S, Heinze G: Evaluating Methods for Lasso Selective Inference in Biomedical Research: A Comparative Simulation Study. BMC Medical Research Methodology (2022) 22(1):206.
Abstract:
Background: Variable selection for regression models plays a key role in the analysis of biomedical data. However, inference after selection is not covered by classical statistical frequentist theory, which assumes a fixed set of covariates in the model. This leads to over-optimistic selection and replicability issues.
Methods: We compared proposals for selective inference targeting the submodel parameters of the Lasso and its extension, the adaptive Lasso: sample splitting, selective inference conditional on the Lasso selection (SI), and universally valid post-selection inference (PoSI). We studied the properties of the proposed selective confidence intervals available via R software packages using a neutral simulation study inspired by real data commonly seen in biomedical studies. Furthermore, we present an exemplary application of these methods to a publicly available dataset to discuss their practical usability.
Results: Frequentist properties of selective confidence intervals by the SI method were generally acceptable, but the claimed selective coverage levels were not attained in all scenarios, in particular with the adaptive Lasso. The actual coverage of the extremely conservative PoSI method exceeded the nominal levels, and this method also required the greatest computational effort. Sample splitting achieved acceptable actual selective coverage levels, but the method is inefficient and leads to less accurate point estimates. The choice of inference method had a large impact on the resulting interval estimates, thereby necessitating that the user is acutely aware of the goal of inference in order to interpret and communicate the results.
Conclusions: Despite violating nominal coverage levels in some scenarios, selective inference conditional on the Lasso selection is our recommended approach for most cases. If simplicity is strongly favoured over efficiency, then sample splitting is an alternative. If only few predictors undergo variable selection (i.e. up to 5) or the avoidance of false positive claims of significance is a concern, then the conservative approach of PoSI may be useful. For the adaptive Lasso, SI should be avoided and only PoSI and sample splitting are recommended. In summary, we find selective inference useful to assess the uncertainties in the importance of individual selected predictors for future applications.Akbari N, Heinze G, Rauch G, Sander B, Becher H, Dunkler D: Causal Model Building in the Context of Cardiac Rehabilitation: A Systematic Review. International Journal of Environmental Research and Public Health (2023) 20(4):3182.
Abstract: Randomization is an effective design option to prevent bias from confounding in the evaluation of the causal effect of interventions on outcomes. However, in some cases, randomization is not possible, making subsequent adjustment for confounders essential to obtain valid results. Several methods exist to adjust for confounding, with multivariable modeling being among the most widely used. The main challenge is to determine which variables should be included in the causal model and to specify appropriate functional relations for continuous variables in the model. While the statistical literature gives a variety of recommendations on how to build multivariable regression models in practice, this guidance is often unknown to applied researchers. We set out to investigate the current practice of explanatory regression modeling to control confounding in the field of cardiac rehabilitation, for which mainly non-randomized observational studies are available. In particular, we conducted a systematic methods review to identify and compare statistical methodology with respect to statistical model building in the context of the existing recent systematic review CROS-II, which evaluated the prognostic effect of cardiac rehabilitation. CROS-II identified 28 observational studies, which were published between 2004 and 2018. Our methods review revealed that 24 (86%) of the included studies used methods to adjust for confounding. Of these, 11 (46%) mentioned how the variables were selected and two studies (8%) considered functional forms for continuous variables. The use of background knowledge for variable selection was barely reported and data-driven variable selection methods were applied frequently. We conclude that in the majority of studies, the methods used to develop models to investigate the effect of cardiac rehabilitation on outcomes do not meet common criteria for appropriate statistical model building and that reporting often lacks precision.
Host
A joint seminar by the Medical University of Vienna and the Wiener Biometrische Sektion der Region Österreich-Schweiz (ROeS) der internationalen Biometrischen Gesellschaft
Funding
FWF Austrian Science Fund: 
DFG Deutsche Forschungsgemeinschaft: 