Estimating and Correcting the Effects of Model Selection Uncertainty

Most applied statistical analyses are carried out under model uncertainty, meaning that the model which generated the observations is unknown, and so the data are first used to select one of a set of plausible models by means of some selection criterion. Generally the data are then used to make inferences about some quantity of interest, ignoring model selection uncertainty, i.e. the fact that the selection step was carried out using the same data, and despite the known fact that this leads to invalid inferences. This thesis investigates several issues relating to this problem from both the Bayesian and the frequentist points of view, and offers new suggestions for dealing with it. We examine Bayesian model averaging (BMA) and point out that its frequentist performance is not always well-defined because, in some cases, it is unclear whether BMA methodology is truly Bayesian. We illustrate the point with a "fully Bayesian model averaging" that is applicable when the quantity of interest is parametric.