Markov chain Monte Carlo

Predictive inference

World Wide Web

Engineering

Bayesian probability

Aerospace engineering

Statistical inference

Statistics

Materials science

Bayes' theorem

Point estimation

Frequentist inference

Range (aeronautics)

Data mining

Computer science

Fiducial inference

Mathematics

Inference

Interval estimation

Artificial intelligence

Composite material

Confidence interval

Lasso (programming language)

Model selection

Prediction interval

Statistical model

Machine learning

Bayesian inference

Proceedings of the National Academy of Sciences

Probability models are used for many statistical tasks, notably parameter estimation, interval estimation, inference about model parameters, point prediction, and interval prediction. Thus, choosing a statistical model and accounting for uncertainty about this choice are important parts of the scientific process. Here we focus on one such choice, that of variables to include in a linear regression model. Many methods have been proposed, including Bayesian and penalized likelihood methods, and it is unclear which one to use. We compared 21 of the most popular methods by carrying out an extensive set of simulation studies based closely on real datasets that span a range of situations encountered in practical data analysis. Three adaptive Bayesian model averaging (BMA) methods performed best across all statistical tasks. These used adaptive versions of Zellner’s g -prior for the parameters, where the prior variance parameter g is a function of sample size or is estimated from the data. We found that for BMA methods implemented with Markov chain Monte Carlo, 10,000 iterations were enough. Computationally, we found two of the three best methods (BMA with <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>g</mml:mi> <mml:mo>=</mml:mo> <mml:msqrt> <mml:mi>n</mml:mi> </mml:msqrt> </mml:mrow> </mml:math> and empirical Bayes-local) to be competitive with the least absolute shrinkage and selection operator (LASSO), which is often preferred as a variable selection technique because of its computational efficiency. BMA performed better than Bayesian model selection (in which just one model is selected).

Comparing methods for statistical inference with model uncertainty