Bayesian probability

Internal medicine

Bayes factor

Sociology

Frequentist inference

Random effects model

Population

Mathematics

Frequentist probability

Statistical hypothesis testing

Meta-analysis

Statistics

Demography

Econometrics

Null hypothesis

Psychology

Bayesian inference

Bayes' theorem

False positive paradox

Medicine

Psychological Science

Mixed models are gaining popularity in psychology. For frequentist mixed models, previous research showed that excluding random slopes-differences between individuals in the direction and size of an effect-from a model when they are in the data can lead to a substantial increase in false-positive conclusions in null-hypothesis tests. Here, I demonstrated through five simulations that the same is true for Bayesian hypothesis testing with mixed models, which often yield Bayes factors reflecting very strong evidence for a mean effect on the population level even if there was no such effect. Including random slopes in the model largely eliminates the risk of strong false positives but reduces the chance of obtaining strong evidence for true effects. I recommend starting analysis by testing the support for random slopes in the data and removing them from the models only if there is clear evidence against them.

The Importance of Random Slopes in Mixed Models for Bayesian Hypothesis Testing