Physics

Permutation (music)

Mathematics

Type I and type II errors

Data mining

Statistics

Epistemology

Exploratory data analysis

Philosophy

Univariate analysis

Artificial intelligence

Psychology

Multivariate statistics

Acoustics

Univariate

Multivariate analysis

Computer science

Flexibility (engineering)

A priori and a posteriori

Statistical power

Psychophysiology

ERP studies produce large spatiotemporal data sets. These rich data sets are key to enabling us to understand cognitive and neural processes. However, they also present a massive multiple comparisons problem, potentially leading to a large number of studies with false positive effects (a high Type I error rate). Standard approaches to ERP statistical analysis, which average over time windows and regions of interest, do not always control for Type I error, and their inflexibility can lead to low power to detect true effects. Mass univariate approaches offer an alternative analytic method. However, they have thus far been viewed as appropriate primarily for exploratory statistical analysis and only applicable to simple designs. Here, we present new simulation studies showing that permutation‐based mass univariate tests can be employed with complex factorial designs. Most importantly, we show that mass univariate approaches provide slightly greater power than traditional spatiotemporal averaging approaches when strong a priori time windows and spatial regions are used. Moreover, their power decreases only modestly when more exploratory spatiotemporal parameters are used. We argue that mass univariate approaches are preferable to traditional spatiotemporal averaging analysis approaches for many ERP studies.

Having your cake and eating it too: Flexibility and power with mass univariate statistics for ERP data