Cognitive psychology

Neuroscience

Statistics

Social psychology

Replication (statistics)

Interpretation (philosophy)

Computer science

Structural equation modeling

Extrasensory perception

Alternative medicine

Confirmatory factor analysis

Database

Psychology

Mathematics

Statistical hypothesis testing

Commit

Data science

Medicine

Programming language

Perception

Confirmation bias

Machine learning

Replicate

Pathology

Perspectives on Psychological Science

The veracity of substantive research claims hinges on the way experimental data are collected and analyzed. In this article, we discuss an uncomfortable fact that threatens the core of psychology's academic enterprise: almost without exception, psychologists do not commit themselves to a method of data analysis before they see the actual data. It then becomes tempting to fine tune the analysis to the data in order to obtain a desired result-a procedure that invalidates the interpretation of the common statistical tests. The extent of the fine tuning varies widely across experiments and experimenters but is almost impossible for reviewers and readers to gauge. To remedy the situation, we propose that researchers preregister their studies and indicate in advance the analyses they intend to conduct. Only these analyses deserve the label "confirmatory," and only for these analyses are the common statistical tests valid. Other analyses can be carried out but these should be labeled "exploratory." We illustrate our proposal with a confirmatory replication attempt of a study on extrasensory perception.

An Agenda for Purely Confirmatory Research