Quality (philosophy)

Control (management)

Philosophy

Biology

Statistics

Social psychology

Replication (statistics)

Computer security

Computer science

Status quo

Paleontology

Test (biology)

Hierarchy

Psychology

Mathematics

Superordinate goals

Statistical hypothesis testing

External validity

Applied psychology

Political science

Epistemology

Artificial intelligence

Scrutiny

Personality

Key (lock)

Perspectives on Psychological Science

The current debate about how to improve the quality of psychological science revolves, almost exclusively, around the subordinate level of statistical significance testing. In contrast, research design and strict theorizing, which are superordinate to statistics in the methods hierarchy, are sorely neglected. The present article is devoted to the key role assigned to manipulation checks (MCs) for scientific quality control. MCs not only afford a critical test of the premises of hypothesis testing but also (a) prompt clever research design and validity control, (b) carry over to refined theorizing, and (c) have important implications for other facets of methodology, such as replication science. On the basis of an analysis of the reality of MCs reported in current issues of the Journal of Personality and Social Psychology, we propose a future methodology for the post– p &lt; .05 era that replaces scrutiny in significance testing with refined validity control and diagnostic research designs.

Quo Vadis, Methodology? The Key Role of Manipulation Checks for Validity Control and Quality of Science