The New Statistics Why and How

Published on Jan 1, 2014in Psychological Science7.029
· DOI :10.1177/0956797613504966
Geoff Cumming34
Estimated H-index: 34
 (La Trobe University)
Sources
Abstract
We need to make substantial changes to how we conduct research. First, in response to heightened concern that our published research literature is incomplete and untrustworthy, we need new requirements to ensure research integrity. These include prespecification of studies whenever possible, avoidance of selection and other inappropriate data- analytic practices, complete reporting, and encouragement of replication. Second, in response to renewed recognition of the severe flaws of null-hypothesis significance testing (NHST), we need to shift from reliance on NHST to estimation and other preferred techniques. The new statistics refers to recommended practices, including estimation based on effect sizes, confidence intervals, and meta-analysis. The techniques are not new, but adopting them widely would be new for many researchers, as well as highly beneficial. This article explains why the new statistics are important and offers guidance for their use. It describes an eight-step new-statistics strategy for research with integrity, which starts with formulation of research questions in estimation terms, has no place for NHST, and is aimed at building a cumulative quantitative discipline.
Download
📖 Papers frequently viewed together
201547.73Science
270 Authors (Christopher J. Anderson)
2011
1 Author (Geoff Cumming)
20095.54Psychonomic Bulletin & Review
5 Authors (Jeffrey N. Rouder, ..., Geoffrey J. Iverson)
References77
Newest
#1Lisa L. HarlowH-Index: 35
Source
#1Lisa L. HarlowH-Index: 35
Contents: Preface. Part I: Overview. L.L. Harlow, Significance Testing Introduction and Overview. Part II: The Debate: Against and For Significance Testing. J.Cohen, The Earth Is Round. F.L. Schmidt, J. Hunter, Eight Objections to the Discontinuation of Significance Testing in the Analysis of Research Data. S.A. Mulaik, N.S. Raju, R. Harshman, There Is a Time and Place for Significance Testing. R.P. Abelson, A Retrospective on the Significance Test Ban of 1999 (If There Were No Significance Test...
Source
Bootstrap Effect Sizes (bootES; Gerlanc & Kirby, 2012) is a free, open-source software package for R (R Development Core Team, 2012), which is a language and environment for statistical computing. BootES computes both unstandardized and standardized effect sizes (such as Cohen’s d, Hedges’s g, and Pearson’s r) and makes easily available for the first time the computation of their bootstrap confidence intervals (CIs). In this article, we illustrate how to use bootES to find effect sizes for contr...
Source
#1Rex B. Kline (Concordia University)H-Index: 25
Traditional education in statistics that emphasises significance testing leaves researchers and students ill prepared to understand what their results really mean. Specifically, most researchers and students who do not have strong quantitative backgrounds have difficulty understanding outcomes of statistical tests. As more and more people become aware of this problem, the emphasis on statistical significance in the reporting of results is declining. Increasingly, researchers are expected to desc...
Source
Nov 1, 2012·Perspectives on Psychological Science9.84
#1Eric-Jan Wagenmakers (UvA: University of Amsterdam)H-Index: 101
#2Ruud Wetzels (UvA: University of Amsterdam)H-Index: 17
Last. Rogier A. Kievit (UvA: University of Amsterdam)H-Index: 35
view all 5 authors...
The veracity of substantive research claims hinges on the way experimental data are collected and analyzed. In this article, we discuss an uncomfortable fact that threatens the core of psychology’s academic enterprise: almost without exception, psychologists do not commit themselves to a method of data analysis before they see the actual data. It then becomes tempting to fine tune the analysis to the data in order to obtain a desired result—a procedure that invalidates the interpretation of the ...
Source
Source
Nov 1, 2012·Perspectives on Psychological Science9.84
#1Klaus Fiedler (Heidelberg University)H-Index: 41
#2Florian Kutzner (Heidelberg University)H-Index: 9
Last. Joachim I. Krueger (Brown University)H-Index: 48
view all 3 authors...
Several influential publications have sensitized the community of behavioral scientists to the dangers of inflated effects and false-positive errors leading to the unwarranted publication of nonreplicable findings. This issue has been related to prominent cases of data fabrication and survey results pointing to bad practices in empirical science. Although we concur with the motives behind these critical arguments, we note that an isolated debate of false positives may itself be misleading and co...
Source
Nov 1, 2012·Perspectives on Psychological Science9.84
#1Harold Pashler (UCSD: University of California, San Diego)H-Index: 85
#2Eric-Jan Wagenmakers (UvA: University of Amsterdam)H-Index: 101
Source
#1Fiona Fidler (University of Melbourne)H-Index: 32
#2Geoff Cumming (La Trobe University)H-Index: 34
We describe a six-step estimation framework for research that starts with the formulation of research goals in terms of “How much?” questions. Such questions are best answered by effect size (ES) estimates and confidence intervals (CIs) calculated from data, where the ESs estimates and CIs are point and interval estimates of population parameters. These estimates usually provide the best basis for the interpretation of research findings. Such an estimation approach includes use of precision-meas...
Source
Sep 1, 2012·Australian Journal of Psychology2.32
#1Geoff Cumming (La Trobe University)H-Index: 34
#2Fiona Fidler (La Trobe University)H-Index: 32
Last. Jerry Lai (La Trobe University)H-Index: 5
view all 4 authors...
Estimation based on effect sizes, confidence intervals, and meta-analysis usually provides a more informative analysis of empirical results than does statistical significance testing, which has long been the conventional choice in psychology. The sixth edition of the American Psychological Association Publication Manual now recommends that psychologists should, wherever possible, use estimation and base their interpretation of research results on point and interval estimates. We outline the Manu...
Source
12345678
Cited By1748
Newest
Abstract null null Hypothesis testing is a central statistical method in the biomedical sciences. The ongoing debate about the concept of statistical significance and the reliability of null hypothesis significance tests (NHST) and p-values has brought the advent of various Bayesian hypothesis tests as possible alternatives, which often employ the Bayes factor. However, careful calibration of the prior parameters is necessary for the type I error rates or power of these alternatives to be any be...
Source
Jan 1, 2022·Journal of Experimental Social Psychology3.60
#1Dylan Campbell (SUNY: State University of New York System)H-Index: 2
#2Anna-Kaisa Reiman (SUNY: State University of New York System)
Abstract null null Do public perceptions of social psychological research align with this field's ostensible goal of describing and explaining real-world social behavior? There has been mounting concern about a disconnect between social psychological research and reality, which in turn raises concerns about social psychology's public image. But do non-experts agree with social psychologists that this field in its current state has little to say about real-world behavior? In a preregistered study...
Source
We present exploratory research of virtual reality techniques and mnemonic devices to assist in retrieving knowledge from scholarly articles. We used abstracts of scientific publications to represent scientific knowledge in scholarly articles; participants were asked to read, remember, and retrieve knowledge from a set of abstracts. We conducted an experiment to compare participants' recall and recognition performance in three different conditions: a control condition without a pre-specified str...
Source
#2Christian Bokhove (University of Southampton)H-Index: 10
Source
#1Kate A. M. Ford (RMIT: RMIT University)
#2Laurie McLay (RMIT: RMIT University)H-Index: 12
Last. Rosina Gibbs (RMIT: RMIT University)H-Index: 2
view all 4 authors...
Sleep problems are widely reported among children and adolescents on the autism spectrum. There is emerging evidence of the effectiveness and social validity of parent-implemented behavioral sleep interventions for children and adolescents on the autism spectrum. However, most research is focused on multi-component interventions that include the use of extinction, and questions remain about the effectiveness of modification to antecedent variables (including sleep/wake rescheduling) alone. This ...
Source
Nov 15, 2021·Topics in Cognitive Science3.38
#1V. N. Vimal Rao (UMN: University of Minnesota)
#2Jeffrey K. Bye (UMN: University of Minnesota)H-Index: 2
Last. Sashank Varma (Georgia Institute of Technology)H-Index: 17
view all 3 authors...
Traditional statistics instruction emphasizes a .05 significance level for hypothesis tests. Here, we investigate the consequences of this training for researchers' mental representations of probabilities - whether .05 becomes a boundary, that is, a discontinuity of the mental number line, and alters their reasoning about p-values. Graduate students with statistical training (n = 25) viewed pairs of p-values and judged whether they were "similar" or "different." After controlling for several cov...
Source
Nov 15, 2021·Development and Psychopathology4.15
#1Leigh G. Goetschius (UMBC: University of Maryland, Baltimore County)H-Index: 3
#2Vonnie C. McLoyd (UM: University of Michigan)H-Index: 50
Last. Christopher S. MonkH-Index: 49
view all 6 authors...
School connectedness, a construct indexing supportive school relationships, has been posited to promote resilience to environmental adversity. Consistent with prominent calls in the field, we examined the protective nature of school connectedness against two dimensions of early adversity that index multiple levels of environmental exposure (violence exposure, social deprivation) when predicting both positive and negative outcomes in longitudinal data from 3,246 youth in the Fragile Families and ...
Source
#1Jessica J LairdH-Index: 1
#2Bianca KlettkeH-Index: 12
Last. Ian FuelscherH-Index: 11
view all 4 authors...
Pressure to send sexually explicit messages, or ‘sexting coercion’ is associated with adverse mental health outcomes and sexual risk behaviors. This study explores Differentiation of Self (DoS) as a potential protective factor to reduce susceptibility to sexting coercion. A convenience sample of 399 Australian participants, aged 18 to 21 years (Mage = 19.63; SD = 1.14, 68.2% women) completed an online survey measuring sexting behaviors and DoS. Women were four times more likely to send willing u...
Source
Nov 4, 2021·Cognitive, Affective, & Behavioral Neuroscience3.28
#1Elisa Straub (University of Freiburg)H-Index: 1
#2Constantin Schmidts (University of Freiburg)H-Index: 3
Last. David Dignath (University of Freiburg)H-Index: 14
view all 6 authors...
Emotional information receives prioritized processing over concurrent cognitive processes. This can lead to distraction if emotional information has to be ignored. In the cognitive domain, mechanisms have been described that allow control of (cognitive) distractions. However, whether similar cognitive control mechanisms also can attenuate emotional distraction is an active area of research. This study asked whether cognitive control (triggered in the Color Stroop task) attenuates emotional distr...
Source
Source
12345678910
This website uses cookies.
We use cookies to improve your online experience. By continuing to use our website we assume you agree to the placement of these cookies.
To learn more, you can find in our Privacy Policy.