Many Labs 5: Testing pre-data collection peer review as an intervention to increase replicability

Published on Nov 13, 2020
· DOI :10.1177/2515245920958687
Charles R. Ebersole12
Estimated H-index: 12
,
Diane-Jo Bart-Plange2
Estimated H-index: 2
(UVA: University of Virginia)
+ 169 AuthorsBrian A. Nosek93
Estimated H-index: 93
(UVA: University of Virginia)
Sources
Abstract
Replication studies in psychological science sometimes fail to reproduce prior findings. If these studies use methods that are unfaithful to the original study or ineffective in eliciting the pheno...
📖 Papers frequently viewed together
3 Citations
106 Citations
References61
Newest
#1Florian Cova (University of Geneva)H-Index: 13
#2Brent Strickland ('ENS Paris': École Normale Supérieure)H-Index: 13
Last. Xiang Zhou (University of Pittsburgh)H-Index: 3
view all 42 authors...
Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project (XRP) to estimate the reproducibility of experimental philosophy (osf.io/dvkpr). Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We ...
23 CitationsSource
#1Hugh RabagliatiH-Index: 16
#2Martin CorleyH-Index: 23
Last. Ailsa E. MillenH-Index: 9
view all 8 authors...
Crosby, Monin, and Richardson (2008) found that hearing an offensive remark caused subjects (N = 25) to look longer at a potentially offended person, but only if that person could hear the remark. ...
1 CitationsSource
#1Lauren Skorb (BC: Boston College)H-Index: 2
#2Balazs Aczel (ELTE: Eötvös Loránd University)H-Index: 14
Last. Dylan Manfredi (UPenn: University of Pennsylvania)H-Index: 4
view all 29 authors...
As part of the Many Labs 5 project, we ran a replication of van Dijk, van Kleef, Steinel, and van Beest’s (2008) study examining the effect of emotions in negotiations. They reported that when the ...
1 CitationsSource
#1Charles R. Ebersole (UVA: University of Virginia)H-Index: 12
#2Luca Andrighetto (UniGe: University of Genoa)H-Index: 15
Last. Jon Grahe (PLU: Pacific Lutheran University)H-Index: 13
view all 18 authors...
To rule out an alternative to their structural-fit hypothesis, Payne, Burkley, and Stokes (2008) demonstrated that correlations between implicit and explicit race attitudes were weaker when partici...
2 CitationsSource
#1Christopher R. Chartier (Ashland University)H-Index: 11
#2Jack Arnal (McDaniel College)H-Index: 8
Last. Steffen R. Giessner (EUR: Erasmus University Rotterdam)H-Index: 22
view all 20 authors...
In Experiment 5 of Albarracin et al. (2008), participants primed with words associated with action performed better on a subsequent cognitive task than did participants primed with words associated...
2 CitationsSource
#1Katherine S. Corker (GV: Grand Valley State University)H-Index: 9
#2Jack Arnal (McDaniel College)H-Index: 8
Last. Bradford J. Wiggins (Brigham Young University–Idaho)H-Index: 7
view all 10 authors...
Albarracin et al. (2008, Experiment 7) tested whether priming action or inaction goals (vs. no goal) and then satisfying those goals (vs. not satisfying them) would be associated with subsequent co...
1 CitationsSource
#1Erica Baranski (UH: University of Houston)H-Index: 8
#2Ernest Baskin (""St. Joe's"": Saint Joseph's University)H-Index: 11
Last. Danka Purić (University of Belgrade)H-Index: 8
view all 14 authors...
Shnabel and Nadler (2008) assessed a needs-based model of reconciliation suggesting that in conflicts, victims and perpetrators have different psychological needs that when satisfied increase the c...
1 CitationsSource
#1Nick Buttrick (UVA: University of Virginia)H-Index: 8
#2Balazs Aczel (ELTE: Eötvös Loránd University)H-Index: 14
Last. Peter Szecsi (ELTE: Eötvös Loránd University)H-Index: 5
view all 16 authors...
Does convincing people that free will is an illusion reduce their sense of personal responsibility? Vohs and Schooler (2008) found that participants reading from a passage “debunking” free will che...
2 CitationsSource
#1Hans IJzerman (Institut Universitaire de France)H-Index: 23
#2Ivan Ropovik (Charles University in Prague)H-Index: 6
Last. Madhavi Menon (NSU: Nova Southeastern University)H-Index: 5
view all 26 authors...
In a test of their global-/local-processing-style model, Forster, Liberman, and Kuschel (2008) found that people assimilate a primed concept (e.g., “aggressive”) into their social judgments after a...
1 CitationsSource
#1Maya B. Mathur (Stanford University)H-Index: 18
#2Diane-Jo Bart-Plange (UVA: University of Virginia)H-Index: 2
Last. Alan Jern (RHIT: Rose-Hulman Institute of Technology)H-Index: 10
view all 41 authors...
Risen and Gilovich (2008) found that subjects believed that “tempting fate” would be punished with ironic bad outcomes (a main effect), and that this effect was magnified when subjects were under c...
1 CitationsSource
Cited By11
Newest
#1Felix Schafmeister (HHS: Stockholm School of Economics)H-Index: 1
Replication of existing research is often referred to as one of the cornerstones of modern science. In this study, I tested whether the publication of independent replication attempts affects the c...
1 CitationsSource
#1Jieying Chen (UM: University of Manitoba)H-Index: 1
#2Lok Ching Kwan (HKU: University of Hong Kong)
Last. Gilad Feldman (HKU: University of Hong Kong)H-Index: 9
view all 9 authors...
Abstract null null Hindsight bias refers to the tendency to perceive an event outcome as more probable after being informed of that outcome. We conducted very close replications of two classic experiments of hindsight bias and a conceptual replication testing hindsight bias regarding the perceived replicability of hindsight bias. In Study 1 (N = 890), we replicated Experiment 2 in Fischhoff (1975), and found support for hindsight bias in retrospective judgments (dmean = 0.60). In Study 2 (N = 60...
1 CitationsSource
#1Lynda G. Boothroyd (Durham University)H-Index: 21
#2Martin J. Tovée (Northumbria University)H-Index: 62
Last. Elizabeth H. Evans (Newcastle University)H-Index: 12
view all 3 authors...
Abstract Ultra-thin fashion dolls may represent a risk factor for thin-ideal internalisation and body dissatisfaction amongst young girls. We asked thirty one 5- to 9-year-old girls to engage in interactive play with commercially available dolls which were either ultra-thin (Barbie and Monster High) or represented a putative realistic childlike shape (Lottie and Dora) and to indicate their perceived own-body size and ideal body size on an interactive computer task both before and after play. The...
Source
An increased focus on transparency and replication in science has stimulated reform in research practices and dissemination. As a result, the research culture is changing: the use of preregistratio...
3 CitationsSource
#1Maya B. Mathur (Stanford University)H-Index: 18
#2Tyler J. VanderWeele (Harvard University)H-Index: 82
Selective publication and reporting in individual papers compromise the scientific record, but are meta-analyses as compromised as their constituent studies? We systematically sampled 63 meta-analyses (each comprising at least 40 studies) in PLoS One, top medical journals, top psychology journals, and Metalab, an online, open-data database of developmental psychology meta-analyses. We empirically estimated publication bias in each, including only the peer-reviewed studies. Across all meta-analys...
1 CitationsSource
#1Shawn Loewen (MSU: Michigan State University)H-Index: 35
#2Bronson Hui (MSU: Michigan State University)H-Index: 2
1 CitationsSource
#1Roy Groncki (Swinburne University of Technology)
#2Jennifer L. Beaudry (Swinburne University of Technology)H-Index: 9
Last. James D. Sauer (UTAS: University of Tasmania)H-Index: 14
view all 3 authors...
The way in which individuals think about their own cognitive processes plays an important role in various domains. When eyewitnesses assess their confidence in identification decisions, they could be influenced by how easily relevant information comes to mind. This ease-of-retrieval effect has a robust influence on people's cognitions in a variety of contexts (e.g., attitudes), but it has not yet been applied to eyewitness decisions. In three studies, we explored whether the ease with which eyew...
Source
The fact that children are far more likely to successfully acquire a variety of new skills and knowledge than are adults is so clearly evidenced in everyday life that it hardly needs scientific confirmation. However, despite four decades of intensive research, the reason why remains controversial. In fact, the terms of the debate about critical periods in language have hardly changed since the 1960s. I argue that this is because the standard in-lab research paradigms that have otherwise served p...
1 CitationsSource
#1Charles R. Ebersole (UVA: University of Virginia)H-Index: 12
#2Luca Andrighetto (UniGe: University of Genoa)H-Index: 15
Last. Jon Grahe (PLU: Pacific Lutheran University)H-Index: 13
view all 18 authors...
To rule out an alternative to their structural-fit hypothesis, Payne, Burkley, and Stokes (2008) demonstrated that correlations between implicit and explicit race attitudes were weaker when partici...
2 CitationsSource
#1Yoav ZeeviH-Index: 3
#2Sofi AstashenkoH-Index: 1
Last. Yoav BenjaminiH-Index: 54
view all 3 authors...
It is a central dogma in science that a result of a study should be replicable. Only 90 of the 190 replications attempts were successful. We attribute a substantial part of the problem to selective inference evident in the paper, which is the practice of selecting some of the results from the many. 100 papers in the Reproducibility Project in Psychology were analyzed. It was evident that the reporting of many results is common (77.7 per paper on average). It was further found that the selection ...