Extracting the Wisdom of Crowds When Information Is Shared

Published on Feb 21, 2019in Management Science3.935
· DOI :10.1287/MNSC.2018.3047
Asa Palley4
Estimated H-index: 4
(IU: Indiana University),
Jack B. Soll17
Estimated H-index: 17
(Duke University)
Using the wisdom of crowds—combining many individual judgments to obtain an aggregate estimate—can be an effective technique for improving judgment accuracy. In practice, however, accuracy is limit...
📖 Papers frequently viewed together
120 Citations
Cited By15
#1Saul Estrin (Centre for Economic Performance)H-Index: 72
#2Susanna Khavul (SPbU: Saint Petersburg State University)H-Index: 1
Last. Mike Wright (Imperial College London)H-Index: 142
view all 3 authors...
#1Patrick Afflerbach (University of Augsburg)H-Index: 6
#2Christopher van Dun (University of Bayreuth)
Last. Johannes Seyfried (University of Augsburg)H-Index: 4
view all 5 authors...
Research has shown that aggregation of independent expert judgments significantly improves the quality of forecasts as compared to individual expert forecasts. This “wisdom of crowds” (WOC) has sparked substantial interest. However, previous studies on strengths and weaknesses of aggregation algorithms have been restricted by limited empirical data and analytical complexity. Based on a comprehensive analysis of existing knowledge on WOC and aggregation algorithms, this paper describes the design...
1 CitationsSource
Abstract null null Decision-makers often collect and aggregate experts’ point predictions about continuous outcomes, such as stock returns or product sales. In this article, we model experts as Bayesian agents and show that means, including the (weighted) arithmetic mean, trimmed means, median, geometric mean, and essentially all other measures of central tendency, do not use all information in the predictions. Intuitively, they assume idiosyncratic differences to arise from error instead of pri...
#1Joshua Becker (UCL: University College London)H-Index: 6
#2Douglas GuilbeaultH-Index: 7
Last. Ned Smith
view all 3 authors...
Decades of research suggest that information exchange in groups and organizations can reliably improve judgment accuracy in tasks such as financial forecasting, market research, and medical decision-making. However, we show that improving the accuracy of numeric estimates does not necessarily improve the accuracy of decisions. For binary choice judgments, also known as classification tasks--e.g. yes/no or build/buy decisions--social influence is most likely to grow the majority vote share, regar...
#1Tom Wilkening (University of Melbourne)H-Index: 8
#2Marcellin Martinie (University of Melbourne)H-Index: 1
Last. Piers D. L. Howe (University of Melbourne)H-Index: 15
view all 3 authors...
Modern forecasting algorithms use the wisdom of crowds to produce forecasts better than those of the best identifiable expert. However, these algorithms may be inaccurate when crowds are systematic...
#1Ville Satopää (Ad: INSEAD)H-Index: 7
#2Marat Salikhov (Yale University)H-Index: 1
Last. Barb Mellers (UPenn: University of Pennsylvania)H-Index: 1
view all 4 authors...
Aggregating predictions from multiple judges often yields more accurate predictions than relying on a single judge: the “wisdom-of-the-crowd” effect. This aggregation can be conducted by different methods, from simple averaging to complex techniques, like Bayesian estimators and prediction markets. This article applies a broad set of aggregation methods to subjective probability estimates from a series of geopolitical forecasting tournaments. It then uses the Bias-Information-Noise (BIN) model t...
#1Rouba Ibrahim (UCL: University College London)H-Index: 12
#2Song-Hee Kim (SC: University of Southern California)H-Index: 10
Last. Jordan D. Tong (UW: University of Wisconsin-Madison)H-Index: 6
view all 3 authors...
Even when human point forecasts are less accurate than data-based algorithm predictions, they can still help boost performance by being used as algorithm inputs. Assuming one uses human judgment in...
1 CitationsSource
#1Rasheed Olawale Azeez (LASU: Lagos State University)H-Index: 1
#1Steffen Keck (University of Vienna)H-Index: 6
#2Wenjie Tang (NUS: National University of Singapore)H-Index: 5
Drawing on dual-process theory, we suggest that the benefits that arise from combining several quantitative individual judgments will be heightened when these judgments are based on different cogni...
Research on belief formation has produced contradictory findings on whether and when communication between group members will improve the accuracy of numeric estimates such as economic forecasts, medical diagnoses, and job candidate assessments. While some evidence suggests that carefully mediated processes such as the "Delphi method" produce more accurate beliefs than unstructured discussion, others argue that unstructured discussion outperforms mediated processes. Still others argue that indep...