Human Decisions and Machine Predictions

Published on Feb 16, 2017in Quarterly Journal of Economics
· DOI :10.1093/QJE/QJX032
Jon Kleinberg116
Estimated H-index: 116
(Cornell University),
Himabindu Lakkaraju18
Estimated H-index: 18
(Stanford University)
+ 2 AuthorsSendhil Mullainathan86
Estimated H-index: 86
Presented on October 24, 2016 at 10:00 a.m. in the Klaus Advanced Computing Building, room 1116
Figures & Tables
📖 Papers frequently viewed together
2016KDD: Knowledge Discovery and Data Mining
2,424 Citations
467 Citations
259 Citations
#1Jiaming Zeng (MIT: Massachusetts Institute of Technology)H-Index: 3
#2Berk Ustun (MIT: Massachusetts Institute of Technology)H-Index: 13
Last. Cynthia Rudin (MIT: Massachusetts Institute of Technology)H-Index: 44
view all 3 authors...
Summary We investigate a long-debated question, which is how to create predictive models of recidivism that are sufficiently accurate, transparent and interpretable to use for decision making. This question is complicated as these models are used to support different decisions, from sentencing, to determining release on probation to allocating preventative social services. Each case might have an objective other than classification accuracy, such as a desired true positive rate TPR or false posi...
81 CitationsSource
#1Sendhil Mullainathan (Harvard University)H-Index: 86
#2Jann Spiess (Harvard University)H-Index: 3
Machines are increasingly doing "intelligent" things. Face recognition algorithms use a large dataset of photos labeled as having a face or not to estimate a function that predicts the presence y of a face from pixels x. This similarity to econometrics raises questions: How do these new empirical tools fit with what we know? As empirical economists, how can we use them? We present a way of thinking about machine learning that gives it its own place in the econometric toolbox. Machine learning no...
350 CitationsSource
#1Richard A. Berk (UPenn: University of Pennsylvania)H-Index: 73
Objectives: The Pennsylvania Board of Probation and Parole has begun using machine learning forecasts to help inform parole release decisions. In this paper, we evaluate the impact of the forecasts on those decisions and subsequent recidivism.
25 CitationsSource
Jan 1, 2017 in AISTATS (International Conference on Artificial Intelligence and Statistics)
#1Himabindu Lakkaraju (Stanford University)H-Index: 18
#2Cynthia Rudin (Duke University)H-Index: 44
21 Citations
#1Jason Abaluck (Yale University)H-Index: 9
#2Leila Agha (Dartmouth College)H-Index: 7
Last. Arjun K. Venkatesh (Yale University)H-Index: 24
view all 5 authors...
A large body of research has investigated whether physicians overuse care. There is less evidence on whether, for a fixed level of spending, doctors allocate resources to patients with the highest expected returns. We assess both sources of inefficiency, exploiting variation in rates of negative imaging tests for pulmonary embolism. We document enormous across-doctor heterogeneity in testing conditional on patient population, which explains the negative relationship between physicians' testing r...
15 CitationsSource
In this paper we propose methods for estimating heterogeneity in causal effects in experimental and observational studies and for conducting hypothesis tests about the magnitude of differences in treatment effects across subsets of the population. We provide a data-driven approach to partition the data into subpopulations that differ in the magnitude of their treatment effects. The approach enables the construction of valid confidence intervals for treatment effects, even with many covariates re...
335 CitationsSource
#1Aaron Chalfin (U of C: University of Chicago)H-Index: 14
#2Oren Danieli (Harvard University)H-Index: 2
Last. Sendhil Mullainathan (Harvard University)H-Index: 86
view all 7 authors...
Economists have become increasingly interested in studying the nature of production functions in social policy applications, with the goal of improving productivity. Traditionally models have assumed workers are homogenous inputs. However, in practice, substantial variability in productivity means the marginal productivity of labor depends substantially on which new workers are hired--which requires not an estimate of a causal effect, but rather a prediction. We demonstrate that there can be lar...
38 CitationsSource
#1Daniel KahnemanH-Index: 149
#2Andrew RosenfieldH-Index: 1
Last. Tom BlaserH-Index: 1
view all 4 authors...
Organizations expect to see consistency in the decisions of their employees, but humans are unreliable. Judgments can vary a great deal from one individual to the next, even when people are in the same role and supposedly following the same guidelines. And irrelevant factors, such as mood and the weather, can change one person’s decisions from occasion to occasion. This chance variability of decisions is called noise, and it is surprisingly costly to companies, which are usually completely unawa...
58 Citations
#1Anna Aizer (Brown University)H-Index: 27
#2Joseph J. Doyle (MIT: Massachusetts Institute of Technology)H-Index: 19
Over 130,000 juveniles are detained in the United States each year with 70,000 in detention on any given day, yet little is known about whether such a penalty deters future crime or interrupts social and human capital formation in a way that increases the likelihood of later criminal behavior. This article uses the incarceration tendency of randomly assigned judges as an instrumental variable to estimate causal effects of juvenile incarceration on high school completion and adult recidivism. Est...
137 CitationsSource
4 Citations
Cited By203
Abstract null null This paper aims to uncover the deterrent effect of external social ties on online peer-to-peer lending platforms by extracting valuable information from the phone usage data. Contrary to the studies viewing online internal friendships as the signal of credit quality, we find that the total number of outgoing calls made in the month before loan requests is positively associated with the probability of default, which suggests that more social interactions offline mediated by mob...
#1Nadine Schlicker (Saarland University)H-Index: 3
#2Markus Langer (Saarland University)H-Index: 9
Last. Dieter WallachH-Index: 11
view all 7 authors...
Abstract Advances in artificial intelligence contribute to increasing automation of decisions. In a healthcare-scheduling context, this study compares effects of decision agents and explanations for decisions on decision-recipients’ perceptions of justice. In a 2 (decision agent: automated vs. human) × 3 (explanation: no explanation vs. equality-explanation vs. equity-explanation) between-subjects online study, 209 healthcare professionals were asked to put themselves in a situation where their ...
3 CitationsSource
Abstract null null Fairness emerged as an important requirement to guarantee that Machine Learning (ML) predictive systems do not discriminate against specific individuals or entire sub-populations, in particular, minorities. Given the inherent subjectivity of viewing the concept of fairness, several notions of fairness have been introduced in the literature. This paper is a survey that illustrates the subtleties between fairness notions through a large number of examples and scenarios. In addit...
This paper summarizes the empirical evidence on the role of justice in economic development, conflict, and trust in institutions. It finds that justice institutions play a significant role in economic development, particularly through their impact on credit markets and firm growth, the protection of vulnerable populations, their capacity to deter violence, and their influence over people’s trust in formal institutions. The paper then considers the promise of administrative data, machine learning...
A new Big Data cluster method was developed to forecast the hotel accommodation market. The simulation and training of time series data are from January 2008 to December 2019 for the Spanish case. Applying the Hierarchical and Sequential Clustering Analysis method represents an improvement in forecasting modelling of the Big Data literature. The model is presented to obtain better explanatory and forecasting capacity than models used by Google data sources. Furthermore, the model allows knowledg...
#1Magdalena Wojcieszak (UC Davis: University of California, Davis)H-Index: 23
#2Arti Thakur (UC Davis: University of California, Davis)H-Index: 1
Last. Miriam Boon (UvA: University of Amsterdam)
view all 0 authors...
#1Sina Fazelpour (NU: Northeastern University)
#2David Danks (CMU: Carnegie Mellon University)H-Index: 18
Predicting and understanding how people make decisions has been a long-standing goal in many fields, with quantitative models of human decision-making informing research in both the social sciences and engineering. We show how progress toward this goal can be accelerated by using large datasets to power machine-learning algorithms that are constrained to produce interpretable psychological theories. Conducting the largest experiment on risky choice to date and analyzing the results using gradien...
1 CitationsSource
#1Michelle Bao (Stanford University)H-Index: 1
#2Angela Zhou (Cornell University)H-Index: 8
Last. Suresh Venkatasubramanian (UofU: University of Utah)H-Index: 48
view all 0 authors...
Risk assessment instrument (RAI) datasets, particularly ProPublica's COMPAS dataset, are commonly used in algorithmic fairness papers due to benchmarking practices of comparing algorithms on datasets used in prior work. In many cases, this data is used as a benchmark to demonstrate good performance without accounting for the complexities of criminal justice (CJ) processes. We show that pretrial RAI datasets contain numerous measurement biases and errors inherent to CJ pretrial evidence and due t...
1 Citations
#7Adel Daoud (Harvard University)H-Index: 12
Abstract null null Background null Machine learning (ML) has spread rapidly from computer science to several disciplines. Given the predictive capacity of ML, it offers new opportunities for health, behavioral, and social scientists. However, it remains unclear how and to what extent ML is being used in studies of social determinants of health (SDH). null null null Methods null Using four search engines, we conducted a scoping review of studies that used ML to study SDH (published before May 1, ...