This website uses cookies.

By clicking the "Accept" button or continuing to browse our site, you agree to first-party and session-only cookies being stored on your device to enhance site navigation and analyze site performance and traffic. For more information on our use of cookies, please see our Privacy Policy.

« Back to Results

Machine Learning in Experiments

Paper Session

Saturday, Jan. 4, 2020 8:00 AM - 10:00 AM (PDT)

Marriott Marquis, Grand Ballroom 12
Hosted By: American Economic Association
  • Chair: Jeffrey Naecker, Wesleyan University

Predicting and Understanding Initial Play

Annie Liang
,
University of Pennsylvania
Drew Fudenberg
,
Massachusetts Institute of Technology

Abstract

We use machine learning to uncover regularities in the initial play of matrix games. We first train a prediction algorithm on data from past experiments. Examining the games where our algorithm predicts correctly, but existing economic models don’t, leads us to add a parameter to the best performing model that improves predictive accuracy. We obtain even better predictions with a hybrid model that uses a decision tree to decide game-by-game which of two economic models to use for prediction. Finally, we explore the usefulness of crowd-sourced predictions formaking better predictions, and for discovering additional relevant game features.

Using Machine Learning to Understand Bargaining Experiments

Colin Camerer
,
California Institute of Technology
Juan Pablo Franco
,
University of Melbourne
Gideon Nave
,
University of Pennsylvania
Alec Smith
,
Virginia Tech

Abstract

We study dynamic unstructured bargaining with deadlines and one-sided private information about the amount available to share (the “pie size”). "Unstructured" means that players can make or withdraw any offers and demands they want at any time. Such paradigms, while lifelike have been displaced in experimental studies by highly structured bargaining because they are hard to analyze. Machine learning comes to the rescue because the players' wide range of choices in unstructured bargaining can be taken as "features" used to predict behavior. Machine learning approaches can accommodate a large number of features and guard against overfitting using test samples and methods such as penalized LASSO regression. In previous research we found that LASSO could add power to theoretical variables in predicting whether bargaining ended in disagreement. We replicate this work with higher stakes, subject experience, and special attention to gender differences. In addition, we add biomarkers, skin conductance and facial emotion reading, to further predict private information in order to lay the groundwork for biologically-informed mechanism design that can, in principle, eliminate all bargaining inefficiencies.

Supervised Machine Learning for Eliciting Individual Demand

John Clithero
,
University of Oregon
Jae Joon Lee
,
Claremont Graduate University
Joshua Tasoff
,
Claremont Graduate University

Abstract

Direct elicitation, guided by theory, is the standard method for eliciting individual-level la- tent variables. We present an alternative approach, supervised machine learning (SML), and apply it to measuring individual valuations for goods. We find that the approach is superior for predicting out-of-sample individual purchases relative to a canonical direct-elicitation approach, the Becker-DeGroot-Marschak (BDM) method. The BDM is imprecise and systematically biased by understating valuations. We characterize the performance of SML using a variety of estimation methods and data. The simulation results suggest that prices set by SML would increase revenue by 22% over the BDM, using the same data.

The Model You Know: Benchmarks for Models of Risk Preferences

B. Douglas Bernheim
,
Stanford University
Christine Exley
,
Harvard Business School
Jeffrey Naecker
,
Wesleyan University
Charles Sprenger
,
University of California-San Diego

Abstract

Cumulative prospect theory is viewed as a successful model of risk preferences because of its ability to better explain choice patterns than expected utility. However, utility-based models must also be tested by out-of-sample predictive power. We show that while CPT does indeed have better in-sample fit than EU, both models fail to make accurate out- of-sample predictions as compared to a model that makes predictions from non-choice responses. We show that the utility-based models fall behind because of their failure to predict across situations with differing amounts of certainty.
Discussant(s)
Monica Capra
,
Claremont Graduate University
Ben Gillen
,
Claremont McKenna College
Alec Smith
,
Virginia Tech
JEL Classifications
  • C9 - Design of Experiments
  • C1 - Econometric and Statistical Methods and Methodology: General