Learning More from Field Experiments
Paper Session
Saturday, Jan. 8, 2022 3:45 PM - 5:45 PM (EST)
- Chair: Roland Rathelot, University of Warwick
Anticipation and Consumption
Abstract
Cash transfer payments are an increasingly widespread policy tool in developed and developing countries, used for both short-term and long-term objectives. We study the design of these policies by examining how the time horizon over which households anticipate receiving transfer payments affects consumption and savings. Using Nielsen Consumer Panel data, we estimate higher marginal propensities to spend for US households scheduled to receive the 2008 Economic Stimulus Payments sooner. Analyzing data from randomized experiments in Kenya and Malawi, we document higher savings among households scheduled to wait longer before receiving lump-sum transfers. We discuss implications of our results through a model of mental accounting.How Biased Are Observational Methods in Practice? Accumulating Evidence Using Randomised Controlled Trials with Imperfect Compliance
Abstract
Consider a policy maker choosing between programs of unknown impact. She can inform her decision using observational methods, or by running a randomised controlled trial (RCT). The proponents of RCTs would argue that observational approaches suffer from bias of an unknown size and direction, and so is uninformative. Our study treats this as an empirical claim that can be studied. By doing so we hope to increase the value of observational data and studies, as well as better inform the choice to undertake RCTs. We propose a large-scale, standardised, hands-off approach to assessing the performance of observational methods. First, we collect and categorise data from a large number of RCTs in the past 20 years. Second, we implement new methods to understand the size and direction of expected bias in observational studies, and how bias depends on measurable characteristics of programmes and settings. This will allow us to predict in which context observational studies can provide reliable guidance on the impact of policy interventions.Evaluating Ex Ante Counterfactual Predictions Using Ex Post Causal Inference
Abstract
We derive a formal, decision-based method for comparing the performance of counterfactual treatment regime predictions using the results from randomized experiments. Our approach allows us to quantify and assess the statistical significance of differential performance for optimal treatment regimes estimated from structural models, extrapolated treatment effects, expert opinion, and other methods. We apply our method to evaluate optimal treatment regimes for conditional cash transfer programs across countries where predictions are generated using data from experimental evaluations in other countries and pre-program data in the country of interest.JEL Classifications
- C2 - Single Equation Models; Single Variables
- C5 - Econometric Modeling