Frontiers for Analysis of Experiments and Clinical Trials
Paper Session
Friday, Jan. 3, 2025 2:30 PM - 4:30 PM (PST)
- Chair: Noam Angrist, University of Oxford
Mean Reversion in Randomized Controlled Trials: Implications for Program Targeting and Heterogeneous Treatment Effects
Abstract
Randomized controlled trials (RCTs) with eligibility criteria based on extreme values of lagged outcomes may be subject to mean-reverting statistical or behavioral processes, where outcomes improve over time absent interventions. We investigate this issue in the context of a food-as-medicine program to treat diabetes, where eligibility required elevated hemoglobin A1c (HbA1c). Both treatment and control groups demonstrated significant HbA1c improvements, resulting in a null effect. We employ machine learning techniques to predict mean-reversion and assess whether the program benefited participants who were least likely to improve independent of treatment assignment. Our findings have implications for program targeting and estimating heterogeneous treatment effects.Randomization for Causality, Ethnography for Mechanisms: Illiquid Savings for Liquor in an Autarkic Society
Abstract
What should researchers do when confronted with surprising results? Financial access innovations usually leave “temptation” spending unaffected or reduced. However, we found that promotion of savings lockboxes in a largely autarkic society increased alcohol consumption and blood pressure, despite no one reporting intentions to save for alcohol. To probe mechanisms that could explain this pattern, we then used ethnographic methods, including direct observations of drinking (“scans”) and debriefing interviews to discuss the earlier trial results. We learn that sponsoring drinks confers prestige, but the stigma attached to drinking by outsiders likely discouraged reporting intentions to save for it.Instrumental Variable Methods Reveal Larger Effects of Menopausal Hormone Therapy in the Landmark Women’s Health Initiative Clinical Trial
Abstract
Menopausal hormone therapy (MHT) was the most frequently prescribed drug in the United States in the 1990s. Prescriptions dropped precipitously after widely publicized results from the Women’s Health Initiative (WHI) clinical trial showed the group assigned to treatment suffered elevated invasive breast cancer and other risks. These findings remain controversial, however, with recent analyses arguing that disease risks due to MHT are small, long-run mortality risks are null, and the benefits of MHT given in early menopause are substantial. The first of these conclusions may be misleading: WHI trial analyses to date report intention-to-treat (ITT) effects, ignoring the fact that many women assigned to treatment remained untreated or were not treated for long, while many women assigned to control initiated treatment before the study ended. We use previously neglected data on adherence to random assignment in an instrumental variables (IV) framework to estimate the effects of MHT on the subsample of treated adherents who complied with random assignment. IV estimates show that health risks for women assigned MHT and adhered to MHT as a result are substantially larger than previously reported ITT estimates.Discussant(s)
Kaspar Wuthrich
,
University of Michigan
Adrienne Sabety
,
Stanford University
Isaac Opper
,
RAND
Rita Ginja
,
University of Bergen
JEL Classifications
- C9 - Design of Experiments
- H1 - Structure and Scope of Government