Better Measurement or Larger Samples? Data Collection for Policy Learning with Unobserved Heterogeneity
TLDR
This paper explores how to optimize data collection for policy learning, balancing measurement precision of latent traits with sample size to maximize welfare.
Key contributions
- Characterizes how estimate precision impacts worst-case policy performance, deriving rate-sharp regret bounds.
- Develops a framework for designing tailored data collections to optimize policy learning trade-offs.
- Derives the minimax optimal data collection plan for policies with unobserved heterogeneity.
- Empirically shows including a business skills proxy boosts welfare by 5% in cash transfer targeting.
Why it matters
This paper provides a rigorous framework for policymakers to optimize data collection when dealing with unobserved individual differences. It offers practical guidance on balancing measurement precision and sample size, demonstrating how better data can significantly improve policy outcomes and welfare.
Original Abstract
Empirical research shows that individuals' responses to treatments vary along latent characteristics, such as innate ability or motivation. Therefore, a policymaker seeking to maximize welfare may consider designing policies based on observed characteristics and estimated latent traits. I characterize how the estimates' precision affects the worst-case performance of policies, deriving rate-sharp regret bounds for assignment rules that include or exclude them, highlighting new trade-offs with the policy space complexity. I then study how a policymaker can solve such trade-offs by designing tailored data collections, and derive the minimax optimal collection plan. In an empirical application in development economics, I show that including a proxy for entrepreneurs' business skills in targeting cash transfers increases welfare by 5%, and halves the probability of generating welfare losses. Moreover, I estimate the optimal allocation of resources between improving the precision of the proxy via repeated measurements and increasing sample size.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.