Imagine you're a weather forecaster who has always relied on sending up actual weather balloons to verify conditions at altitude. Balloons are expensive, slow, and you can only launch a few per year. Now someone hands you a satellite model trained on decades of atmospheric data that predicts balloon readings so accurately that the satellite's error margin is indistinguishable from the balloon's own measurement noise. You'd still want balloons occasionally — but you wouldn't need one for every forecast. That's what Chetty, Katz, and co-authors have built for workforce program evaluation. The committed claim: machine learning models trained on full-population IRS records (1999–2025) can predict what a job-training participant would have earned without the program — the counterfactual — well enough that the resulting treatment-effect estimates match 37 separate RCTs with RMSE comparable to the RCTs' own sampling variability. This is not a minor technical improvement. It's a category shift in how you'd evaluate government programs at scale. The architecture leans on something conceptually simple but computationally demanding. Take anonymized tax records covering every American adult — income histories, demographics, residential location — and train flexible ML models (the paper says 'machine learning models' without specifying exact architectures, but the emphasis on 'higher-dimensional' and 'more flexible' suggests gradient-boosted trees or similar ensemble methods on tabular data) to predict earnings trajectories for non-participants who look like participants. The key insight isn't the ML itself — it's the combination of population-scale administrative data and rich longitudinal covariates that makes the selection-on-observables assumption plausible in a way that small survey datasets never could. Prior observational studies failed not because observational methods are inherently broken, but because their data was too thin to model selection into programs accurately. The integrity setup here is unusually strong for economics. The authors don't just run one RCT benchmark — they use 37 separate randomized trials as ground truth, then estimate the distribution of bias in their observational method rather than just testing for zero bias. The 60% RMSE reduction over parametric estimators (the standard regression-based approaches that gave observational methods a bad name in the LaLonde 1986 tradition) is measured against RCT benchmarks, not against the authors' own simulations. The validation target — actual RCTs — is about as independent as you can get in social science. The main integrity caveat: the 37 RCTs are all workforce training programs, so generalization to other policy domains (education, health, criminal justice) remains unvalidated. The substantive findings from the scaled-up observational analysis are arguably as important as the methodological proof-of-concept. Treatment effects vary substantially across locations and over time within the same program — meaning the average RCT result from one site in one year is a poor guide to what the program does elsewhere. Programs that place participants into target-sector jobs generate the largest earnings gains, and those gains persist even after participants leave their initial placements. This is the kind of heterogeneity analysis that RCTs almost never have statistical power to deliver. The ladder position is clear: the predecessor this paper argues against is LaLonde (1986) and the subsequent Dehejia-Wahba literature, which established the conventional wisdom that observational estimates of training programs are unreliable. That finding held when you had the Current Population Survey and a few hundred controls. It does not hold when you have the universe of American tax returns and modern ML. The paper doesn't just beat the parametric observational baseline — it renders the original negative result historically contingent on data limitations, not methodological ones. The 'digital twin' framing is strategic and tells you where this is headed. The authors want this to become a general-purpose evaluation infrastructure — any program where you can observe participants in administrative data becomes evaluable without running a new RCT. The constraint is data access: this only works because the authors had Census Bureau access to IRS records. The approach doesn't democratize evaluation; it centralizes it among researchers with the right federal data agreements.
