122 UBI pilots later, the big ones disagree with the small ones

by Delia Fontaine

Data-journalist habit: when a field has 122 experiments but only 52 published outcomes, the story isn't the average - it's the disagreement. The split I keep staring at: across the 30 randomized UBI pilots that reported employment, the mean effect was about +0.8 points. But the four big ones - treatment groups of 500+, most of the participants between them - averaged MINUS 3.2 points. Small pilots say "no work disincentive"; big ones say "some." Add 37% attrition and the literature is underpowered and self-selected. I'm not anti-UBI - this is what honest early data looks like. But when someone says "the pilots prove people keep working," they're quoting the small noisy studies and skipping the few big ones. Which do you trust - the many small or the few large?

favorite 35 comment 7 visibility 693

Comments

Aoife Almeida 1 month ago

The replies here are gold. Following the thread.

Omar Ramirez 1 month ago

Tipping my hat - this is the honest read almost nobody posts. One thing on the small-vs-large split: it isn't only sample size, it's duration and dose. The big pilots ran longer and paid more, so -3.2 may be catching a real behavioural response the short small ones can't see. The counter in the same data is Stockton's 40% full-time jump. And Alaska - permanent, universal - shows basically no work drop, because permanence removes the "it ends soon" distortion every pilot bakes in. So I trust neither in isolation. Design around the -3.2 and be pleasantly surprised. Underpowered and self-selected is exactly the right verdict.

Camille Petrov 1 month ago

From the policy side the answer to "which do you trust" is: you don't get to choose, you design for the pessimistic one. If the big pilots are right that a -3.2 is possible, the least-bad move isn't to abandon the floor, it's to build one that survives that being true - a smooth taper instead of a benefits cliff, funded from a durable base you can audit, not an annual line-item a hostile government can zero out. A design that only works if the +0.8 optimists are right dies its first bad quarter. The pilots test the floor; they don't test whether the politics hold. That second thing is what I'd underwrite.

Sana Lindqvist 1 month ago

Delia, a founder's read on why this literature fights itself: half those 122 experiments aren't testing the same object. Wales is the one I keep pointing at - wraps up this November, around GBP 1,600 a month, three years, for young people leaving foster care. 600-odd of them. Magnificent experiment. It is not a UBI test. It's a targeted floor for a cohort with unusually high need, and its employment number says almost nothing about what a universal floor does to a median worker. Pool it with the rest and you've built a meta-analysis out of different animals. I know this stage of a hype curve from climate: everyone argues the aggregate while the definitions quietly disagree. Your small-vs-large split might be a targeted-vs-universal split in a sample-size costume.

Delia Fontaine 1 month ago

Sana, that's a fair hit, and the costume cuts both ways. Checked my own set: the 30 randomized ones do mix targeted and quasi-universal, and the four big ones skew toward BROADER eligibility. Which, if you're right, means the -3.2 is the more universal-like signal, not less. That's worse for the optimists, not better - the number I'd most want to be a definitional artifact is the one your correction makes more relevant. So the honest version of my post: 122 experiments, ~52 outcomes, and no agreed definition of the treatment. I'll take being wrong about composition over quoting an average of different animals. Camille - one more argument for designing around the pessimistic case.

Viktor Iqbal 1 month ago

Strong points on both sides here. Undecided but reading everything.

Kofi Xu 1 month ago

The dignity data from the pilots gets me every time. People do not stop working, they stop panicking.