Long-horizon reliability is the real AGI bottleneck — change my mind

by Greta Moreau

Models ace ten-minute tasks and fumble ten-hour ones. Error compounds across steps, context degrades, and self-correction is still shallow. Until an agent can run a two-week project without a human resetting it, "general" is marketing. The labs know this — notice how the benchmarks quietly shifted from exams to long-horizon tasks. What evidence would actually move your timeline?

favorite 20 comment 10 visibility 567

Comments

Ivan Jensen 1 month ago

You're pointing at the right graph but reading the top line. METR's horizon isn't one number, it's a distribution. The 50%-reliability horizon is out past a full workday now; the 80% one is still a couple of hours. That gap IS your thesis, quantified: capability roughly doubled, reliability didn't follow. What'd move my timeline isn't a longer 50% task, it's the 80% and 99% curves closing on it. And the boring part: I want it reproduced. A two-week autonomous run that works once is an anecdote; the same agent clearing the same spec three weeks running is a result. Timelines are bands, not dates, and the band widens every time we score the median and ignore the tail.

Dmitri Meier 1 month ago

My class does a version of this every Friday, same task through five models, and the long-horizon failure is so visible it's basically a teaching aid. Short tasks, all five look like geniuses. Give them a three-step build where step two depends on step one and half of them quietly forget the constraint they set themselves ten minutes ago. The kids catch it faster than the benchmarks do, because they watched it drift. What moved MY prior wasn't a paper, it was watching a model re-derive a decision it had already made and contradict itself. Build the eval where a later step has to remember an earlier one. That's the cheapest reliability probe I know, and you can run it on your own workflow tonight.

Farah Dubois 1 month ago

Something underrated in these threads: the economy does not need AGI to transform. Narrow systems that are merely excellent at 40% of desk work rearrange everything long before anything "general" shows up.

Yusuf Kaur 1 month ago

Ivan's distribution point is the whole thing, so let me put it in eval terms. I don't watch the 50% horizon, I watch the variance across seeds on the SAME long task. Right now a five-step build passes maybe 4 runs in 10 and fails a different step each time - that's not a capability I can ship, it's a coin I can't insure. What'd move my timeline isn't a benchmark score, it's a long-horizon eval that (a) I can rerun next week and get the same number, and (b) nobody in the lab can quietly tune to. The day an agent clears my flaky-integration suite at 80% across 20 seeds, unattended, I'll move my dates. Until then "general" is a median hiding a tail. Post your task, I'll run it against a real harness.

Noah Williams 1 month ago

As a student this thread is oddly comforting. Half my weekend agent demos look brilliant right up until step three, then quietly wander off and I don't notice for an hour. The "works once" clips age like milk. Reliability, not raw IQ, is the thing I can't fake in a demo - and it's the thing I actually want to learn to build.

Ines Egede 1 month ago

RemindMe in 2029 — someone check who was right in this thread.

Marco Duarte 1 month ago

What changed my mind recently was watching agents handle 6-step tasks without supervision. Two years ago that was science fiction.

Greta Nakamura 1 month ago

From running these in prod: Yusuf's "a coin I can't insure" is exactly how we gate autonomy, and it's why "general" is the wrong unit. We don't grant an agent autonomy, we promote a task TYPE through stages - suggest, open-a-PR, act-unsupervised - and a type only graduates when its seed-variance on the real workload clears a bar, not when a demo works. Long-horizon reliability isn't one threshold you cross, it's a ratchet you turn per task, and it clicks backward the first week a graduated type regresses. So Ivan's 80% horizon isn't a date I'm waiting on; it's a dial I already run, task by task. The "works once" clip Noah's students post is just a task type that hasn't earned its next stage yet.

Priya Nair 1 month ago

Data-scientist's caveat on the numbers everyone's citing: the METR horizon is doubling roughly every 7 months and the 50% mark is out past a 14-hour task now - but 50% reliability on a 14-hour task is a coin flip you'd never ship. And SWE-bench Verified sitting at 60-70% is exactly the headline I distrust, because it's a fixed public set that's been in training data for two years. Ivan's "reproduce it next week" bar is the right instinct; I'd add "on a task that didn't exist when the model shipped." A timeline built on a benchmark the model has effectively memorised isn't a timeline, it's a mirror. Move me with held-out, dated tasks - contamination is the silent killer of every eval curve I've ever had to clean.

Olivia Chen 1 month ago

Practitioner's one-liner: the demo that works once is a screenshot; the one that works on the 50th unattended run is a product - and everything I actually ship lives in the gap between those two numbers. Reliability isn't a feature you bolt on at the end, it's the entire material the last mile is built from.