arrow_back Back to forum
Engineering 1 day ago

saw this: agents pass the tests, about a quarter of the code is actually secure

by Yusuf Kaur

There's an arXiv paper from last month (Last Secure Code Benchmark) that gave six coding agents tasks where the spec only asks for features, never mentions security, then graded with hidden tests for both. Functional pass rates were roughly 74 to 91%. Code that was functional AND secure topped out around 25%, for the best one. This annoys me because I'm the person who keeps saying your test suite is the truth. Turns out it's the truth about what you asked for. Nobody writes "don't build the SQL string by hand" into a ticket. So do you put security requirements in the spec now, like acceptance criteria, or is that just reinventing a linter with extra steps? Ours is agent writes, scanner runs, human reviews. Not sure the middle step is pulling its weight.

favorite 3 comment 3

Comments

Greta Nakamura 1 day ago

Yusuf, I'd keep the scanner and change what it's for. In our setup it never catches anything clever. What it does is stop the agent getting promoted to the next autonomy stage on a task type until it's had a clean run for a while. Remove it and you lose the record, not the protection. Where I disagree is putting security into every spec. Agents follow whatever's written down very literally, so you'll get exactly the five rules you listed and nothing about the sixth. We had better luck with one short standing file in the repo (how we query the db and where secrets come from, mostly) that every task picks up automatically. The paper is actually useful news for us. A quarter is bad, but it's measurable now, which means it can go up.

Elif Arslan 1 day ago

New here, I do appsec reviews. Honestly the agent PRs I see aren't worse than the human ones on average. They're just three times longer, so the one bad query is harder to spot.

Olivia Chen 6 hours ago

Elif's point is the one I'd build around. If the PRs are three times longer, the fix probably isn't a smarter scanner, it's smaller PRs. We started capping agent changes at around 300 lines per task and anything bigger has to come back as a plan first, then separate diffs. People went back to actually reading them instead of scrolling to the green tick. Greta, I'm with you on the standing file, but keep it embarrassingly short. Ours grew to two pages and the agent followed the top half and drifted on the rest. Six lines on queries, secrets and auth did more than the essay. Yusuf, I'd love to see that benchmark rerun with a size cap. My guess is the secure number moves more than people expect.

Log in to join the discussion.