Agent Autonomy Limitations

Models unable to one-shot complex issues, need for human guidance, hardcoding problems, slop generation without oversight, 20% time spent addressing slop

← Back to Harness engineering: Leveraging Codex in an agent-first world

While AI agents excel at rapid prototyping and boilerplate tasks, experienced developers argue that full autonomy remains a myth, as models frequently generate "slop" and hardcoded hacks that require humans to spend roughly 20% of their time on corrections. The consensus suggests that true competitive advantage has shifted away from raw code velocity toward human-led architectural design and customer discovery, which AI cannot yet replicate in complex, long-lived codebases. Users highlight that without deterministic guardrails and constant steering, these tools often hit token limits or fail to navigate novel problems, functioning more as high-speed "vibe coding" assistants than independent engineers. Ultimately, the transition to agentic workflows emphasizes that while these tools are better than ever, the most critical "engineering" still happens in the human mind rather than the model's output.

16 comments tagged with this topic

View on HN · Topics
AI coders are great for making scrapers, possibly because AI companies use their own tools to make an awful lot of scrapers.
View on HN · Topics
It is likely better because AI agents make access to domain knowledge easier. However, I would wager that the problem is people don’t remember the code well. The problems are going to be long-term as the pace of change increases. If you think about it, successful products rely on designing well-thought-out experiences, customer discovery (see all the Forward-Deployed Enginneer job listings at OpenAI) so the code velocity somewhat becomes irrelevant. If you’re solving the right problem and you’ve got a good team then competitive advantage comes from somewhere OUTSIDE of code velocity. The more important question I think is does faster code yield more value long-term? At the moment, it’s like yeah we do 3.5 pull requests per day. I’m thinking, great, good for you. You could also combine three pull requests into one and then you’re doing 1 per day. This is quantitative data that doesn’t really mean anything tangible.
View on HN · Topics
1. Yes! Many teams internally have adopted a lot of the same practices we outlined in the blog post. Ryan has also been spending time both internally and externally helping companies figure out how to do this in their code bases. 2. Hmm, kind of. There have definitely been issues the models can’t one shot. But we still use Codex to write all the actual code with human guidance. 3. More agents :) Some teams are experimenting with centralized Agent mediated integration queues, others use normal merge queues, many have local Codex threads that monitor CI to resolve and land conflicts or failures. 4. Today’s models and codex app. We started doing all this with gpt-5 and codex-cli. The tools today, 9 months later, are so much better than what we had then.
View on HN · Topics
it does not result in great results left unattended, it’ll start creating slop or hardcoding solutions but overtime if you adjust your verification rubric, it’s not too bad, gets pretty good, if you do make it do TDD, it gets kinda crazy and you’ll have 2000-3000 tests after awhile, or on my common case, 6000-7000 lines of code in single files (i usually have a cron to audit files for decomposition and create tickets) i wouldn’t use it at my job yet, but it’s been fun to use for personal projects - it’s like modded minecraft automation or factorio
View on HN · Topics
This might work only if you have “infinite” compute and infinite tokens. As someone that used the $20 plan, this pure agentic approach is impossible to do because I’d hit the limit fast and I would end up with less outcome. What I found that work incredibly well was to provide a human written code as reference, and ask it to extend it. So I scaffold the entire thing, architect it, write few samples code (controllers, services, models, components, database schema, how auth works, etc) so the LLM can have a headstart on their attention (pun intended) I usually wrote a stub with a lot of details on how to implement it. Something like a higher abstraction pseudo code. Then ask the LLM to implement it. When it fails, it is often better to undo the whole changes, adjust the stub so it catches what fails before, and try again. Or, commit the changes, and use a new fresh context and only address what went wrong. - Whenever I tried this agentic from scratch approach, I always end up disappointed; both on the outcome and on the limit that I hit before an hour even passed.
View on HN · Topics
I worry most about blindspots with this kind of approach. Let's say that this repository goes on for years, at which point the docs folder is several MB in size. Would Codex be able to think outside of the box? Or would the aggregate of the Markdown content fundamentally cover enough ground to prevent it from thinking of novel new approaches to existing problems?
View on HN · Topics
It’s not a self coding machine. There is human in the loop, they even added MORE engineers to the team of this project! 7 engineers should be able to collaborate with the AI to find good solutions to problems.
View on HN · Topics
I don’t think it’s solvable. And I think Anthropic etc know it. LLMs can only reconstitute things in its training data and they are so hungry they can’t do a good job in long lived codebase full of complexity and novelty. There’s never going to be enough similar code on the open internet.
View on HN · Topics
> LLMs can only reconstitute things in its training data Such as a 4D raytracing engine in Metal? Or integrating APIs for features first released months after their knowledge cut-off date? LLMs have shown an ability to transfer "knowledge" and capabilities across domains, languages, and use-cases outside their training data. Case in point: GPT-2 "learning" to translate English to French and vice versa despite non-English examples having been voluntarily (and almost entirely) removed from the dataset.
View on HN · Topics
This hasn’t been my experience. State of the art models available to the public still do all sorts of bandaids and bad hacks. Putting code where it doesn’t belong. Stapling types onto variable (in TypeScript) when abstractions/types already exist to use. I use it to generate code, but still have to review every line and have corrections/steering basically every time. Maybe you have access to some other model?
View on HN · Topics
> The latest frontier models will write code better than you and more elegant They often do, but they often don’t. I regularly have to push for more elegant, or less lazy solutions.
View on HN · Topics
Q1 - How much effort did you put into deterministic guardrails like AST linters, etc? I find there’s a ton of slop unless hard guardrails are added, eg step 1 is just around syntax, step 2 is to enforce mental models You still need someone steering direction and have a logically consistent idea of what you actually want to build Q2 - I find that vibe coding really accelerates FE projects because it’s possible to run everything locally and check results For pure distributed infra backend more investments have to be made into the devloop to be able to shift left the feedback loop and decouple it from humans or real deploys
View on HN · Topics
I like how they said they were spending 20% of their time addressing slop. Sounds like they’ve tried to automate the slop correction but it’s a good honest reminder. Additionally it’s an internal tool, which is likely much more amenable to slop.
View on HN · Topics
I started using chatgpt for functions and checking, then for single file changes and checking, now for multiple changes and checking. I am at a point where the only changes I correct are architectural. So it may start to become smarter to learn how to see only the architectural directions while multiple agents work, test, and commit both on unit and against live deployment.
View on HN · Topics
I understand that the’ve written zero lines of code for this application, but would it kill them to write a few lines of the blog post by hand? Forcing readers to wade through an unceasing string of LLM clichés demonstrates the opposite of the point you’re trying to make—that the consumers of your work are worse off because you exercised no human judgment in creating it.
View on HN · Topics
"Engineering" These people are so delusional it feels like a mental desease by now. I really hope no one gets hurt by all this slop code in the future by these wanna be engineers.