Human Review Limitations - Harness engineering: Leveraging Codex in an agent-first world

Comparison to factory workers testing products, inability to meaningfully review hundreds of lines of AI-generated code per PR

Human review of massive AI-generated pull requests is often likened to a rushed factory assembly line, where the sheer volume of output forces superficial, performative inspections. While some developers admit to skimming large blocks of code and deferring the heavy lifting to automated test suites, others insist that such a hands-off approach is premature and dangerous. This counter-perspective maintains that even state-of-the-art models frequently produce "hacks" and structural flaws, necessitating meticulous, line-by-line human steering to maintain quality. Ultimately, the discussion highlights a growing tension between the high-speed efficiency of AI generation and the practical, cognitive limits of the humans tasked with verifying it.

View on HN · Topics

The other day I came across to a video showing workers in a e-vape factory. They pick up a bunch of e-vapes from the conveyor belt (each has 6 e-vape think), stick in their mouth and vigorously vape all of them for about 5 seconds, then test the next bunch. Humans reviewing hundreds of lines of change in a PR written by AI is not very different.

View on HN · Topics

Very true. If a PR has 1000 lines I would check only a handful full of them and leave the rest for test suit .

View on HN · Topics

This hasn’t been my experience. State of the art models available to the public still do all sorts of bandaids and bad hacks. Putting code where it doesn’t belong. Stapling types onto variable (in TypeScript) when abstractions/types already exist to use. I use it to generate code, but still have to review every line and have corrections/steering basically every time.

Maybe you have access to some other model?