Language and Density - Harness engineering: Leveraging Codex in an agent-first world

Observations that concise languages like Clojure may perform better with LLMs, relationship between code density and agent effectiveness

The high density and simple primitives of languages like Clojure may actually enhance LLM performance, allowing models to reason more effectively with fewer tokens compared to more verbose languages like Python or Go. While smaller, dense files seem to yield faster and more accurate results, there are concerns that this efficiency could degrade when agents are forced to navigate complex dependencies across large codebases. Furthermore, the push toward "agent-readable" code raises significant risks regarding human interpretability, potentially creating a future where technical debt is impossible for humans to manage without AI intervention. To mitigate this, developers suggest that AI training needs new reward systems that prioritize "taste" and refinement over mere production speed.

View on HN · Topics

Seconded on smaller files. I feel like I tend to get better responses faster.

A notable flaw here is that I’ve not tried large vs small files in a large codebase. Most of my experimentation there has been on personal projects where even a small file contains a significant part of the project. I could see degradation when it has to load 5 files to figure out how something works.

Total LOC (tokens, really, literal lines probably don’t matter) is interesting as a factor. That might go some way towards explaining why LLMs are weirdly good at Clojure.

Eg last I checked Anthropics one-shot performance on Clojure was about the same as Python or Go despite almost certainly being less represented in training data. The combination of density and simple primitives might be easier for an LLM to wrangle, ameliorating the impact of a less popular language.

View on HN · Topics

You can't really do that here because one of the key arguments for this, as people in the thread focus on is "1/10th of time" estimate, the comparison with humans is here already, albeit it is just an estimate and no actual comparison has been done.

This is a problem of conflicting incentives that exists today in my opinion. Companies will market greater human-AI collaboration in science and engineering but focus on releasing things like this where it is clear that downstream goal is complete agent ownership over the product, from inception to testing to monitoring. Maybe the speculative future agents will use their own very efficient language to code that won't be readable for people at all. They focus on agent code being readable by agent in the article, as you've said. But in my mind in at least near future, there is a case where your prod will break, you won't be able to understand it or the attempted fixes. Maybe agent will fail to fix it at all and start a massive rewrite. In any case is this different from kicking technical debt down the road along with worse interpretability of what you have built?

I do think there is a way where agent can write great solid code that we can read, but with the way LLMs are built this requires something new in terms of reward that accounts for "taste" and constant refinement so it might take more than 1/10th of a time to produce something good.