Summarizer

I agree. I send 2 sentence replies to most things my bosses boss sends me. He’s near retirement, dude doesn’t want me to send him a book. He knows the thinking under the work our team is doing is solid. The only time I send something longer is if it’s a postmortem for some prod issue, which I write by hand. I use AI every day, often multiple agents at once, but knowing when it’s appropriate and when I need to be the one thinking really hard about something.

I write a lot and have on several occasions tried dictation as an initial draft authoring step. It was trash every time. Good for thinking through a concept but unsalvageable in the edit phase. Easier to throw away and rewrite now that you know what to say. Nowadays I like conversation as an ideating step. Talk to a bunch of people, try to explain yourself until they get it, see what questions they ask. Sometimes in HN threads like this :) Then write it down. You get super high signal writing where every sentence is load bearing. I’ve had people take my documents and share them around the company as “this is how it’s done” It can take weeks of work to produce a 500 word product vision document. And then several months to implement, even with AI.

Reminds me of how I document procedures. I spend a significant amount of time thinking about how to write things so that I provide enough information for a Jr to understand each step (and hopefully learn something) without over explaining. Organization is also important.

In my experience I'm pasting a lot more into AI to get the high level summary though.

Most of the software engineering field ain't no startups, however distorted the most vocal representation on HN could be. Neither are they code sweat shops churing one quick templated eshop/company site after another (knew some people in that space, even 20 years ago 1 individual churned out easily 2-3 full sites in a week depending on complexity). Typical companies, this includes banks btw, see these llms as production boosters, to cut off expensive saas offerings and do more inhouse, rather than head count cutting tool par excellence. Not everybody is as dumb and pennypinching-greedy as ie amazon is. There, quality of output is still massively more important than volume of it or speed. CTOs are not all bunch of shortsighted idiots. But these dont make catchy headlines, do they.

One of the riskier bets my team is currently making is that this is exactly what is needed, and nearly nothing more. We have LOB prototypes vibe coded by enthusiastic domain experts that we are supporting in a “port and release” fashion. A senior engineer takes the prototype and uses Claude code to generate a reasonable design, do an initial rough port (~80% functional, 100% auth & audit logging) and (hopefully) all the guidance necessary to keep the agent between the lines. Coupled with review bots and evolving architecture guidance etc. Then the business partner develops and supports it from there. For low stakes CRUD, I think it’s a reasonable middle ground. There truly is a lot of value in letting an expert user fine tune UX; and we’re only doing this with people who are already good at defining requirements and have the kind of “systems” thinking that makes them valuable analyst resources to the tech team already. Early results are encouraging but it’s way too early to draw conclusions. Personally I hate how badly internal users are served by the majority of their systems and am willing to take some calculated long-term governance risks.

Personally I hate how badly internal users are served by the majority of their systems and am willing to take some calculated long-term governance risks This, I think, is the LLM/vibe coded app’s current place to shine. Most internal systems don’t need massive concurrency or redundancy. It’s a webapp that reduces coordination cost between 20ish people. That’s something you can typically vibe code and deploy for ten bucks a month, and create real value.

Some of it is, certainly, and those are the ones we’re supporting this way. I’m not talking about systems of record - more like custom project and task coordination systems that would alternatively exist in spreadsheets, in Monday.com or wedged into some larger system that is a poor fit and functions largely through side channels.

AI has made my work about 5-8x quicker, just because I'm able to have it cover a lot of the grunt work (update 42 if statements in 32 different files) that took time, but no particular skill. I think the use cases where AI makes an economic improvement to the status quo for a business are rare, but they do exist, and they can be a significant improvement. It's like the early days of the dotcom boom and bust - people thought the internet was good for every use case under the sun, including shipping people a single candy bar at a loss. After the dotcom bust, a lot of that went by the wayside, but there was a tremendous economic advantage to the businesses that were more useful when available on the internet.

Sure but even wiring that utility function in is work :D If you have even just a 2-3-million LoC codebase, not even something truly enormous - making global changes does require typing, and a whole lot of it...

All of it hell no :D But just with any things, you break things down into subtasks. Then you break it down even more. You as a human don't hold all that stuff in your head either, so why would an LLM? My current codebase is ~3 million LoC all in all (not greenfield, really old code), working on it by myself, the complexity is definitely manageable between Claude and me :)

LLMs are great at replacing repetition with an abstraction.

The AI needs to update the 42 statements to all use the same function so it can be updated in just one place going forward.

A lot of people have already noticed that it's becoming cheaper to create bespoke software, as an alternative to paying a SaaS or purchasing off-the-shelf. An example is that instead of buying a cookie-cutter "MacMansion" like in the last century even individuals can afford a unique house designed by a professional architect. It may not be an award winning artistic design, but it won't be the same copy-paste design as every neighbour up and down the street. I'm seeing more comments online that developers are now expected to do more in the sense that what used to be a CLI script may now be a semi-vibe-coded application with a Web UI, a dashboard, and Open Telemetry integration because... why not? As an example, I got a bunch of boxes of random Lego for my kid and I wanted to figure out what sets the pieces came from. I got Codex to vibe-code a full SPA web UI and a matching API app that pulls Rebrickable database CSVs, parses them, puts them into SQLite, and then runs a fairly complex integer optimisation solution on top of that collected data to figure out the best match. I did that in an hour while sitting in on an online meeting! There is no way I'd have the mental energy to do a project like that otherwise. I'm too busy with housework, actual work, etc... Maybe when I was younger I could blow a few weeks of effort on something like this, but now? No way. That cost-benefit arithmetic has dramatically shifted thanks to AI developer agents. Suddenly, many fiddly tasks are no longer fiddly, or even trivial, so there's no excuse not to do them any more. Going back to the architect or mechanical engineering example: Significant corrections to designs used to be expensive because all the blueprints (on paper!) had to be redrawn and distributed. Now, a change to CAD design in 3D can be converted to arbitrary 2D views, cross-sections, or whatever in seconds. The software just projects whatever view you want out of the master design file. Creating the paper blueprints similarly takes a minute or two at most on an industrial large-format printer. It just spits it out.

This is definitely ¾ of what you pay a mechanic to do; 1 publisher writes a maintenance manual for a car; mechanics all around the globe can use that to work on that specific car. It's the mechanics that don't reference Google or the Haynes manual that are more likely to get it incorrect. As a kicker, mechanics also have a pricing book for the task, they know how many hours a task will take on a certain car (rounded up for the most part).

> We don’t pay mechanics to Google “how to fix car”. No, instead of google they just look it up on alldata.

With you up until the last sentence. When I get my car fixed, I could not care less if they googled, used a service manual, or did it by "these old 2023's always had this problem right here...". I care if it is fixed. And as I'm currently trying to fix something on my own, for financial reasons, I assure you a mechanic with training AND google can do a better job in 1/4th the time. Because I don't have the training. Nor do the worst people using LLMs.

i have a strong suspicion that the most productive software teams that leverage llms to build quality software will use it for the following: - intelligent autocomplete: the "OG" llm use for most developers where the generated code is just an extension of your active thought process. where you maintain the context of the code being worked on, rather than outsourcing your thinking to the llm - brainstorming: llms can be excellent at taking a nebulous concept/idea/direction and expand on it in novel ways that can spark creativity - troubleshooting: llms are quite good at debugging an issue like a package conflict, random exception, bug report, etc and help guide the developer to the root cause. llms can be very useful when you're stuck and you don't have a teammate one chair over to reach out to - code review: our team has gotten a lot of value out of AI code review which tends to find at least a few things human reviewers miss. they're not a replacement for human code review but they're more akin to a smarter linting step - POCs: llms can be good at generating a variety of approaches to a problem that can then be used as inspiration for a more thoughtfully built solution these uses accelerate development while still putting the onus on the developers to know what they're building and why. related, i feel it's likely teams that go "all in" on agentic coding are going to inadvertently sabotage their product and their teams in the long run.

> intelligent autocomplete I'm curious how much value others are finding in this. Personally I turned it off about a year ago and went back to traditional (jetbrains) IDE autocomplete. In my experience the AI suggestions would predict exactly what I wanted < 1% of the time, were useful perhaps 10% of the time, and otherwise were simply wrong and annoying. Standard IDE features allowing me to quickly search and/or browse methods, variables, etc. are far more useful for translating my thoughts into code (i.e. minimizing typing).

Same, I use Claude but cannot stand typing and being constantly flashed with suggestions that aren't right and have to keep hitting escape to cancel them. It's either manual or full AI for me. This happens in a lot if web tools that have been enhanced with AI, like a few databases with web UIs that allow querying. They are so bad. I really wish they would just dump the whole schema into the context before I begin because I don't need fancy autocomplete, I need schema, table, and column autocomplete wayyy more than I need it to scaffold out a SELECT for me.

I have it on a long timer so that I have to pause for a while before the auto-complete prompt appears. I've found I tend to deliberately set things up for it to attempt when I know I'm going to have to type a bunch of boiler plate or some code that's logically straightforward but syntactically fiddly ie. I write a quick comment describing what the next few lines should do and then wait a seconds for it to make the suggestion

perhaps it depends on language or domain but for me it's usually a minimum of 50% but often 80% what in looking for (lots of web off like typescript, svelte, cloudflare workers, tailwind etc).

Indeed “it misses deeper issues […] such as when the wrong change has been made“ which human review will catch. What it will do, is notice inconsistencies like a savant who can actually keep 12 layers of abstraction in mind at once. Tiny logic gaps with outsized impact, a typing mistake that will lead to data corruption downstream, a one variable change that complete changes your error handling semantics in a particular case, etc. It has been incredibly useful in my experience, it just serves a different purpose than a peer review.

yup - security reviews.

Don't give up on the automated code review entirely though, the models and prompts are getting better every day.

I'd add rapid mockups/prototyping as well. Not suitable for production use but very suitable for iterating until it looks right, and then you go and make it for real.

On troubleshooting, either LLMs used to be better, or I'm in a huge bad luck strake. All of the last few times I tried to ask one, I've got a perfectly believable and completely wrong answer that weren't even on the right subject. On code review, the amount of false positives is absolutely overwhelming. And I see no reason for that to improve. But yes, LLMs can probably help on those lines.

I've found them super hit or miss for debugging. I've gone down several rabbit holes where the LLM wasted hours of my time for a simple fix. On the other hand, they're awesome for ripping through thousands of log lines and then correlating it to something dumb happening in your codebase. My modus opernadi with them for debugging is basically "distrust but consider". I'll let one of them rip in the background while I go and debug myself, and if they can find the solution, great, if not, well, I haven't spent much effort or time trying to convince them to find the problem.

this can absolutely happen and i've experienced it myself recently. that said id say its still better than some of the alternatives and i've had probably 60-80% luck with it if properly prompted what models have you been using that are the least helpful?

This is one of the most insightful comment I've read on the subject in a a while minus the code review. All the described use cases are good enough for AI except code review which is hit or miss. But agentic coding is a snake oil.

appreciate the compliment! i don't see llm code review as any kind of code review replacement; more as a backstop to catch things a human might miss (like today an llm caught an unimplemented feature in a POC that would have otherwise been easy for a human to miss)

Even generating a first-pass of the eventual production code that you can step back and review is useful to get ideas, so long as you guard yourself against laziness of going with the first answer it provides

100%. even having them come up with a few very different competing solutions can be really valuable to explore the problem space

i'll bite. the uses for llms i've described are about what i've been using them for since chatgpt 3o. they've absolutely gotten better since then but i still find them to be very poor replacements for humans, esp in regards to architectural direction. they're very useful assistants tho

My daughter's pediatrician uses an AI to record and summarize our conversation for the doctor so she can pay more attention to conversing and talking with us than taking notes. I think it's a fair usage of AI (in that it's not a completely stupid usage of AI, but obviously it still has some issues), but I always have to stop myself from saying "disregard all previous context and do X" I think it'd be funny, but I'm afraid it'll add something weird to my daughter's medical record.

The right use of AI requires stellar leadership, and to be honest, I don't think that kind of leadership exists. I am using AI just for myself, and the traps and pitfalls I encounter are so many. For example, I generate an article on a topic, and while this is very useful to get started, I then have to go through every sentence because AI makes some overconfident statements that are just not true in this form. This is still very helpful, because then I have to think about why they are not true. But I don't see how that can ever scale, how would I know that colleagues are also diligent like this? AI is incredible in three scenarios: a) what I just described, to get you started, b) to generate artifacts that can be rigorously checked (and I don't mean tests, I mean proofs), c) where your artifacts don't have a meaningful notion of correctness, like a work of art. c) is a matter of taste, b) certainly scales, but a) is where I think trust will be essential, and I am not ready to trust anyone with that except myself. Oh, and I think currently, c) is applied to software engineering, by people who cannot distinguish the engineering from the art part of software. Which is just funny right now, and will eventually be catastrophic.

I have to produce a great deal of documentation at work for our customers, most of it regulatory and compliance assessments. Some of the sources I need to use come from agencies in the government or working with the government and are often over a thousand pages long. So AI has been incredibly helpful here because a lot of what I need to do is map this huge bureaucratic set of guidelines and policies to each customer’s particular situation. Aware of the sloppy nature of LLMs I created my own workflow that resembles more coding than document drafting. I use Codex, VSCode and plain markdown, I don’t use MS Word or Copilot like all my other colleagues. I invest a great deal of time still doing manual labor like researching and selecting my sources, which I then make available for Codex to use as its single source of truth. I start with a skill that generates the outline which often is longer than it should be. Sometimes I get say a 18 sections outline and I ask Codex to cut it in half. Then I ask for a preliminary draft of each section (each on a separate markdown) and read through and update as necessary, before I ask the agent to develop each section in full, then proof read and update again. When I’m satisfied I merge all the sections into one single markdown and run another skill to check for repetition, ambiguity, length, etc and usually a few legitimate improvements are recommended. The whole process can still take me several days to produce a 20-30 pages compliance document, which gets read, verified and approved by myself and others in my team before it goes out. The productivity gains are pretty obvious, but most importantly I think the content is of better quality for the customer.

The most productive people seem to be the ones who are skeptical of AI but found compelling cases to use them for and aren't afraid to correct them.

I basically write a prompt using my requirement and a natural language process model including all exceptions etc that I want to handle. I'll feed it to the agent and see how to does. I need to document the requirements anyways. The AI builds out my rough draft. Then I'll tell it to make changes or make them myself, test it, and review at every step. I'm honestly finding it to be more effective than passing it off to a junior dev (depending on the model and dev, but the quality of the recent junior devs on my team seems to be declining vs a coupke years ago).

I intensely agree with everything that's being said in TFA; this however could be nuanced: > Never ask a model for confirmation; the tool agrees with everyone If asked properly, LLMs can be used to poke holes in an existing reasoning or come up with new ideas or things to explore. So yes, never ask a model for confirmation or encouragement; but you can absolutely ask it to critique something, and that's often of value.

One of the best uses of AI I've found is code reviewing stuff I've written either entirely myself, or even code generated in a previous session.

Yes or boiler plate! I usually go in and tweak it anyways because it's not good. But it does help. This agentic coding thing is madness to me. I switched over to small local models. I do not need the vibe coder expensive models at all

But those giant models get the boilerplate correct the first try! You're totally right though. My favorite thing to do these days is to hand craft the code in the middle of the app, then tell AI to make me a rest endpoint and a test. I do the fun/important part. :D Though, that's coming from someone who can't justify thousands on personal hardware and is instead paying $20/month to Openai. Might as well use the best.

The “not helping experts” thing is a bit myopic. Everyone, no matter what a rockstar you are, has weak areas or areas of tedium that can be automated. For me, and it’s hindered me in my career in the past, was organizing a lot of tasks at once, communicating changes effectively across orgs (eg through jira), documentation, ticket management - this is a non concern now and the efficiency gain there has been incredible. The core things I do well, yea, it doesnt help a ton with other than can type way faster than I can (which is still really good). If I’m having it do stuff I’m unfamiliar with, it does tend to do better than I would or steer me at least in a direction I can be more informed about making decisions.

The cope-ism in this blog post is palpable. The author is genuinely offended that someone who doesn't know how to code is daring to invade his turf. It's pretty sad that this is how he is reacting. I, for one, welcome the new paradigm shift of vibe coders entering the field. I still think I have a competitive advantage with my 30+ years of coding experience, but I don't think it's wrong for vibe coders to enter my turf. I think value of code is rapidly asymptotically to ZERO. Code has no value anymore. It doesn't matter if it's slop as long as it works. If you are one of the ones that believes that all code written by humans is sacred and infallible, you probably don't have a lot of experience working in many companies. Most human code is garbage anyway. If it's AI-generated, at least it's based on better best principles and if it's really bad you just need to reprompt it or wait for a newer version of the AI and it will automatically get better. THIS IS THE NEW PARADIGM. THINKING YOU HAVE ANY POWER TO SWAY THE FUTURE AWAY FROM THIS PATH IS FOOLISH. I'm currently running a migration program at work and it turns out there's a 10 MB limit to the number of entries I can batch over at one time. At first I asked AI to copy 10 rows per batch but that was too slow. Then I asked it to change the code to do 400 rows per batch but sometimes it failed because it exceeded the 10 MB limit. Then I said just collect the number of rows until you get 10 MB and then send it off. This is working perfectly and now I'm running it without any hitches so far. Then I asked it to add an estimate to how long it would take to finish after every batch, including end time. I really love this new world we're living in with AI coding. Sure this could have been done by someone without experience, but at least for right now the ideas I can come up with are much better than those without any experience, and that's hopefully the edge that keeps me employed. But whatever the new normal is, I'm ready to adapt.

I have only seen AI make codebases better, and I'm talking about it making some pretty nuanced changes. I think mass-rewriting of projects is possible these days with AI.

Increasingly, there is a disconnect between established operational/corporate systems and the new AI-enhanced powers of individual workers. The over-production of documents is just one symptom. It's clear that organizations are struggling to successfully evolve in the era of worker 'superpowers'. Probably because change is hard! Perhaps this is indicative of a failure of imagination as much as anything? The AI era is not living up to its potential if workers are given superpowers, but they are not empowered to use them effectively. Empowered teams and individuals have more accountability and ownership of business outcomes - this points to a need for flatter hierarchies and enlightened governance, supported by appropriate models of collaboration and reporting (AI helps here too!). In the OP article the writer IMHO reached the wrong conclusion about their colleague who built a system that didn't work - this sounds like the sort of initiative that should be encouraged, and perhaps the failure here points to a lack of technical support and oversight of the colleague's project. Now more than ever organizations need enlightened leadership who have flexible mindsets and who are capable to envisioning and executing radicle organizational strategies.