Summarizer

It's going to depend on the type of team and environment you work in. Probably on how senior you are as well. If your boss asks you for specific documents and expects a quick turnaround, and you regularly take 3 weeks or whatever to produce them, then yeah probably. If your boss generally leaves you alone to find and solve problems on your own, then probably not.

Reminds me of how I document procedures. I spend a significant amount of time thinking about how to write things so that I provide enough information for a Jr to understand each step (and hopefully learn something) without over explaining. Organization is also important.

I watched a video of some (unemployed) programmer lamenting over the current job situation market. He had been coding for a good while, but had recently been laid off. The vid was mainly concerning the searching and interview process, but it also did highlight something I find somewhat true and important: Right now we're in a gold rush. Companies, that be established ones or startups, are in a frenzy to transform or launch AI-first products. You are not rewarded for building extremely robust and fast systems - the goal right now is to essentially build ETL and data piping systems as fast as humanly (or inhumanly) possible, and being able to add as many features as possible. The quality of the software is of less importance. And, yes, senior engineers with other priorities are being overshadowed - even left in the dust - if they don't use tools to enhance their speed. As the article states, there are novice coders, even non-coders that are pushing out features like you wouldn't believe it. As long as these yield the right output, and don't crash the systems, that's a gold star. Of course there are still many companies whose products do not fall under that, and very much rely on robust engineering - but at least in the startup space there's overwhelmingly many whose product is to gather data (external, internal), add agents, and do some action for the client. You need extremely competent, and critically thinking technical leaders on the top to tackle this problem. But we're also in the age where people with somewhat limited technical experience are becoming CTOs or highly-ranked technical workers in an org, for no other reason than that they know how to use modern AI systems, and likely have a recent history of being extremely productive.

If you are 8x quicker by having the AI do these for you, I think you are a junior intern or something? It must mean most of your time is spent doing these things.

It does have real benefits, but also, of course, all of the downsides you mentioned. The best analogy is the outsourcing / offshoring fad of the last decade. Managers hated that senior developers were getting highly compensated (often higher than the management class!) and pounced on every opportunity to replace expensive people with (much!) cheaper options, quality be damned. For the few companies that paid attention to the quality, this worked out swimmingly. Apple is probably the best example, they've outsourced almost all of their manufacturing to China and other similar countries. So yes, my mental picture is that every manager is drooling right now because they think they can replace someone getting paid six figures with an AI that costs six dollars a day, if that. A virtual employee that doesn't talk back, doesn't argue, doesn't question, doesn't go off on "unproductive tangents" like refactoring (whatever that's even supposed to mean), and just pumps out code 24/7 like a good little slav... employee. The very rare smart managers out there are looking at this more like the transition that happened to architect firms when CAD became available. They used to have a dozen draftsmen for every architect . Now there are virtually none, I haven't even heard that job title being used in decades! We still have architects, and if anything, they're paid even more.

Yeah Bentley (and in some cases Haynes) make good aftermarket manuals too. And you can find good information on some forums. But you can also find a lot of bad information. Reliably sifting the good from bad only comes with experience--much like in software.

> I think for a lot of companies, AI is a destabilizing force that their managerial structure is unable to compensate for. From the article: > because the competence the work reflects is not the novice’s competence at all The core of the problem is that AI allows engineers who were previously inexperienced or downright mediocre, pretend that they are talented, and a lot of management isn’t equipped to evaluate that. It’s like tourists looking at a grocery store in North Korea from their tour bus. It looks like a fully functioning grocery store from the outside, but it is mostly cutouts and plastic fruit.

Have you not seen the principals and seniors being offered the door or buyouts?

Did this recently to a junior engineer myself, who sent me an AI slop chart in response to simple questions about what he thought about my senior direction about vercel-shipping something fast over AWS-architecting something over thought and over engineered. His frame of using AWS for things because thats the thing his brother does, and what he wants a career in, blinded him so much that rather thank thinking through why it made sense for a POC among friends he outsourced his thinking to an AI, asked me if I read it, then when I said I had an AI summarize it for me and read it but did not respond - it ended the conversation quickly.

I've noticed early into AI adoption in the workplace that some colleagues took advantage of the technology by appearing to be hyper-proactive; New TODs weekly, fresh new refactoring ideas, novel ways to solve age-old problems with shiny new algorithms. Fast-forward to today, and this is occurring two-fold. Not only are they trying to appear more proactive, combining this with the fear of AI layoffs, they're creating solutions to problems before the problem has even been fully defined. For example, I was tasked to look into a company-wide solution for a particular architectural problem. I thought delivering a sound solution would give me some kudos, alas, I wasn't fast enough. An intern had already figured it out and wrote a TOD. I find myself too tired to compete.

I basically write a prompt using my requirement and a natural language process model including all exceptions etc that I want to handle. I'll feed it to the agent and see how to does. I need to document the requirements anyways. The AI builds out my rough draft. Then I'll tell it to make changes or make them myself, test it, and review at every step. I'm honestly finding it to be more effective than passing it off to a junior dev (depending on the model and dev, but the quality of the recent junior devs on my team seems to be declining vs a coupke years ago).

While I agree with some of these observations - the research cited in the article really do not match the claims at all from what I can tell. > An NBER study of support agents [2] found generative AI boosted novice productivity by about a third while barely helping experts. Harvard Business School researchers found the same pattern in consulting work [3]. The first work cited was a research study on GPT-3(!) from 2020. Which is a barely coherent model relative to today's SOTA. The second HBS research study literally finds the opposite of what's claimed: > we observed performance enhancements in the experimental task for both groups when leveraging GPT-4. Note that the top-half-skill performers also received a significant boost, although not as much as the bottom-half-skill performers. Where bottom-half skilled participants with AI outperformed top-half skilled participants without AI. (And top-half skilled participants gained another 11% improvement when pared with AI). Again, GPT-4 model intelligence (3 years ago) is a far cry from frontier models today.

The “not helping experts” thing is a bit myopic. Everyone, no matter what a rockstar you are, has weak areas or areas of tedium that can be automated. For me, and it’s hindered me in my career in the past, was organizing a lot of tasks at once, communicating changes effectively across orgs (eg through jira), documentation, ticket management - this is a non concern now and the efficiency gain there has been incredible. The core things I do well, yea, it doesnt help a ton with other than can type way faster than I can (which is still really good). If I’m having it do stuff I’m unfamiliar with, it does tend to do better than I would or steer me at least in a direction I can be more informed about making decisions.

That's a very good revert on horse-riding analogy. But you might still be making an assumption that the horse package doesn't come with a weapon. It might boil down to saying "AI can not achieve the skills of a senior engineer" - which might not have a strong basis.

i too find lots of value in llms but your example describes a scenario a programmer could have also easily solved and maybe even had writing it correctly in the first or second shot. that isn't to say an llm can't be useful but your post implies it's inevitable that llms will replace humans entirely from writing code, which i think is incredibly optimistic at best. that said we will see!

I think it's interesting that the data suggests that novices can increase productivity by a third and experts not at all. That sounds very similar to Dunning-Kruger- the novices literally don't know what productivity looks like. I'm finding it difficult to agree on document creation now being zero cost whereas consumption is high cost. I think you can actually spend time giving AI enough context to consume docs for you. I think the other thing worth pointing out with the article is understanding what your company will recognise. Yes, it's totally correct that your company won't thank you for poopoo-ing the idiot with AI. Yes, they'll run into a buzz saw when they hit a stakeholder who can choose to buy in. Don't burn your career protecting theirs. In fact it's not even certain that the idiot is damaging their career (for many reasons). This was a really interesting article.

So essentially, AI is exacerbating the Dunning-Kruger effect in society.

Excellent article! Aptly describes what I have been feeling and thinking about the claims many AI optimists make. --- > He produced a great deal of code, [...] He could not, when asked, explain how any of it actually worked. [...] When opinions were voiced even as high as a V.P., he fought back. AI has democratized coding, but people have yet to understand that it takes expertise to actually design a system that can handle scale. Of course, you can build a PoC in a few hours with Claude code, but that wouldn't generate value. The reason why we see such examples in the workplace is because of the false marketing done by CEOs and wrapper companies. It just gives people a false hope that "they can just build things" when they can only build demos. Another reason is that the incentives in almost every company have shifted to favour a person using AI. It's like the companies are purposefully forcing us to use AI, to show demand for AI, so that they can get a green signal to build more data centers. --- > So you have overconfident, novices able to improve their individual productivity in an area of expertise they are unable to review for correctness. What could go wrong? This is one much-needed point to raise. I have many people around me saying that people my age are using AI to get 10x or 100x better at doing stuff. How are you evaluating them to check if the person actually improved that much? I have experienced this excessively on twitter since last few months. It is like a cult. Someone with a good following builds something with AI, and people go mad and perceive that person as some kind of god. I clearly don't understand that. Just as an example, after Karpathy open-sourced autoresearch, you might have seen a variety of different flavors that employ the same idea across various domains, but I think a Meta researcher pointed out that it is a type of search method, just like Optuna does with hyperparameter searching. Basically, people should think from first principles. But the current state of tech Twitter is pathetic; any lame idea + genAI gets viral, without even the slightest thought of whether genAI actually helps solve the problem or improve the existing solution. (Side note: I saw a blog from someone from a top USA uni writing about OpenClaw x AutoResearch, I was like WTF?! - because as we all know, OpenClaw was just a hype that aged like milk) --- > The slowness was not a tax on the real work; the slowness was the real work. Well Said! People should understand that learning things takes time, building things takes time, and understanding things deeply takes time. Someone building a web app using AI in 10 mins is not ahead but behind the person who is actually going one or two levels of abstractions deeper to understand how HTML/JS/Next.js works. I strongly believe that the tech industry will realise this sooner or later that AI doesn't make people learn faster, it just speeds up the repetitive manual tasks. And people should use the AI in that regard only. The (real) cognitive task to actually learn is still in the hands of humans, and it is slow, which is not a bottleneck, but that's just how we humans are, and it should be respected.