llm/3a862c31-848e-4e32-be93-99402d2b43b6/topic-9-9096f938-c7fd-4140-b1dd-64d19d7a297e-input.json
You are a comment summarizer. Given a topic and a list of comments tagged with that topic, write a single paragraph summarizing the key points and perspectives expressed in the comments. TOPIC: Abstraction and Complexity Management COMMENTS: 1. This was well talked about in Hyrums Law, which came from a Googler as well. https://www.hyrumslaw.com/ > With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody. 2. If upvoting doesn’t require justification neither should downvoting. But let me try to express why people disagree. Change is software compared to physical systems is comparatively incredibly cheap. Unlike in building something known, design at the start of a software project is unlikely to be the one the client actually wanted nor would be the one that is one going to be build. Or at least it shouldn’t be. The “brick-laying” part of software isn’t the hard part. Depending on want to analogise as “brick-laying” in software, that part could automated. Push to main and the deployment pipeline runs tests, makes sure things are working and voila! You have a new “house”. If its ugly or falls apart in software, easy , just revert to the previous version and its like nothing happened. Client wants a try different layout, it can be done affordably. Most of the time in software engineering you don’t know exactly how to do something, there is always a degree of discovery, experimentation and 3. I first learned about the "innovation tokens" idea in "Novelty is a loan you repay in outages, hiring, and cognitive overhead" from this, still one of my favorite essays on software architecture: https://boringtechnology.club/ Likewise, "Abstractions don’t remove complexity. They move it to the day you’re on call." made me think of this 23 year old classic from Joel Spolsky, the Law of Leaky Abstractions: https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a... 4. Nothing can remove complexity other than simplifying requirements. It can only be shuffled around and distributed to other areas of the system (or library, or vendor functionality etc) 5. I think this is true for essential complexity. And indeed it's one of the best reasons to release early and often, because usage helps clarify which parts of the requirements are truly required. But plenty of projects add quite a lot of incidental complexity, especially with technology choices. E.g., Resume Driven Development encourages picking impressive or novel tools, when something much simpler would do. Another big source of unneeded complexity is code for possibilities that never come to fruition, or that are essentially historical. Sometimes that about requirements, but often it's about addressing engineer anxiety. 6. You absolutely can remove unnecessary complexity. If your app makes an http request for every result row in a search, you'll simplify by getting them all in one shot. Learn what's happening a level or two lower, look carefully, and you'll find VAST unnecessary complexity in most modern software. 7. If - to take a convenient example - I use a library sorting function instead of writing my own sorting code, it's true that I haven't removed the complexity of the work my program is doing: It sorts. But I have arguably reduced the complexity of my code. Similarly, if I factor out some well-named function instead of repeating the same sequence actions in multiple places - the work to be done is just as complex, and I haven't even removed the complexity from my code, but - I have traded the complexity of N different pieces of code for 1 such piece plus N function calls. Granted, that tradeoff isn't always the right thing to do, but one could still claim that, often, that _does_ reduce the complexity of the code. 8. I think most people don't really claim, that complexity is gone when properly abstracted, but claim that you don't have to deal with it every single time. That's the purpose of abstracting something. Simple example: You are not dealing with the complexity of process management of the OS, every time you start any application. Sometimes you might need to, if you are developing software. Or if your application hangs and you need to kill it via some task manager. Most users however, never deal with that, because it is abstracted "away". That's the whole point. Nevertheless, the actual complex work is always done. Behind the scenes. 9. I don't follow. Following the robustness principle doesn't necessarily introduce novelty. Perhaps a bit more complexity, but just how much depends on how clever you try to be. What did you mean? 10. Like most of the things Spolsky says in that article it’s pretty dubious. Following it to its logical conclusion, presumably on-call debugging work be even easier if the software had been handwritten in assembler. 11. > 4. Clarity is seniority. Cleverness is overhead. Clarity is likely the most important aspect of making maintainable, extendable code. Of course, it’s easy to say that, it’s harder to explain what it looks like in practice. I wrote a book that attempts to teach how to write clear code: https://elementsofcode.io > 11. Abstractions don’t remove complexity. They move it to the day you’re on call. This is true for bad abstractions. > The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise. (Dijkstra) If you think about abstraction in those terms, the utility becomes apparent. We abstract CPU instructions into programming languages so we can think about our problems in more precise terms, such as data structures and functions. It is obviously useful to build abstractions to create even higher levels of precision on top of the language itself. The problem isn’t abstraction, it is clarity of purpose. Too often we create 12. I agree with you re: abstraction - one of the author's only points where I didn't totally agree. But also worth noting that whenever you make an abstraction you run the risk that it's NOT going to turn out increase clarity and precision, either due to human limitation or due to changes in the problem. The author's caution is warranted because in practice this happens really a lot. I would rather work with code that has insufficient abstraction than inappropriate abstraction. 13. Broad strokes: absolutely. The practical reality gets tricky, though. All programming abstractions are imperfect in some regard, so the question becomes what level of imperfection can you tolerate, and is the benefit worth the cost? I think a lot of becoming a good programmer is about developing the instincts around when it’s worth it and in what direction. To add to the complexity, there is a meta dimension of how much time you should spend trying to figure it out vs just implement something and correct it later. As an aside, I’m really curious to see how much coding agents shift this balance. 14. But that is still better than nothing at all, which is the point. The people you want (or want to be) are the engineers who are smart and experienced enough to get a first draft down that is pretty much right without a long drawn out process of figuring out the best way to do X, Y and Z with all the lengthy ADRs, discussions, debates, POCs, revisions etc. over and over again. That may be necessary if you don't have people in the room who know what they're doing and have the intuition through deep experience to choose good tools, patterns and abstractions at the start. Begin closer to the target, rather than far away and iterate to it. 15. > Abstractions don’t remove complexity. They move it to the day you’re on call. As someone who has been on call a lot, this is only true for bad or incomplete abstractions. When you are on call (or developing) you can't possibly know everything about the system. You need abstractions to make sense of what is going on, how the system as a whole works, and know which parts to hone in on when things go wrong. And it is extremely useful to have standard ways of changing configuration for things like timeouts, buffer sizes, etc. in a central place. 16. I don't think it's meant to be a point against abstraction or a point against complexity. I think it's widely understood that abstraction is part of how advancement is made in our practice, as well as in other disciplines. I have taken this saying to be an observation that there is almost always possible failure beneath the façade provided by the abstraction. Therefore, yes, you avoid having to let that complexity enter your brain, but only when the abstraction is holding. Beyond that point, often after pages are sent, you will still have to engage with the underlying complexity. A proactive measure following from this idea would be to provide support in or alongside your abstractions for situations where one must look under the bonnet. 17. To offer some additional causes for the degradation of UX: 1. An increasing part of industry profits started coming from entertainment (or worse, psychological exploitation) instead of selling the customer a useful tool . For example, good budgeting-software has to help the user understand and model and achieve a goal, while a "good" slot-machine may benefit from confusion and distraction and a giant pull-handle. 2. "Must work on a touchscreen that fits in a pocket" support drags certain things to a lowest common denominator. 3. UX as a switching-cost for customers has started happening more on a per-product rather than a per-OS basis. Instead of learning the Windows or Mac "way" of screens and shortcuts, individual programs--especially those dang Electron apps--make their own reinventions of the wheel. 18. For the all the necessary complexity and race-to-the-bottom features, I am a fan of Jetbrains. I like using Uber, Twitch (wrote a plugin for it one weekend to integrate with chrome), Netflix, Discord. There are plenty of companies that manage to be enjoyable to end users and expose apis without the inscrutable abstractions and terminology I encounter using google products. It feels the same as working with Oracle. 19. > very single one of their services is a pain to use Uhm, no? Google Cloud Platform is way more convenient to use than AWS, the IAM is way better designed, and documentation is leagues ahead of AWS. 20. Every engineer should read this. It's a wonderful collection of heuristics that might seem banal, but which are shimmeringly true. The two that stand out are > Novelty is a loan you repay in outages, hiring, and cognitive overhead. and > Abstractions don’t remove complexity. They move it to the day you’re on call. as a warning against about being too, too clever. 21. Google still suffers the most from not understanding those two. Probably more than other companies. 22. > The engineer who truly understands the problem often finds that the elegant solution is simpler than anyone expected. > The engineer who starts with a solution tends to build complexity in search of a justification. I do agree this is a good point, I just find it funny that it comes from "staying 14 years at Google". This is literally the reason why I left Google first, and Meta second. Finding simple solutions will get you absolutely nowhere in a place like those. You have to find complex solutions with a lot of stakeholders, alignment, discussions, escalations... Why ship one button if you can ship 100 and get you, your team and your manager promoted in the process? 23. > Before you build, exhaust the question: “What would happen if we just… didn’t?” Well said! So many times I have seen great products slide down. If they just froze the features and UI, and just fixed performance, compatibility and stability issues for years, things would be better. (this applies to any company). Many programs I use are years old. They are great programs and don't need constant change! Updates can only make it worse at that point (minus critical security issues, compatbility, performance regressions) 24. Lesson 11 (Abstractions move complexity) and Lesson 20 (Time > Money) are two sides of the same coin. In engineering, we talk about "leaky abstractions" in our code. But the biggest leaky abstraction is often our own health. We treat our bodies as a "boring dependency" that will always work, but burnout and RSI (Repetitive Strain Injury) are essentially the ultimate system outages. Just as "Novelty is a loan" (Lesson 5), neglecting your physical "hardware" early in your career is a high-interest loan that you end up repaying in your 40s. Real seniority isn't just about navigating people and politics—it's about managing your personal energy so you actually have the health to enjoy the "compounding" (Lesson 21) that comes at the end. 25. I agree to both 3) and 8) but I find it a dilemma that if you don't get it perfect the first time, you will waste thousands of man-hours for everyone to upgrade even though it only took you 10 minutes to release the new version. 26. > Abstractions don’t remove complexity. They move it to the day you’re on call. Gold. 27. The biggest one that resonates with me is that cleverness is overhead. My #1 issue with mid level engineers is that they like complexity and find complexity fun and interesting. An experienced engineer knows that complexity is irritating and frustrating and that a simple solution is harder and superior. A solution that simultaneously solves the problem and reduces complexity is almost the definition of genius. If you are good you will do this a few times in your whole career. 28. Yeah, "resume driven development" is a second major force pushing complexity that I didn't mention. People want to be able to get experience with as many buzzwords and technologies and stacks as they can for obvious personal self interest reasons. The incentive is real. A great programmer who does a great job simplifying and building elegant maintainable systems might not get hired because they can't say they have X years experience with a laundry list of things. After all, part of their excellence was in making those things unnecessary. It's a great example of a perverse incentive that's incredibly hard to eliminate. The net effect across the industry is to cost everyone money and time and frustration, not to mention the opportunity cost of what might have been had the cognitive cycles spent wrangling complexity been spent on polish, UI/UX, or innovation. There's also a business and VC level version of this. Every bit of complexity represents a potential niche for a product, servic 29. Again, very well put. It often becomes a chain-reaction as well. 30. > Abstractions don’t remove complexity. They move it to the day you’re on call. Then they are bad abstractions. I get where he is coming from, but the entire field is built on abstractions that allow you to translate say a matmul to shuffling some electrons without you doing the shuffling. 31. >Abstractions don’t remove complexity. They move it to the day you’re on call. Damn that's a real one. Nothing like struggling through a bunch of indirection to figure out what the heck a clever abstraction was supposed to do 32. This has to be the 50th or 100th version of this article that repeats the same thing Every single point in this article was already explicitly described between roughly 1968 and 1987: Brooks formalized coordination cost and the fallacy of adding manpower in The Mythical Man-Month Conway showed that system architecture inevitably mirrors organizational communication structure in 1968 Parnas defined information hiding and modularity as organizational constraints, not coding style, in 1972 Dijkstra *repeatedly warned* that complexity grows faster than human comprehension and cannot be managed socially after the fact None of this is new, reframed, or extended here; it is a faithful re-enumeration of half-century-old constraints. These lists keep reappearing because we refuse to solve is the structural one: none of these constraints are enforceable inside modern incentive systems. So almost like clockwork somebody comes out of nowhere saying hey I’ve I’ve observed these things that a 33. The fact that people don’t learn from the older books is somewhat annoying, but rewriting them makes sense precisely because people will likely trust it more. Software engineers are prone to novelty bias. Thats in contrast to some other demographic groups who very much prefer ancient texts. 34. Some people think clarity means abandoning language idioms and writing simple code that a first year computer science student could understand and follow. If you do this, your team will write verbose, repetitive code, and put more emphasis on procedures instead of data structures and how they change over time. Use the language features to write powerful concise code that really takes some skill and expertise in the language to understand. Force your team to become more skilled, don’t stoop down to the lowest common denominator. In time, this code will become as easily understood as any other simple program. And when shit breaks down at 2 AM, you do nothing, because your code is clever enough to handle problems itself. But don’t obfuscate. Write a concise, engaging paragraph (3-5 sentences) that captures the main ideas, notable perspectives, and overall sentiment of these comments regarding the topic. Focus on the most interesting and representative viewpoints. Do not use bullet points or lists - write flowing prose.
Abstraction and Complexity Management
34