Summarizer

Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years. It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all the dry powder out of the private markets. Folks are now starting to ask difficult questions about their burn rate and revenue. It is increasingly looking like they might not commit to the purchase order they made which kick-started this whole panic over RAM. Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?

Eh I feel like the memory bus width thing was more a case of binning memory controllers and the like. Designing a part with a wide bus and putting the traces down on the board is what I would expect to be the easy part these days (surely). But yield, yield comes for us all.

And the bottleneck at the time was HBM interposers, not actual ram dies.

There are a lot of cyclical businesses that make money every year. It requires careful management. Factories can produce less than full capacity - but you better design for that. you can make money in the worst years without laying anyone off even - but it requires careful attention to details and not over hiring in good times as if they will never end.

Factories working at (significantly) less than full capacity gets a bit harder when you've got one of the most expensive machines on earth working in them, and production lines that'll be out of date in a couple of years

the normal way to do that is by hiring/firing to meet demand, but in the fab business, you have 10s of billions of dollars of capex with relatively little opex. if you're running at <90% capacity, you're losing money.

That is the common way, but there are companies that manage without hiring/firing. (or they hire temp workers). There is a minimum level of capacity you need to run just to keep the lights on, and figuring out how to get that low without impacting your ability to serve the highs is hard. Memory manufactures have not gotten very low, probably for good reasons, but it is something they should work on.

All of the capital intensive businesses face this issue. Chemicals, Shipping, Semiconductors etc. You get market signals that the demand is there, you acquire the necessary capital, you spend 5 years to build capacity, but guess what, 5 other market players did the same thing. So now you are doomed, because the market is flooded and you have low cash flow since you need to drop prices to compete for pennies. Now you cannot find capital, you don't invest, but guess what, neither your competitors did. So now the demand is higher than the supply. Your price per unit sold skyrocketed, but you don't have enough capacity! Rinse and repeat. Capitalists claim that this is optimal.

Forecasting demand 5 years into the future is intrinsically highly unreliable. It doesn’t matter if it is capitalism or a command economy. The bet is always going to be risky and someone will have to pay for that risk. At least with capitalism you have many different people with different perspectives on the risk making independent bets. That mitigates the more extreme negative outcomes.

To add to that, investors who do make the bet get punished for over-building, which is better than tax payers paying for it. And before someone says it, big corps do get bailed out by gov't, but that's definitely goes against capitalist ideas.

> Capitalists claim that this is optimal. Because that does not happen exactly as you say for all players. The demand signals will be processed and long-term risk is balanced against short-term gain in a distributed fashion, so not everyone will do the same.

> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag? The memory makers specifically did not scale up capacity to avoid being left holding the bag.

I am betting the pendulum swings faster to the other side to excess capacity as all the construction lies of Altman fall through with financiers waking up the the fact they can't build the infrasctructure as fast nor make any profits on that infrastructure that will get built.

I think I’m missing something. Financially, what bag would the memory makers be holding here? I don’t think I’m well informed regarding how these deals were structured.

Memory makers make capital investements (build different factories, convert physical production lines, etc.) to meet orders that have been place for the next ~5 years. OpenAI (or whoever) crashes and can't pay for the order leaving the memory makers in a tough spot.

If they don't expand capacity much, the only negative consequences I foresee happening for them is that they might lose spending discipline, and that systems will be set up to make do with a little less memory. Apart from that, it's just very high profits followed by more or less regular profits.

Surely this can be solved with financial engineering. The memory makers build more capacity, but they finance it with something like floating-rate notes linked to an index of memory prices, or even catastrophe bonds or AT1s. Or more crudely, set up special purpose vehicles to build the extra capacity, and issue convertible bonds from those; if the memory market collapses, investors don't get paid, but they do get a memory factory.

not all DRAM capacity can switch to HBM quickly. That lag is where volatility comes from

Do the memory makers not have a contract in place for an order this large? I assume that they aren't going to take "trust us bro" as good enough for several million dollars in orders, and even if there is a way to cancel the order it won't be free. I would assume so at least, but i would like if anyone knew for certain.

* several Billion dollars in orders.

It's not going to last until 2028, it'll last until 'min(AI_bubble_burst, 2028)', which I expect will be a lot smaller than just '2028'. So the real question is, how long will it take to retool for non-HBM, and will there be a fire sale as they scramble to recover? Which also explains why production is falling behind demand, companies aren't going to sink billions into creating product for a market that could dry up overnight.

They won't be, prices are high because they are refusing to build capacity for demand that may evaporated by the time they are done. They are holding back and building only enough so when the bubble pops they will be fine.

You can't build capacity overnight, and even with that in mind, it's hard to say if it is sensible to increase capacity now that we are in an AI bubble. For all we know, the bubble might burst.

its not like all the RAM is passing the same machine, they can gradually increase machines and observe the change in demand, and smoothly match it.

If they gradually increase production capacity then prices stay high for 10+ years (or for as long as it takes for demand to crash) because a gradual increase in production takes that long for them to add enough capacity for current demand. If they add enough capacity to meet current demand quickly then if demand crashes they still have billions of dollars in loans used to build capacity for demand that no longer exists and then they go bankrupt. The biggest problem is predicting future demand, because it often declines quickly rather than gradually.

Machines take up space in buildings (factories); both of which are discrete rather than continuous functions. If your factory is already full of memory-making machines, and want to add one more, it will cost you billions and many months to build another factory. If you suppose you have cracked the smooth-ramping problem, perhaps you should throw your hat in the ring and soak up all the pent-up demand that SK Hynix, Samsung and Micron are neglecting.

Think of the factory problem from physics first principals instead, as Elon would say. Musk says he will outcompete earth fabs by building them on the moon in just a few years, deploy radiation harden versions of the chips into space, and beat out TCO vs doing this on earth. If he can do all that that fast, the RAM makers should be able to at least 1000X their fab capacity on earth in one year. One year for scaling up existing tech is an eternity compared to Elon's timeframe for moon-fabs given the relative complexity of the challenge.

The most well known thing about Musk besides being an asshat, is that his timelines are almost always imaginary. He is not building fabs on the Moon in "just a few years".

But how is that at all related to the DRAM manufacturers' short term production limitations? Unless I'm being dumb and you're saying that these manufacturers could also just kick the can down the road for a few fab generations?

It's a business with huge up-front capital expenses and typically very low margins. Supply is scaling up slowly because it's hard, and if you overshoot, you go out of business. Nobody is "allowing" this. It's a natural property of being both advanced technology and a commodity at the same time.

I believe this is exactly what's going on -- they're buying parts in bulk, often months in advance, and locking in deals that a single consumer can't easily go get on the open market right now. Hardware pricing and availabilty pre-COVID was pretty predictable and stable, which meant the consumer could extract a meaningful cost advantage if they were willing to do the relatively modest amount of work of sourcing components individually and personally assembling the build. Right now, though, some places like Microcenter appear to have a cost advantage that fundamentally relies on market and pricing instability and can only be achieved through deeper integration with the supply chain and bulk purchasing in advance -- something a retailer like Microcenter can do, but I personally cannot.

I think RAM shortages would be the least of our problems… Assuming China takes TSMC in one piece (unlikely without internal sabotage in the best case scenario), it would still probably take years before it produces another high end GPU or CPU. We would probably be stuck with the existing inventory of equipment for a long time…

I am surprised we consider TSMC like a natural resource: isn't it really a combination of know-how and build-out according to that know-how? If smarts leave the country, perhaps this moves with them. The risk with China taking over Taiwan is that they mostly expedite their own production research by a couple of years.

It kinda does resemble a natural resource though. The machines and technology in use at TSMC are so insanely complex, that there isn't a single person on earth who knows everything about how it works. TSMC functions only because of all of the pieces of the puzzle being together in the right place and arranged in just the right way. It's a very fragile balance that keeps it all running, and a major disruption could mean we get thrown back by a decade in chip-making technology.

> I am surprised we consider TSMC like a natural resource: isn't it really a combination of know-how and build-out according to that know-how? Have you seen how many states and countries look enviously at Silicon Valley’s tech companies, China’s manufacturing dominance, or London’s financial sector and try to replicate them? Turns out it’s way harder than you’d expect. Hell, Intel can’t match TSMC despite decades of expertise, much greater fame, and regulators happy to change the law and hand out tens of billions in subsidies.

What you say is absolutely true, and is a serious problem—but the way our system operates does not allow us to correct for it. Anyone trying to spin up a competitor to TSMC would have to first overcome a significant financial hurdle: the capital investment to build all the industrial equipment needed for fabrication. Then they'd have to convince institutions to choose them over TSMC when they're unproven, and likely objectively worse than TSMC, given that they would not have its decades of experience and process optimization. This would be mitigated somewhat if our institutions had common-sense rules in place requiring multiple vendors for every part of their supply chain—note, not just "multiple bids, leading to picking a single vendor" but "multiple vendors actively supplying them at all times". But our system prioritizes efficiency over resiliency . A wealthy nation-state with a sufficiently motivated voter base could certainly build up a meaningful competitor to TSMC over the course of, say, a decade or two (or three...). But it would require sustained investment at all levels—and not just investment in the simple financial sense; it requires people investing their time in education and research. Dedicating their lives to making the best chips in the world. And the only reason that would work is that it defies our system, and chooses to invest in plants that won't be finished for years, and then pay for chips that they know are inferior in quality, because they're our chips, and paying for them when they're lower quality is the only way to get them to be the best chips in the world.

China is 10 years into what you describe, no?

This bit, I mean: > A wealthy nation-state with a sufficiently motivated voter base could certainly build up a meaningful competitor to TSMC over the course of, say, a decade or two (or three...).

Yes. And then taking down TSMC will be a nuclear bomb that wipes everyone else’s economy at once.

the scientists will switch sides with minimal issues, like they did after WWII

The chance of significant disruption is higher than the chances of a full blown invasion. China has hybrid options like a quarantine of chip exports.

It seems that RAM manufacturers are still reluctant to increase production. They know something that investors don't about long term RAM demands?

The semiconductor industry has been a boom and bust industry for over 50 years. https://imgur.com/a/cDLoeZm I've been in the industry for 30 years and I've worked at companies with fabs were demand was high and customers would only get 30% of what they ordered. Then just 2 years later our fab was only running at 50% capacity and losing money. It takes about $20 billion and 3-4 years to make a modern new fab. If you think that AI is a bubble then do you want to be left with a shiny new factory and no products to sell because demand has collapsed?

They are not happy to invest tens of billions for new capacities for Sam Altman’s “I will surely buy once I have money, pinky promise”.

They're doing this but it takes time and if they overshoot even a little they're screwed.

The issue is supply is inelastic so even as prices soar they can only make more so fast and that won’t get fixed until 2028.

If only we have not allowed oligopolies to exist. Meanwhile, EU is not in the race at all and US has very few fabs.

It sounds to me like an incentive for new companies to make RAM.

RAM makers are not increasing their capacity. If AI bubble bursts we might see a momentary drop in RAM prices but it won't be dramatic. Return to "normal" is the best scenario I can imagine but my gut tells me we're probably never going back to early 2025 memory prices.

I fear the author and most commenters are not aware of the law of demand and supply. If there is demand for consumer RAM, there will be supply for consumer RAM. It just takes time and risk-assessment to scale up operations. We have RAM shortage now, we will have very cheap RAM tomorrow. It’s not like production is bottlenecked by raw materials. Chip companies just need to assess if the demand by AI companies will last so it’s better to scale up, or perhaps they should wait it out instead of oversupplying and cutting into their profits.

We're talking about advanced semiconductor manufacture. It takes years and 100s millions to billions of dollars to scale up operations. That's something you don't do unless you know there's demand to sustain it in future.