Summarizer

Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years. It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all the dry powder out of the private markets. Folks are now starting to ask difficult questions about their burn rate and revenue. It is increasingly looking like they might not commit to the purchase order they made which kick-started this whole panic over RAM. Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?

Don’t the memory makers always get left holding the bag? I feel this has happened at least three times before.

Last year ai folks are all over wallstreet and articles, decrying how hardware folks are a roadblock to new frontiers in AI. They just couldn't print and pack the new chips faster.

The problem in this case seems to have sprung from a lack of collusion. Altman reportedly approached Samsung and SK independently to strike deals for a large chunk of both companies' production. Neither party apparently knew he was negotiating with the other. If they had actually been communicating or colluding with each other, they would have put the screws to him, making it harder for OpenAI to assert control over the vast majority of the DRAM market. Failing that, you'd like to think a regulatory agency somewhere would step in to keep a single player from hosing everybody else, but...

Is the DRAM industry really capitalist? Focusing on just the Korean parties, it functions like a command economy. I would say the same about most high end semi-conductor manufacturing, TSMC, Intel, ASML are being commanded and driven by nation-state level decision making. Right now the command is to focus on high wattage centralized AI systems at the expense of everything else.

The sheer fucking blazing ignorance of this comment > Capitalists claim that this is optimal. Compared to starving under communism coz someone at top got the number wrong, yes. And it only really happens when there are massive, unpredictable market movements and governments not doing their job. Govt should look at the whole thing and just say "no", blame them. No market system self regulates well enough, and it's government job to file down the edge cases like this. But the revolution happened in country which has two utterly incompetent parties, both in pockets of billionaires, fighting for power, and the clowns from one that won last battle use AI to smokescreen the economic growth their actions cratered

> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag? The memory makers specifically did not scale up capacity to avoid being left holding the bag.

I am betting the pendulum swings faster to the other side to excess capacity as all the construction lies of Altman fall through with financiers waking up the the fact they can't build the infrasctructure as fast nor make any profits on that infrastructure that will get built.

Those financiers can’t risk not being involved in a company with even just a slight potential for AGI.

> Those financiers can’t risk not being involved in a company with even just a slight potential for AGI. Do recent actions of Open AI give you the impression of a company that believes it is about to attain AGI imminently?

All it matters that gullible investors do. Hell, all it matters to investors is not being left holding the bag in the end so they don't even need to believe it

What if it takes 100 years to get to AGI or we never achieve it? All bets on AGI will just fail over and over again for decades in that case. It seems a bit like saying financiers can't risk not being involved with Faster-than-light travel technology. Yeah, it would change everything if we got it, but betting that we'll get it soon over and over again is probably not going to get you a lot of money. We've been projecting both FTL and AGI as future possibilities for almost 100 years now. Do LLMs get us a lot closer to AGI? I think they get us a little closer and Moore's "law" making compute faster probably is a much bigger factor, but I think we're still a very very long ways away.

Who has been projecting FTL as a realistic technology ever? FTL is not possible according to the current laws of Physics, while AGI is at least not forbidden by them.

I think this should be something you can answer for yourself by looking at human media and news over the lasy 100 years. I find it hard to belive you haven't ever noticed anyone seriously saying we may possibly have FTL sometime in the future. Incidentially I think I read on HackerNews that Sam Altman has been talking about building Dyson Spheres in the future. I suppose they're not forbidden by the current laws of phyisics either, but I don't know if I would call them a realistic technology.

We don't even know what human consciousness is. We can't even answer if we have free will or not. And you are proposing that AGI..is what exactly ?

The IQ of the smartest human, the perfect memory storing and recollection of computers, the fact that it never tires. I don't know if it's AGI but it's already something greater than us.

ok...there's no 'general' IQ afaik

Is the IQ measured by tests created for and answered by humans in the training data?

If it was greater than humans already it wouldn't need humans to help it work.

> We don't even know what human consciousness is. I think Douglas Hofstadter satisfactorily answered this question. > We can't even answer if we have free will or not. Sure we can, it's just that most people don't like the answer.

> I think Douglas Hofstadter satisfactorily answered this question. He didn't prove anything. It's a theory, just like many others: https://plato.stanford.edu/entries/consciousness-higher/ > Sure we can, it's just that most people don't like the answer. Again, not proven

OpenAI does have projections for making money with ads that would make Google and Meta blush.

To show ads you need people to stay on your platform. This is especially true once ads become more intrusive or of lower quality, something the big players seem to gravitate towards to keep revenue up. Google and Meta have ways to lock in users (networking effects, the best search engine available, having your data stored there). I am not sure if OpenAI has that. Their edge regarding models is small, their strategy currently seems to be "buy ALL the hardware so nobody else can". Users can quite easily switch to other models.

They can just switch back to normal DRAM if HBM demand drops, you ain't getting it cheap just coz AI flops

> Folks are now starting to ask difficult questions about their burn rate and revenue. this view isn't updated correctly post-claude code and codex. there will clearly be sufficient demand.

Seriously? One release is all it took to turn the whole ship around?

I think I’m missing something. Financially, what bag would the memory makers be holding here? I don’t think I’m well informed regarding how these deals were structured.

Memory makers make capital investements (build different factories, convert physical production lines, etc.) to meet orders that have been place for the next ~5 years. OpenAI (or whoever) crashes and can't pay for the order leaving the memory makers in a tough spot.

> leaving the memory makers in a tough spot Oh noes! Think of a poor memory makers! The amount of money flowing both from the AI bubble and from quite literally scalping both the server and consumer market... They gambled on the opportunity and if they fail - it's their problem.

And how is that a problem and more importantly how's that a problem of Average Joe? Capitalists did their gamble things. If they fail in that gamble what forbids them to sell the regular RAM they made for AI bubbleists to the regular consumers? Besides HBM it's just the regular chips which are exactly the same for the consumer/server market, why it would be any different?

The FOMO is strong, but can also indicate a bubble. Demand is from circular deals and APIs are being locked down already.

Memory makers did get themselves into this situation by selling all wafers for empty promises and alienating everyone but OpenAI tbh. I do hope they end up holding the bag once again, cause after covid and the cartel thing they don't seem to ever learn their lesson on how to have the tiniest amount of integrity.

That's only sound is it not. If you take away the hype - nothing critical actually depends on LLMs. You can remove them all today, and nothing bad would happen.

Did OpenAI's order actually kickstart the "panic" over RAM? Are they really such a big RAM buyer?

Blows my mind that it's non-binding. If I booked half a hotel's rooms then suddenly said "yeah never mind. Half my friends cancelled and we're not staying", basically any hotel would be coming at me for my money because there's no way they can fill their rooms now and they're losing revenue. But OpenAI can really get the whole world to pivot towards it then say "cool but we don't need your product anymore" and RAM makers are just going to let it go. Whoever decided that was a good idea needs to be fired and publicly shamed.

Do the memory makers not have a contract in place for an order this large? I assume that they aren't going to take "trust us bro" as good enough for several million dollars in orders, and even if there is a way to cancel the order it won't be free. I would assume so at least, but i would like if anyone knew for certain.

I was interning at a company that made networking gear that was put out of business when their largest customer canceled an order within a week of the delivery date. The customer ran out of money. In terms of where you are in line of debtors when you haven't even delivered the product to a customer, it's so far back as to be assured you won't get your money. If the memory makers got a deposit from OpenAI as part of this deal, that is likely to be the only money they will get for any undelivered memory, particularly if OpenAI runs out of capital.

That would also be a pretty obvious admission of defeat for their original business model, though...

* several Billion dollars in orders.

You can’t squeeze blood from a stone as they say. If AI companies go bankrupt there’s no money to be had.

It's not going to last until 2028, it'll last until 'min(AI_bubble_burst, 2028)', which I expect will be a lot smaller than just '2028'. So the real question is, how long will it take to retool for non-HBM, and will there be a fire sale as they scramble to recover? Which also explains why production is falling behind demand, companies aren't going to sink billions into creating product for a market that could dry up overnight.

This will result in demand destruction which will starve the enterprise which will starve the hyperscaler. theres no situation where people not being able to afford hardware for 4 years results in the bubble not popping

The people who fucked over consumers are left holding the back that they sold us out over? Oh no!

You can't build capacity overnight, and even with that in mind, it's hard to say if it is sensible to increase capacity now that we are in an AI bubble. For all we know, the bubble might burst.

So the ML hate is weaponized in the form of memory demand collapse FUD, and the public at large has to pay through their nose for it... thanks party poopers!

It has the makings of a natural monopoly, except its compounded by RAM cartels colluding to shut out the last of the competitors. Recently they had a second price fixing lawsuit thrown out (in the US). Now with the state of things I'm sure another lawsuit will arrive and be thrown out because the government will do anything to keep the AI bubble rolling and a price fixing suit will be a threat to national security, somehow. Obviously thats speculative and opinion but to be clear, people are allowing it. There are and more so were things that could be done.

I would expect that OpenAI gets as much money as they ask for for the next 10 years. There’s virtually infinite capital: if needed, more can be reallocated from the federal government (funded with debt), from public companies (funded with people’s retirement funds), from people’s pockets via wealth redistribution upwards, from offshore investment. They will be allowed to strangle any part of the supply chain they want.

If China capitalises on the big three focusing on data centre team, the big three might have a very hard time post bubble

I think the article has a giant blind spot as far as China is concerned , considering they have already a mature enough memory ecosystem via YMTC that Apple was considering sourcing from them. As well as continued expansion in the DRAM and HBM Fabs [1]. It feels like the memory cartel once again trying to incentivise their various govt to cough up some more tax breaks/funding to cushion the AI buildout bet that they made and the bubble seeming about to pop. In any case if they leave the consumer market underserved it should be no surprise if before that 2030 prediction we are all on cheaper YMTC memory modules. [1] https://www.tomshardware.com/tech-industry/semiconductors/ym...

I'm guessing they become pets.com within the year. At least I hope.

I think you're massively overestimating how much money is really accessible here. The parent comment's right that all of the easily available VC & private equity investment is basically used up. OpenAI was struggling to sell $600M of private equity, the big multi-billion dollar investment packages had lots of conditions and non-cash in it. > more can be reallocated from the federal government (funded with debt) While this is the most reliable funding, it's still not very accessible. OpenAI is a money pit, and their demands are growing quickly. The US government has started a bunch of very expensive spending. If OpenAI were to require yearly bundles of it's recent "$120B" deal, that's 6% of the US' discretionary budget. 12.5% of the non-military discretionary budget. (And the military is going to ask for a lot more money this year) Even the idea of just issuing more debt is dubious because they're going to want to do that to pay for the wars that are rapidly spiralling out of control. None of this is saying that the US government can't or wouldn't pay for it, but it's non trivial and it's unclear how much Altman can threaten the US government "give me a trillion dollars or the economy explodes" without consequences. Further deficit-spending isn't without it's risks for the US government either. Interests rates are already creeping up, and a careless explosion of deficit may well trigger a debt crisis. > from public companies (funded with people’s retirement funds) This would be at great cost. OpenAI would need to open up about it's financial performance to go public itself. With it's CFO being put on what is effectively Administrative Leave for pushing against going public, we can assume the financials are so catastrophic an IPO might bomb and take the company down with it. Nobody's going to be investing privately in a company that has no public takers. Getting money through other companies is also running into limits. Big Tech has deep pockets but they've already started slowing down, switching to debt to finance AI investment, and similarly are increasingly pressured by their own shareholders to show results. > from people’s pockets via wealth redistribution upwards The practical mechanism of this is "AI companies raise their prices". That might also just crash the bubble if demand evaporates. For all the hype, the productivity benefit hasn't really shown up in economy-wide aggregates. The moment AI becomes "expensive", all the casual users will drop it. And the non-casual users are likely to follow. The idea of "AI tokens" as a job perk is cute, but exceedingly few are going to accept lower salary in order to use AI at their job. There's simply not much money to take out of people's pockets these days, with how high cost of living has gotten. > from offshore investment. This is a pretty good source of money. The wealthy Arabian oil states have very deep slush funds, extensively investing in AI to get ties to US businesses and in the hope of diversifying their resource economies. ... ... "Was". Was a good source of money.

I'm genuinely curious to find out how many billions they get every year from now.

are you kidding? if spent all that money on food you guys would just use it to bullshit all day and make funny pictures, while if we spend it on AI..

Something I haven’t been able to reconcile: If AI makes software easier to create, that will drive the price down. How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software? This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out.

Analyst says it would require a new $35/month subscription for every iPhone user, or a new $180/month for every Netflix subscriber. Claude Max subscriptions have gone up, but do you think every Netflix user will pay for one?.. https://www.tomshardware.com/tech-industry/artificial-intell...

Everyone's betting on Jevons paradox the hope is that Ai is "the next semiconductor" and "the next internet"

> This demand for RAM is built on a foundation of sand Not exactly. LLMs are already quite useful today if you use them as a tool, so they are there to stay. The remaining problem is scalability, a.k.a. how to make LLMs cheap to use. But scalability is not really a requirement when you look the bigger picture. If smaller software company/projects can't afford to use AI, the bigger ones might just. Eventually they will discover variable use cases for such tech, even if it only serves big firms i.e. defense, resource extraction, war, finance etc. To the other end, if scalability is achieved, the use of LLM products will be cheaper too, so smaller project can also use them. But of course, if LLM usage is too cheap, then many were-to-be-consumers will just create software projects by themselves at their homes.

Would they be considered as useful if users are required to pay non-subsidized price though.

The usage of LLMs is continuing to increase ~exponentially. I'm going to bet on that rather than some half-baked scenario analysis that only takes into account one scenario and assigns a 100% probability to it.

> The usage of LLMs is continuing to increase ~exponentially I would like a source for that statement. Additionally, I want to know by who? Because it certainly isn't end users. Inflating token usage doesn't make it any more economically viable if your user base, b2b or not, hasn't increased with it. On the contrary, that is a worse scenario for providers.

> I would like a source for that statement The recent enterprise revenue numbers of Anthropic

So as I said, a self interested metric who also controls how many tokens it takes to get a desirable result from their models.

Users are willingly paying for larger volumes of tokens. You are layering your own unproven interpretation onto that. I would have arrived at an opposite interpretation given the available facts. Models are becoming more token efficient for the same task, such as ChatGPT 5.3 versus 5.2 which halved the token count, and capabilities show a log relationship with the number of tokens since o1 preview was revealed in September 2024.

No, you have gone off in your own tangent. The person you're responding to is talking about money and my point is that you're using a misleading metric. Even if the current user base is paying more for the "exponential token usage", it does not add up to the industry's cost of maintaining and building on this technology, especially since we are not taking into account what that token usage costs the provider. First you said Anthropic as your source, but now you're talking about OpenAI's ChatGPT, who are floundering for a product and user base, which they themselves claim will be profitable through subscriptions at numbers never seen before in a subscription business model.

> Additionally, I want to know by who? 1. As a consultant pretty much every company I have worked with in the last 2 years are doing some kind of in-house "AI Revolution", I'm talking making "AI Taskforce" teams, having weekly internal "AI meetings" and pushing AI everywhere and to everyone. Small companies, SMEs and huge companies. From my observation it is mainly due to C-level being obsessed by the idea that AI will replace/uplift people and revenue will grow by either replacing people or launching features 10x quicker. 2. Did you see software job-boards recently? 9/10 (real) job listings are to do with AI. Either it is fully AI company (99% thin wrapper over Anthropic/OpenAI APIs) or some other SME that needs some AI implementations done. It is truly a breath of fresh air to work for companies that have nothing to do with AI. The biggest laugh/cry for me are those thin wrappers that go down overnight - think all the "create your website" companies that are now completely useless since Ahtropic cut the middleman and created their own version of exactly that.

Yeah, my only hope is that this is unsustainable, admittedly for selfish reasons. I know plenty of engineers being forced to use these tools whether they want to or not. A lot of which are okay with using AI liberally, but don't particularly like generative AI and see it as pretty irresponsible (which feels more true by the week and it is clear from first hand experience). I don't know, there is a huge gradient of users, but I would argue that in previous revolutionary technologies, we didn't have to force people to use a good tool. I didn't have to be forced to use Google search or Google Maps, tech that is now ubiquitous with western society. It seems really suspect that suits have to enforce the use of something that is supposed to change the way we work and be a force multiplier.

From my limited experience in multiple companies, as stated before I see one very common pattern - The process from feature idea to development is just bad. PMs do not know what exactly they want. C-level interjects in the middle and changes requirements. QAs are unsure what to test because acceptance criteria is vague. C-level strongly believes that AI will fix all these issues. They believe that AI will fix their broken processes. I see strong resemblance with "Agile Development" ~15 years ago. Extremely hyped, noone asked if their org even is a fit for it or need it, and most importantly - the only way to fix agile is to do more agile. Same with AI right now.

It's a trojan horse. They expect the world will get hooked on it.

> If AI makes software easier to create, that will drive the price down. Supposedly AI drives down the cost of producing software,not the "price". > How are software companies going to make enough revenue to pay for AI, when the amount of money being spent on AI is already multiples of the current total global expenditure on software? Currently, the cost of AI is between $20/month and around $200/month per developer. I think the huge billions you're seeing in the news are the investment cost on AI companies, who are burning through cash to invest in compute infrastructure to allow both training and serving users. > This demand for RAM is built on a foundation of sand, there will be a glut of capacity when it all shakes out. Who knows? What I know is that I need >64GB of RAM to run local models, and that means most people will need to upgrade from their 8Gb/16GB setup to do the same. Graphics cards follow mostly the same pattern.

Right, that's where the major push is right now. Not with shrinking down some code libraries.

The semiconductor industry has been a boom and bust industry for over 50 years. https://imgur.com/a/cDLoeZm I've been in the industry for 30 years and I've worked at companies with fabs were demand was high and customers would only get 30% of what they ordered. Then just 2 years later our fab was only running at 50% capacity and losing money. It takes about $20 billion and 3-4 years to make a modern new fab. If you think that AI is a bubble then do you want to be left with a shiny new factory and no products to sell because demand has collapsed?

They are not happy to invest tens of billions for new capacities for Sam Altman’s “I will surely buy once I have money, pinky promise”.

The same thing everyone who's paying attention to the real world (and not the financial fantasy world) does: that OpenAI's purchase commitments are wildly unrealistic and unsustainable.

Of course, alternatively, the AI companies could go bust before finding profitability. Then, there’d be a ton of supply, prices would crash, and one or two of the current memory suppliers would go out of business. After that, the new Chinese memory companies might be producing at volume, and Renesas could be up and running. At the moment, nothing is certain. Could this last? Sure. Could it not last? Yup.

When I personally use chatgpt and friends, I am not seeing any slowdowns or anything, meaning that their servers can handle the loads just fine. So then, why are these companies spending so much building new capacity if the current capacity is enough?

Frontier labs flagship models are ~2T params at the moment, but they intend to ship 10T models like Claude Mythos, which would require substantial datacenter expansion. Same thing for training.

But thank god we were all able to generate some SVGs of pelicans, right guys?

This could be great. There's a future where RAM makers tool up for this massively increased demand, then the AI companies go broke as the bubble bursts, so RAM is cheap as. So laptop manufacturers get on that and start making laptops with 1TB+ memory so we can run decent LLMs on the local machine. Everyone happy :)

I'm personally hoping that one of the AI or data center companies is suddenly unable to pay for their bills and deflate the entire industry. Probably the only hope of things getting better before the 2030s.

That’s likely to happen if all the talks about OpenAI pulling out of their wafer deals are true.

This is simple extrapolation from current demand, nothing more. And that's a borderline silly analysis because it assumes the AI bubble won't burst. The great misadventure in the Persian Gulf probably accelerates that because we're almost certainly going to be facing a recession. Another thing I've been thinking about is what happens when the next generation of NVidia chips comes out? I suspect NVidia is going to delay this to milk the current demand but at some point you'll be able to buy something that's better than the H100 or B200 or whatever the current state-of-the-art for half the price. And what's that going to do to the trillions in AI DC investment? I'm interested when the next bump in DRAM chip density is coming. That's going to change things although it seems like much of production has moved from consumer DRAM chips to HBM chips. So maybe that won't help at all. I do think that companies will start seeing little ot no return from billions spent on AI and that's going to be aproblem. I also think that the hudnreds of billions of capital expenditure of OpenAI is going to come crashing down as there just isn't any even theoretical future revenue that can pay for all that.