One aspect Paul Kedrosky mentioned recently is the concept of „duration mismatch“. The price per token goes down over time (either because the AI vendor reduces due to competition pressure, or because customers are now incentivized to use older cheaper models). But datacenters are financed through debt, with the assumption their revenue increases over time. Quoting him: „[AI vendors are] paying for a fixed cost with a depreciating commodity“[0]. So you have on one end the token revenue trending down, on the other end the training cost going up for the next frontier models, and you need to pay back your 10y debt. 0: https://youtu.be/wGZboZcSGDY?is=64GuKyqBh_4aSjTE

"So you have on one end the token revenue trending down, on the other end the training cost going up for the next frontier models, and you need to pay back your 10y debt." Not necessarily, the bond holders could simply take a massive hair cut and lose shitloads of money. On the topic of bubbles and exuberance, Jeff Bezos made the salient point that there was a massive over-invested biotech boom in the 1990s and tons of sophisticated investors ended up losing lots of money. But humanity still kept the medical advancements made by the boom. Stocks going down didn't un-research drugs, and it won't un-research new GPUs or un-build datacenters.

> Stocks going down didn't un-research drugs Drugs cost pennies to manufacture after they are researched and make their way through the approval pipeline. There are many generic drug manufacturers who can work off the existing formulas. The more apt comparison is that LLMs won't be un-trained. Opus 4.8 now exists. Even if Anthropic somehow went bankrupt, that particular asset could, at the very least, be sold for proverbial pennies on the dollar to a "generic" inference provider.

Research does get lost over time. The whole point of the patent system is keeping that from happening; if the drug company goes bankrupt, even if they lose all their internal documentation in the process, hopefully the patents and other public paperwork provides enough information for an unrelated company -- either having acquired the patent rights, or after the patent period ends -- to reconstruct the processes with less investment then the original research. If a bankrupt AI company maintains enough of a skeleton crew to consolidate and archive its intellectual property it could be sold off to another company, but there are also timelines where it all ends up digital dust in the wind.

Those data centers are specifically for AI workloads. Let’s say everything crashes and we now have all the data centers, what do you do with them? GPU are pretty specialized hardware, without AI a data center full of outdated graphics cards isn’t really too valuable. It’s really not obvious the infrastructure we are building for AI stuff is something that will benefit humanity over time. Without talking about the fact that bubbles are extremely destructive. Bezos is obviously someone who came out ok from the dotcom bubble but we are talking about something that destroys a lot of value globally. That has real, direct consequences, not just investors losing some money. The US economy is currently only growing because of the AI bet

AI data centers are being already used at max capacity, aren't they? I have a hard time imagining people would suddenly use AI less than they do as of today, let alone collectively drop it altogether. So the worst case scenario is that they'd need to be auctioned off way under what they'd be worth now, but still for someone to use them for AI. Dotcom infra buildup was completely different, in that it wasn't even close to being all utilized.

You sell the GPU's to remote gaming companies. Replace servers with regular compute.

> Those data centers are specifically for AI workloads. Let’s say everything crashes and we now have all the data centers, what do you do with them? You just run the models and sell the tokens. The demand will still be there even if there will be less money in chasing new frontier model > GPU are pretty specialized hardware, without AI a data center full of outdated graphics cards isn’t really too valuable. AI accelerators used in DC are not really "graphic cards" any more, you ain't running gaming on it

Datacentres aren't the same as infrastructure or research though. All the hardware in them has a finite, useful lifespan. In 10 years time it'll be totally useless Hardware fails, and also scales out in terms of efficacy to run it as more power efficient, modern hardware turns up. It requires constant investment to keep it useful, and cost efficient When AI pops, we'll temporarily have some extra compute capacity that will be horrendously uneconomical to run due to the high grid load and low consumer demand, before they get shutdown. There's simply no real use for them at this scale

In order to not un-build the data centers, they at least have to make more than it costs to operate them, and also not have some attractive liquidation value (the land, maybe). I could imagine something like “inference is done at home or in China, that’s the price to beat” and it’s not worth keeping all those GPUs cool out in Nevada.

But the parent comment was that one of the bigger costs in these data centers was the interest expense on the borrowed money. A restructuring removes or heavily reduces that amount. The fiber laid during the dotcom bubble never paid back the investors or lenders, but it's still profitably connecting customers all these years later.

It’s true once built the data center can operate right up to a financed data center value of zero. The investors will loose money but the costs of AI will go down as they do

Yup, that is the real economic benefit of bankruptcy - a reset.

Current AI datacenter/model development investment rate is roughly 1T/year. That's a lot. But the US economy is 33T/year. So the investment pays back (roughly) over ten years if, each year, the AI investments increase overall productivity by 0.6%, assuming the AI companies can capture half of the value of that productivity gain. > „[AI vendors are] paying for a fixed cost with a depreciating commodity“ That's just a confusing way to say you don't think future models will be worth the development costs. Because if future models are significantly better, why would the price of tokens to access those models deprecate?

The $1T number seems more promises than reality, which is closer to the $300B to $500B level. Still a big number, but between a third and a half of the value used in the popular media.

The cost of power cost increase alone on industry gonna erase all gains from it. You can't consider it in vacuum. AI takes limited resources. So far it winded up cost on near every consumer electronics that runs an OS, and it winded up cost of energy that is used by the entire industry and every single customer It's not just the cost of datacenters, it's cost of infrastructure (that given current direction of US govt will just be paid from people's fucking taxes and bills..) and cost of other industries turning outright unprofitable "thanks" to demands of AI

I really wouldn’t be surprised if we saw some of these data centers scrapped in the next few years

There are data centers that use and rent out 10 year old server GPUs. They can't run larger modern models. They can't run smaller models as fast as newer servers. So their remaining market is applications where customers are okay with older, smaller models and slower performance. They have to price the service lower than competitors due to the lower performance. The older GPUs are less efficient so it costs them more to keep them running. They're paid off, but they're taking up valuable power, space, and cooling in a data center. Eventually there is a tipping point where it's better to replace that space and power budget with something new that has more demand. The parts are sold off on the open market. There's an equilibrium demand for the parts from other data centers keeping older servers running and from hobby people who are okay with a jet engine sounding toaster of a GPU running in their home.

I can no longer edit this, but want to expand on my comment. I've seen those vision researchers want to train on H100s at the time and being told know, wait for the T4s. I've seen T4s running BERT models for document classification. When there are enough Blackwells in data centers that H100s are useless for inference by your standards (I don't know if we've arrived there or not yet), there will be people who, say, want to run the Taco Bell ordering chatbot on them. There will be people who have applications that are just fine with Qwen 2.5 who will be happy renting them. There seems to be this crazy consensus that hyperscalers are going to go into their datacenters and throw away their old GPUs. The reality is they have a ton of paying customers for them. And there may be insect identification apps from 2019 that say "you know what? H100s have gotten cheap enough I can use a VLLM so the user can describe where they saw the insect too", or the McDonald's website support chatbot developers say "Hey, the bigger cheapers have gotten cheap enough we can upgrade our models to Qwen 2.5". The frontier level GPUs in e.g. AWS have a huge premium. When the newer generations come out, they will be able to cut prices to a bit of a premium over the operational costs and still make a profit, and there are a ton of down-market customers who will be interested, who aren't willing to try to outbid Anthropic for Blackwells.

I used to work in datacenters, during spinning disk era we had technicians from vendors basically every couple of days to replace some broken part. When the massive switch to ssd happened instead of having them every couple of days it was 3 or 4 times per month. Despite no moving parts things broke anyway and, even if it doesn't break, the vendor can make you change the technology just by playing with maintenance cost of the older one, limiting or removing spare parts from the market.

They do degrade physically, but the bigger thing is they stop being competitive quickly. Each year or so we see doubling of GPU speeds for the same amount of power. If you build a 100MW data center with GPU compute and three years laster a new data center opens with the same cost for GPUs and same electricity cost you do, but can do twice as much compute, you quickly lose business unless the market is just so constrained customers can't afford to be picky. But the moment there's slack in the market you'll see major migrations off of providers that have the same cost but half, or quarter of the same performance. So when you see someone talking about GPUs fully deprecating in value in 1-3 years this is what they're talking about. Right now it's not a big deal because there's no slack in the market. But once there is, the bottom will drop out.

I assumed the issue was similar to crypto mining, where given finite amounts of space and power it makes sense to always be running the latest and most powerful GPUs instead of keeping older hardware running. There's definitely a secondary market for these GPUs as well.

Nothing is stopping them, it's just not worth it: Have a look at e.g. vast.ai's pricing ( https://vast.ai/pricing ). The V100 (2017 -> 9 years old) can be rented from $0.02 to $0.37/h (right now I can find a V100 with a Xeon Gold 6140 and 48GB RAM for $0.165/h). Let's assume the guy you rent it to pins it at its 250W TDP and let's ignore the running costs of CPU/RAM/etc... Then you draw 1/4 kwh for that compute hour. The industrial electricity prices in the US vary between 7.5 and 25 ct per kwh (depending on state, time of day, etc...), so at 100% efficiency, assuming nothing ever breaks, and the CPU consumes 0W you earn about 14ct/h. And remember: V100s hours are sometimes sold at 1/10th the price. If I pick average conditions you need to start thinking of whether it is worth it to rent them out: Usually it isn't unless you have them anyways and just sell idle capacity. It's barely worth it to run them in a pure "is it profitable" sense, if we also account for the opportunity cost of taking up a slot in your datacenter it seizes to be worth it really quickly.

i think its reasonable to give up 15% of speed for a decade more lifetime. This depreciation change alters economics of GPU

That extra decade might provide almost no revenue. The long tail isn’t profitable

When everything is said and done it'll be datacenters in American competing with ones in China that have several times lower electricity prices. Token prices will drop to a level that will be unprofitable for American data centers and they will need to close. Thats the main issue here.

Yes, even if the hardware is untouched. As technology advances, the power cost per compute cycle goes down. A gpu using old tech costs progressively more to operate compared to the newer models. So its value goes down over time = depreciation. As for duty cycles, the chips are perfectly happy at 100% operation. Cooling and power componants fail, not the chips. But it costs manpower to repair such things and manpower is inconveniant these days. A gpu with any sort of fault just gets dumped.

Meanwhile, Google...

Google also needs fabs to build their TPUs.

> The world is out of fab capacity. Can anyone expand on this point? I read an article saying that the big AI co's datacentre spend was a bunch of lies because they can't build datacentres at anywhere near the rate they want to.

From what I understand it’s mostly TSMC and the memory providers being out of capacity over the next few years. So it’s not even about datacenters. Here’s a Reuters article about TSMC: https://www.reuters.com/world/asia-pacific/broadcom-flags-su... So this is actual committed contracts with all kinds of companies such as Apple, NVidia, AMD. Also, the whole reason they can’t build data centers faster is precisely because of this.

> they can't build datacenters at anywhere near the rate they want to That was because the supplies the datacentre needed were constrained - supply-constrained, not end-user demand constrained, so would be in agreement with the GP comment (and the article I read didn't imply anything about lying).

> Do we know that AI providers are going to keep these per-token prices, or eventually lower them because of competition from China? Raise, they are going to raise the prices. We will spend more on AI infrastructure in 2026 and 2027 than the gross sales of the entire global software and services sector. Current pricing is at a major loss for current providers.

Agreed, this is where google is really, really set up to win the market. They can combine gemini subscription with a moderately more expensive google workspace and steal MSFTs entire $50 billion enterprise productivity software market. MSFT is quickly trying to get copilot in a good enough state but without TPUs I think itll be tough for them to serve a good enough model at a price people will accept.

I agree with all of this. So my question remains the same: How are the players investing 100s of billions in buildout going to hope to make this back? Market capture looks bleak, inference looks like a race to the bottom. End users look like they could be beneficiaries. Where do the big boys go?

> Speculation here but I think openAI/antrhopic api inference is insanely profitable, it just needs more volume to amortize the training costs. Well, they just rent their hardware, so I'm not so sure. But they'll both be public soon and we should get that breakout in their cost structures, somewhat.

The Dotcom bubble is an interesting comparison. The general thrust that everything would be online was correct, it was just that the market mistimed and misallocated of capital by a decade or more. There was massive spending on infrastructure capacity that we wouldn't end up needing until the 2010s. There were hype driven valuations completely disconnected from business fundamentals just because a company was an 'internet' company. Things were going from cutting edge to obsolete in less than a year. There were breathless promises that this was business 2.0! Of course, none of that sounds remotely like what is going on today... I'm optimistic about AI, but I also don't think that it is going to change everything as fast as promised.

As much as I love to hate on Uber, that website is from 2022. Uber has been profitable since 2023. It's profit margin seems to have stabilized around 10%. The real economic crime is losing at least $40bn over 10 years scaling a business that ended up having retail profit margins (i.e. low profit margins).

> I've tried the open weight models ... You tried that on a personal machine for yourself once. It's completely different calculation when serving a model to 3000 employees with ever evolving hardware and software requirements. You'll need dedicated hardware in data centers and experts to run them. A company will need to figure out how to manage acquisition, assets and expenses plus 1000 other things, in addition to its actual business. Guess who has figured out all of that already? AWS/Azure/OpenAI etc.

This completely ignores all the other huge costs the AI labs are paying in data center builds, researcher salaries, experiments, and training models. The fact that Anthropic is rumoured to have a profitable quarter indicates that their margins on API priced inference are very strong.

AI companies have more expenses than inference.

yes, and theres no evidence that they arent (or can't) use profitable inference to subsidise those other expenses. Some companies will keep spending massively to train better models, and some other companies will not, and offer good api prices. Which will end up being used? That depends on whether the spending turns into better value models

> theres no evidence that they arent (or can't) use profitable inference to subsidise those other expenses as far as we know there's no evidence that they can produce any profits at all

It’s not just about the model but also setting up the system to create and share compute (GPUs) which is quite complicated on its own. Ubers primary business focus isn’t infrastructure.

Electricity actually is only a small part of the data center costs. There are challenges in getting enough electricity that create problems, but the cost of the electricity really isn’t an issue.

Token costs rising because data center build costs must be paid down.. is not the whole picture. It is actually possible for token costs to fall despite the spending frenzy. Naively you’d expect to always keep paying more - but growth in token usage is what changes the equation. Amortizing debt over an exponentially growing amount of spend across a growing customer base (not per customer) lets the debt be paid off & costs covered even as each individual’s spend stays steady or even goes down - but it only works if there’s growth beyond some threshold that makes the whole thing hang together. No one on the outside knows how much growth that is, and everyone chases maximum growth. Jevons Paradox ends up being your friend as well as the friend of the inference providers as well as the friend of the inference financiers. If it’s a strong enough effect, it has potential to cancel out all the circular financing too, and let everyone ride out the bursting of the bubble.