- The factories don't stop working after 3 years. They can keep making GPUs.
- The AI boom isn't just about GPUs. There have been massive investments in CPUs, RAM, networking and routing components, physical data centers, power plants, cooling, and even fiber (the thing you yourself claim lasts decades). None of this is going to disappear in a couple years.
Shameless self promotion: I put together an AI infra built-out financials data-vis project/page recently. It includes graphs and tracks updates to stuff like GPU depreciation schedules like this.
Same with the consumer variants. RTX 3090 launched at $1499 and used ones on ebay are over $1500. GPUs large enough and new enough to do inference are not losing value.
Volta generation prices are collapsed, so GPU depreciation is currently around 8 or 9 years
There is a use for older GPUs, but the chips do have reliability issues. Turns out running chips hard 24/7 causes them to burn out. So it’s less will they be “useful” a while from now but more will they still work.
Folks are leaning it’s like these LED light bulbs that say they’ll last 15 years but after 2 years they start flaking out at a high rate.
Separately, the broader point is still directionally correct. We’re still using fiber from the .com network infra implosion today and will for a long time. The deprecation cycle is incredibly long. The increasing reports of GPUs sitting on shelves because there aren’t data center to power them is very concerning considering all the high interest debt being used to buy what is ultimately a rapidly depreciating asset. Financially, that’s a disaster waiting to happen.
Turns out running chips hard 24/7 causes them to burn out
I don't have direct experience running CPUs or GPUs 24/7, but when I was looking into acquiring used GPUs I looked into this and it seemed like "burning out GPUs" was almost entirely related to the thermal paste cooking off and other heat-related issues. The tribal wisdom seemed to be that GPUs could run pretty much indefinitely if you kept their thermals under control and reapplied the thermal paste every N months.
Hopefully somebody with direct experience can correct me if I'm wrong...
Fluctuations. If the use of the GPU is steady state, I think you're right; the thermals would be consistent. However, the reality is oftentimes loads come and go, and the rise+fall+rise of temps can lead to structural problems due to thermal expansion. Micro fractures in solder, thermal paste degradation, sagging PCBs, etc.
No, take a look into people who bought used GPUs that were from crypto farms. It's not just thermal paste, the chips become unreliable because of degradation of the circuits etched on the chips themselves.
GPUs may last 6 yrs or more. Fiber vs GPUs is however not the right comparison. Fiber vs Open models are. Open models have mostly caught up with frontier models. Now, with a reasonable harness and tools they can easily be useful for vast majority of users for a decade or more.
There are multiple YT channels proclaiming Russia is finally collapsing/losing war with Ukraine, huge "It is happening" in thumbnail; few of them put out a variation of such video every week, not even bothering to excuse their previous, wrong prediction.
And any reporting related to AI bubble is exactly as terrible. I might as well just check Bloomberg/FT/QZ, news there will be devoid of technical details and still more useful than those endless armchair-experts' speculations.
If you use opencode for your AI work or becomes affordable. To the point where you would reasonably build applications with intelligence the LLM version. I think in the near future the intelligence will be free and there is no business in free. I know it is a big statement but if you look around there is already a large amount of free models of gradually increasing strength. For most applications you don't need a gpt6
Lots of respect for him for staying unwaveringly principled while knowing when and how to adjust his expectations and predictions. He is really well informed and it's something else to watch him speaking live on the podcasts and be so specific with numbers. He really knows his stuff.
His track record of prediction on this topic is stunningly bad, and he's made a job out of being counter to the AI hype, so he seems likely unable to honestly update his beliefs to adapt to changing evidence without jeopardizing the attention people pay to him.
Predicting global events is next to impossible. Don't look at the predictions, as his prediction game has slowed and he has recently admitted that he was wrong on his predictions. Look at his exposing of financial shenanigans which let the bubble grow.
Sure, which is part of why most people try not to do it publicly with much confidence, lest they look like they're not worth listening to on the topic. Like, why are we listening to his opinions on this when he so clearly isn't an expert?
And I think a big part of why you shouldn't listen to him is that he's said repeatedly that he doesn't really use generative AI outside of occasional experiments - he hasn't seen its potential enough to bother using it himself beyond some initial tests. Which is fine, but I think it also means that he likely doesn't have a good intuition of the likely future demand, which is pretty key in modeling whether debt loads are sustainable or not. Because investing is fundamentally about trying to predict the future and putting your money into the things that will likely be successful there. And he's demonstrably been terrible at predicting the future of this stuff.
Some GPUs may easily last more than 10 years of light use, but I am not aware of public data about the reliability of datacenter GPus, which would give the percentage of A100s that die during the first 3 years of 24/7 use and during the first 6 years of 24/7 use.
I doubt that many of the 6-year old A100s have been in use 24/7.
Another thing to pay attention to is the margins of hardware companies like nvidia or micron. Hyperscalers are spending huge amount of money but the money mostly goes to the profits of hardware companies.
I don't want to argue here, but once models are trained, using them is quite cheap. Also in the last few months most improvements I've seen are in the smaller sized models, which might make using models even cheaper.
If the models are cheap to run, while good for customers, isn't that a problem for hyperscalers and all the investment poured into the AI data center build out?
its a false equivalence.... the dotcom boom wasnt just a fibre rollout, it was all the internet company startups that were overpriced and crashed. no doubt some of the hyperscalars will get busted out, but the models and LLMs are here to stay. just like dot com websites didnt disappear after the dot com crash, neither will AI transformer models
There's this weird tension where small models are getting a lot better, but the gap between the frontier and "what I can run on my gaming GPU" at home isn't necessarily getting smaller. That said, Qwen-3.8-27B-Q3 runs on my 9700 XT and performs well enough for tool calling in agents, which is a critical floor in local model performance imo.
Do you use qwen for any coding? Do you make a frontier model orchestrate it? Are you running with llama.cpp? Sorry for the barrage of questions, but I've been interesting in local AI for a while, yet I've struggled to set up anything usable.
- GPUs last a lot longer than 3 years.
- The factories don't stop working after 3 years. They can keep making GPUs.
- The AI boom isn't just about GPUs. There have been massive investments in CPUs, RAM, networking and routing components, physical data centers, power plants, cooling, and even fiber (the thing you yourself claim lasts decades). None of this is going to disappear in a couple years.
Shameless self promotion: I put together an AI infra built-out financials data-vis project/page recently. It includes graphs and tracks updates to stuff like GPU depreciation schedules like this.
https://artificialliabilities.com
I love the presentation of this. Phenomenal work, thank you.
Bears are gonna eat.
Oops, I read it as bullish. 98.6% utilization? Seems like demand for more capacity.
I recommend looking at any data regarding A100 rents (6 year old GPU) and then save 50 minutes of your time.
Same with the consumer variants. RTX 3090 launched at $1499 and used ones on ebay are over $1500. GPUs large enough and new enough to do inference are not losing value.
Volta generation prices are collapsed, so GPU depreciation is currently around 8 or 9 years
What are we looking for? What’s your time-saving insight?
The fact that the claim "GPUs last three years" is idiotic.
Thats the weird thing, scarcity drives value up so long as there is scarcity.
Once there is enough, or even a glut, it's over.
There is a use for older GPUs, but the chips do have reliability issues. Turns out running chips hard 24/7 causes them to burn out. So it’s less will they be “useful” a while from now but more will they still work.
Folks are leaning it’s like these LED light bulbs that say they’ll last 15 years but after 2 years they start flaking out at a high rate.
Separately, the broader point is still directionally correct. We’re still using fiber from the .com network infra implosion today and will for a long time. The deprecation cycle is incredibly long. The increasing reports of GPUs sitting on shelves because there aren’t data center to power them is very concerning considering all the high interest debt being used to buy what is ultimately a rapidly depreciating asset. Financially, that’s a disaster waiting to happen.
Hopefully somebody with direct experience can correct me if I'm wrong...
Fluctuations. If the use of the GPU is steady state, I think you're right; the thermals would be consistent. However, the reality is oftentimes loads come and go, and the rise+fall+rise of temps can lead to structural problems due to thermal expansion. Micro fractures in solder, thermal paste degradation, sagging PCBs, etc.
Which is almost exactly the same story that applied to the flaky LEDs - bad thermal designs leads to the LEDs or their driver circuitry breaking.
No, take a look into people who bought used GPUs that were from crypto farms. It's not just thermal paste, the chips become unreliable because of degradation of the circuits etched on the chips themselves.
[delayed]
GPUs may last 6 yrs or more. Fiber vs GPUs is however not the right comparison. Fiber vs Open models are. Open models have mostly caught up with frontier models. Now, with a reasonable harness and tools they can easily be useful for vast majority of users for a decade or more.
There are multiple YT channels proclaiming Russia is finally collapsing/losing war with Ukraine, huge "It is happening" in thumbnail; few of them put out a variation of such video every week, not even bothering to excuse their previous, wrong prediction. And any reporting related to AI bubble is exactly as terrible. I might as well just check Bloomberg/FT/QZ, news there will be devoid of technical details and still more useful than those endless armchair-experts' speculations.
If you use opencode for your AI work or becomes affordable. To the point where you would reasonably build applications with intelligence the LLM version. I think in the near future the intelligence will be free and there is no business in free. I know it is a big statement but if you look around there is already a large amount of free models of gradually increasing strength. For most applications you don't need a gpt6
Ed Zitron is going to become the Simpsons of ai critiques...
Lots of respect for him for staying unwaveringly principled while knowing when and how to adjust his expectations and predictions. He is really well informed and it's something else to watch him speaking live on the podcasts and be so specific with numbers. He really knows his stuff.
Go read "Some Zitron predictions" here: https://danluu.com/zitron/
His track record of prediction on this topic is stunningly bad, and he's made a job out of being counter to the AI hype, so he seems likely unable to honestly update his beliefs to adapt to changing evidence without jeopardizing the attention people pay to him.
Predicting global events is next to impossible. Don't look at the predictions, as his prediction game has slowed and he has recently admitted that he was wrong on his predictions. Look at his exposing of financial shenanigans which let the bubble grow.
Sure, which is part of why most people try not to do it publicly with much confidence, lest they look like they're not worth listening to on the topic. Like, why are we listening to his opinions on this when he so clearly isn't an expert?
And I think a big part of why you shouldn't listen to him is that he's said repeatedly that he doesn't really use generative AI outside of occasional experiments - he hasn't seen its potential enough to bother using it himself beyond some initial tests. Which is fine, but I think it also means that he likely doesn't have a good intuition of the likely future demand, which is pretty key in modeling whether debt loads are sustainable or not. Because investing is fundamentally about trying to predict the future and putting your money into the things that will likely be successful there. And he's demonstrably been terrible at predicting the future of this stuff.
He really does not.
Good point, very well structured argument here
Siri, what's the spot price for an A100?
GPUs might be 3 years but the rest of the hardware: processor cores, ram, storage can power other cloud applications for a decade or more no?
Except the GPUs last a lot longer than 3 years - A100s are 6 years old at this point and still in heavy use.
Some GPUs may easily last more than 10 years of light use, but I am not aware of public data about the reliability of datacenter GPus, which would give the percentage of A100s that die during the first 3 years of 24/7 use and during the first 6 years of 24/7 use.
I doubt that many of the 6-year old A100s have been in use 24/7.
Bad for hyperscalers. Good for Nvidia.
Another thing to pay attention to is the margins of hardware companies like nvidia or micron. Hyperscalers are spending huge amount of money but the money mostly goes to the profits of hardware companies.
I don't want to argue here, but once models are trained, using them is quite cheap. Also in the last few months most improvements I've seen are in the smaller sized models, which might make using models even cheaper.
If the models are cheap to run, while good for customers, isn't that a problem for hyperscalers and all the investment poured into the AI data center build out?
its a false equivalence.... the dotcom boom wasnt just a fibre rollout, it was all the internet company startups that were overpriced and crashed. no doubt some of the hyperscalars will get busted out, but the models and LLMs are here to stay. just like dot com websites didnt disappear after the dot com crash, neither will AI transformer models
All of this spending only has a chance of paying off if video generation takes off.
> using them is quite cheap.
until the other guys comes out with a slightly better model, then using yours wont be cheap enough.
There's this weird tension where small models are getting a lot better, but the gap between the frontier and "what I can run on my gaming GPU" at home isn't necessarily getting smaller. That said, Qwen-3.8-27B-Q3 runs on my 9700 XT and performs well enough for tool calling in agents, which is a critical floor in local model performance imo.
Do you use qwen for any coding? Do you make a frontier model orchestrate it? Are you running with llama.cpp? Sorry for the barrage of questions, but I've been interesting in local AI for a while, yet I've struggled to set up anything usable.
Just go download and install lemonade server. Open web ui for front end. There are mobile apps to connect to it.
https://lemonade-server.ai/