The title seems misleading, and reading the article explains the reason more clearly. There's nonsense OKR's and objectives at these companies to burn as many tokens as possible. It turns out that when you make a metric out of token usage, it unsurprisingly ends up becoming extremely expensive.
Inference is affordable, and you don't need a SOTA proprietary model to get a lot of use out of this technology. While you likely will still need a human engineer for quite a while longer, I don't agree that some number of humans + an LLM is going to be (or will ever remain) more expensive than just hiring more humans.
They may as well have just said:
Company institutes an OKR that the IT division must spend over $1000/day/developer (fictious number).
Company is surprised when IT division is costing far more than it did before. Company increases this to $1500/day/developer to build a system to identify why this has happened.
I feel like vibe coding is less of an issue than vibe leadership at this point, and vibe leadership has nothing inherently to do with AI. These people are getting a vague feeling in their giblets, and then chasing it to the illogical conclusion no matter the cost or outcome.
But aren’t the revenue numbers that have investors foaming at the teeth based on that “tokens as a metric” world? It can’t be both an explosive growth business and also only ROI with more disciplined spend.
I am afraid that the TL would be uncomfortable if they have no human team members but only agents, which means they have no space to pass the bulk and have to take responsibilities for the business results.
The media seems hellbent on torching AI. My news feeds are nothing but stories about the evils of data centers, how useless AI is, and how much everyone hates it.
The media is hellbent on torching it, and on propping it up against all reason too, both things can be true. HN is no exception. It's another noisy room problem where the distortion in dialogue is rapidly leading us into a distorted reality. https://thenoisyroom.com/
For people who are actually interested in reality, participation in the mainstream discourse either way is a strategic error. The best thing to do is to check out from all of it, actually read the literature and listen to the technical heros who are working at the edge, and stop reading the pro/anti marketing noise from the media or corporate PR
The premise of this article is incorrect - MS isn't cancelling Claude code internal usage because of AI costs too much, they're cancelling it because GitHub copilot is the compete product and they want their employees to use their product.
It's the same reason Teams got so much attention during lockdown.
Yeah, they conflate Microsoft's actions (which are not about cost) with a random quote from the "vice president of applied deep learning at Nvidia," who says that compute costs more than people on his team -- but his team isn't using LLMs for software development, they're literally a deep learning team that is burning compute in deep learning development ways.
If people would do even a little bit of math, they'd see that Microsoft can't possibly be paying more for AI than for developers: They have about 80K employees in product development roles. Senior developers probably cost them $400K all-in.
Do they have a $32 billion Claude bill? I suspect they do not.
The 'tokenmaxxing' trend is probably the more inane ideas emanating out of this whole AI wave. It goes in the opposite direction of efficiency and productivity maximization. Yet, it has wide acceptance.
The usage of AI has to be put in context for cost analysis.
A lot of people I see are using AI to beautify their documents, their slack conversations, emails, generating big enough documents with small prompts. Sending a slack message or email should not have required AI within the company. Its wastage of resources and time, just to make it sound better without changing much of the meaning.
Burning tokens is as easy as throwing dollars in a furnace.
Token usage is not a good measure of productivity.
Problem is nobody has really been able to figure out how to gauge productive AI engagement.
Are your developers maximizing productivity or are they burning tokens or resisting change.
They've made a hardware LLM that reaches over 14k TPS on Llama 3.1 8B, and you can try it here: https://chatjimmy.ai/
So clearly hardware LLMs are the future, and the cost will be drastically reduced. But I know that all the AI labs want to create a perception of high prices forever.
Somehow this 'ai companies will never be profitable' is believed by so many. It's often used by those who don't like AI. There is no doubt that ai is the most lucrative business currently out there. It will only get better. Faster hardware, better algorithms
The title seems misleading, and reading the article explains the reason more clearly. There's nonsense OKR's and objectives at these companies to burn as many tokens as possible. It turns out that when you make a metric out of token usage, it unsurprisingly ends up becoming extremely expensive.
Inference is affordable, and you don't need a SOTA proprietary model to get a lot of use out of this technology. While you likely will still need a human engineer for quite a while longer, I don't agree that some number of humans + an LLM is going to be (or will ever remain) more expensive than just hiring more humans.
They may as well have just said: Company institutes an OKR that the IT division must spend over $1000/day/developer (fictious number). Company is surprised when IT division is costing far more than it did before. Company increases this to $1500/day/developer to build a system to identify why this has happened.
I feel like vibe coding is less of an issue than vibe leadership at this point, and vibe leadership has nothing inherently to do with AI. These people are getting a vague feeling in their giblets, and then chasing it to the illogical conclusion no matter the cost or outcome.
Goodhart's Law: When a measure becomes a target, it ceases to be a good measure.
But aren’t the revenue numbers that have investors foaming at the teeth based on that “tokens as a metric” world? It can’t be both an explosive growth business and also only ROI with more disciplined spend.
Also, from the article it seems they just switched from one LLM (Claude Code) to another one (GitHub Copilot) rather than abandoning "AI"...
I am afraid that the TL would be uncomfortable if they have no human team members but only agents, which means they have no space to pass the bulk and have to take responsibilities for the business results.
I kind of doubt they ever needed the number of humans they have, but I am genuinely open to being wrong about that.
The media seems hellbent on torching AI. My news feeds are nothing but stories about the evils of data centers, how useless AI is, and how much everyone hates it.
The media is hellbent on torching it, and on propping it up against all reason too, both things can be true. HN is no exception. It's another noisy room problem where the distortion in dialogue is rapidly leading us into a distorted reality. https://thenoisyroom.com/
For people who are actually interested in reality, participation in the mainstream discourse either way is a strategic error. The best thing to do is to check out from all of it, actually read the literature and listen to the technical heros who are working at the edge, and stop reading the pro/anti marketing noise from the media or corporate PR
> listen to the technical heros who are working at the edge
Sounds like a great way to get the rose colored view.
well datacenters should go near power plants or cool mountain areas
for ML training loads, it just doesn't make sense to build them near residential areas for few millisecs
The premise of this article is incorrect - MS isn't cancelling Claude code internal usage because of AI costs too much, they're cancelling it because GitHub copilot is the compete product and they want their employees to use their product.
It's the same reason Teams got so much attention during lockdown.
Yeah, they conflate Microsoft's actions (which are not about cost) with a random quote from the "vice president of applied deep learning at Nvidia," who says that compute costs more than people on his team -- but his team isn't using LLMs for software development, they're literally a deep learning team that is burning compute in deep learning development ways.
If people would do even a little bit of math, they'd see that Microsoft can't possibly be paying more for AI than for developers: They have about 80K employees in product development roles. Senior developers probably cost them $400K all-in.
Do they have a $32 billion Claude bill? I suspect they do not.
The 'tokenmaxxing' trend is probably the more inane ideas emanating out of this whole AI wave. It goes in the opposite direction of efficiency and productivity maximization. Yet, it has wide acceptance.
Literally nowhere in the article does Microsoft report AI is more expensive than paying human employees.
Wow... what happened to Fortune? I thought they're far above this kind of clickbait.
There may be a word missing in the post title. Should be "Microsoft reports show AI is more expensive...".
The fact that AI is more expensive still comes through, even though Microsoft does not state this explicitly.
Builders you pay to equip with hammers are more expensive than builders you do not equip with hammers too. More news at 11.
The usage of AI has to be put in context for cost analysis.
A lot of people I see are using AI to beautify their documents, their slack conversations, emails, generating big enough documents with small prompts. Sending a slack message or email should not have required AI within the company. Its wastage of resources and time, just to make it sound better without changing much of the meaning.
Microsoft cancelled Claude because they need to dog food Copilot since it sucks. This was acknowledged internally and it’s not a secret.
It is probably more expensive for Microsoft now since the Anthropic tokens were subsidized.
http://archive.today/l3EEo
Burning tokens is as easy as throwing dollars in a furnace. Token usage is not a good measure of productivity. Problem is nobody has really been able to figure out how to gauge productive AI engagement. Are your developers maximizing productivity or are they burning tokens or resisting change.
It's the new "lines of code".
reminds me of "The Feeling of Power" by Isaak Asimov
When you unleash it without a plan or discipline, they are correct.
With discipline, it’s an aggregator.
https://devarch.ai/
I do hope the quality of your astroturfing is not indicative of the quality of your companies product.
AI is not more expensive than paying a human employee. AI is not capable of replacing a human employee yet so the premise of the title is wrong.
Taalas: https://taalas.com/products/
They've made a hardware LLM that reaches over 14k TPS on Llama 3.1 8B, and you can try it here: https://chatjimmy.ai/
So clearly hardware LLMs are the future, and the cost will be drastically reduced. But I know that all the AI labs want to create a perception of high prices forever.
>> Microsoft has reportedly begun canceling most of its direct Claude Code licenses
Hearsay information and click bait.
Say AI is very expensive and costs a lot. What happens when no one can or is not willing to actually do the work manually?
What if companies both don't see a large return on investment, and at the same time can't reduce their AI spend?
For MSFT: Just download DeepSeek locally and use it.
Or train your own power efficient stack.
Somehow this 'ai companies will never be profitable' is believed by so many. It's often used by those who don't like AI. There is no doubt that ai is the most lucrative business currently out there. It will only get better. Faster hardware, better algorithms
> It will only get better
The important part is how much and over what time period.
AI isn't expensive.
Opus is expensive. And almost always unnecessary.
For now.
Is that why GitHub is always down?
Have you tried closing another strait?