I saw the same kind of comments when the first Deepseek model was initially released. However the relevance that investors assign to something like this does not seem to match the excitement expressed here.
I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
> they do not believe that big corporations in the US or Europe would actually use chinese models
This is exactly right. The only AI that matters from an investor perspective is the AI that is built into your phone, your car, ships with Windows and Mac, and built into legal and business tools like Lexus Nexus and Salesforce.
Also the concern that open weights models could contain dormant adversarial behavior (i.e. they are sleeper agents) is more popular amongst decision makers than one would think.
In that regard it was a clever marketing move by Anthropic to release their paper "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training" ....
to be honest anything built into my phone, car, ships with mac is a hobby
what's important is where big enterprise point their budgets. and there we agree, it's not to chinese models (as hopeful as I am that we can avoid a duopoly and great local models)
There were pretty specific rumors about IPOs happening early this year, up to apparent insider bets on prediction platforms. Don't you think the apparent delays in IPOs could have something to do with the open weights/open source competition? The monetary conditions (interest rates,...) were better early in the year, arguably predictably so, so what would be your alternative explanation why they chose to postpone?
> I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
European business users (+ consumers) can access GLM-5.3 via Mistral.ai hosting. (with GDPR)
Can you explain how this removes their moat? Genuine question, I'm not sure I understand enough about LLMs to get how an open source inference engine undermines them.
Today the closed-source giants maintain a substantial cost-per-token advantage due to proprietary inference tricks like speculative decoding kernels, ultra-optimized FlashAttention derivatives, multi-GPU pipeline orchestration and dynamic prefix-caching algorithms.
I'm not sure whether the DeepSeek's open-sourced inference engine does all of these, but it appears that plenty of inference tricks have just been revealed for the open source community to learn from and start improving upon.
So... this makes it easier for someone in the position to run chatgpt class models as a service to get similar returns to openAI/anthropic, if I got you.
And I imagine this engine is by default mostly just deepseek compatible and inference engines are not universally applicable to all models?
It sounds significant for competition but I'm not sure how it sinks Anthropic/openAI. Presumably they still have their frontier models, the training data they're vacuuming from current users and access to hardware to serve at a ridiculous scale, right?
I think you’re correct, although suggesting that open AI is a trickle drown of AI companies is a bit naive.
But the key point is that something good enough always attracts more interest and contributions because a productive outcome is very realistic and tangible.
Open Weight models and competing open source inferences engines massively undermine monopolies; in theory at least.
See how cloud is easy to divest and “nationalise”.
Well, whatever real or perceived moat Openai/anthropic had. OP claims this open source inference engine reduces their chances of high valuation, I'm assuming this is due to their in house engine being something they could use as differentiation before?
"Based on this framework, we have achieved high-throughput, low-latency large-scale inference services for DeepSeek V4.1 Flash. Currently, the service throughput of a single cluster is 155 trillion tokens/day, and the overall revenue is $38 per card per hour (at off-peak pricing)"
It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments. Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
Not to say it can't still be a bubble, just that there is legitimately profit to be had and perhaps some sustainable equilibrium in the future.
I can't read the original Chinese text, but if it's revenue, then you still can't say anything about profitability. Housing and powering one GPU should be way less than $38 per hour, so there should be a profit on inference. If that profit can cover training new models we don't know, but it would appear to not be the case.
For the past year or two I think it has generally been agreed that inference is profitable, but the LLM business as a whole is not.
Until the Chinese stuff came online I was completely convinced this is just a bubble waiting to pop and nothing will come of this tech due to its absurd costs. The Chinese stuff changed this picture massively. I'm daily driving DeepSeek 4.1 flash (at API prices) and every now and then I give Opus 5.5 a try and don't notice anything missing for me.
My workflow is one where I am heavily involved in the process and I don't like giving the agent too much autonomy + I need to follow along. Opus 5.5 speak is still cause aneurysms for my brain. It also has this vibe to its prose... like it's looking down on me. This is Prolly my bias speaking though.
With this new data at hand, I have to wonder how profitable would it be for the investors.
The cost of amortising a datcenter, the gpus, the engineering costs attached all to produce inference.
At how much percentage of its inference would it be profitable (if it does), how much returns would it make over the long term and what would be the profit of those inferences/ profit-revenue and how that compares to other industries.
Are there any statistics which show this. Also how efficient token inference has gotten over time?
Today. As new suppy comes online and demand levels off, that 38$ will shrink. Absent legal monopolies, the margin will shrink to a sliver more similar to any other service. That is the danger of open source models. They destroy monopolies and shrink margins.
Deepseeks strategy is that each project must break even within 10 months.
$38/hrs equals $320K in 10 months.
Assume they only have 100 clusters, and as much as 40% operational costs. That would get them about $20M in 10 months, which I suspect is more than what 4.1 flash training has cost them.
Fascinating site. Thanks for linking. However note if you change to "open router" pricing in some cases even B200 which is ~2y old has negative margin (and AFAIK ppl are using longer amortization than 2y - I do not know how cost are calculated here though).
Interesting how a comment asking about the reasoning behind releasing this as open source is getting attacked, as if doing a good thing should not ever be questioned. I do wonder of China's motives behind this as well.
The government would not allow this open work if it did not benefit them somehow, or perhaps they feel that keeping it closed is not very important right now (maybe because they are somewhat behind the frontier).
Comparing a university student Torvalds and state actors open sourcing code is pretty naive...
You seem to severely overestimate the level of centralised command-and-control in China. It's not like every software company is going to the government for explicit sign-off of every open source release.
There is a pretty clear benefit to DeepSeek's reputation for all the other providers offering DeekSeek's open-weights model to have similar performance/benchmark scores.
In the past performance has varied wildly across inference providers, making this available should result in every inference provider implementing the same optimisations
This is based on a video seen on Facebook, so take it with a grain of salt.
AFAIK, BYD batteries are based on "bad" materials, with fabrication method perfected until they are competitive [enough(?)] with batteries made with the "good" materials.
The good stuff is patented, locked by the West. China workarounds it. It's not that it is "bad", is just copying West's strategy doesn't work for them.
I don’t think there’s got to be some deep conspiracy. Opening this is a pretty clear way to commoditize your competition. That’s an age-old strategy. The community may very well get this up to speed faster than if it remained closed, and DeepSeek / China can (probably) run circles around US labs from a hosting / service cost perspective.
For comparison, rental prices seem to be ~$2.75/ per H-100-hour [1]. Meanwhile, Deepseek says "38 USD per hour per card”. They haven’t specified which card as far as I can tell.
It's harder than ever to get h100s and then you have to usually commit to a long time period upfront, but yeah this release should close some of the gap.
Can somebody explain like I'm five what this exactly is? How are providers like Fireworks or Novita providing inference today? Do they use some other inference engine software? If yes, what is the advantage of the DeepSeek inference engine?
For the most part each provider implemented their own inference engines. This release will help them all adopt the common set of optimisations to serve DeepSeek Flash optimally.
I think the reason DeepSeek does this is partly to neutralize competitors' technology, but ultimately the core reason might be that data is what matters.
Chinese labs do not release "open weight" models because they think it is good for humanity. They do it because they are severely GPU compute constrained due to the Nvidia and ASML embargos. All available GPUs have to go toward training which leaves very little capacity to host inference at scale. So they release the models and encourage EU and USA 3rd parties to host them.
China will inevitably produce their own AI chips and then they will lock down their models just the same as everyone else.
The real loser in all this is obviously Europe, which has nothing competitive once the Chinese models are gone.
> The real loser in all this is obviously Europe, which has nothing competitive once the Chinese models are gone.
Those models are open source / open weights. Europeans can continue with Chinese models as a base same like Cursor used Kimi K2.5 a base for their composer 2.0 model or SWE-2 from cognition is post-trained Kimi K3
That's Kami Kaze for US frontier labs. There's no way from here for them towards trillion dollar valuations.
That's strategic sabotage to say the least.
Ban all open weight models! Cried the VCs.
I saw the same kind of comments when the first Deepseek model was initially released. However the relevance that investors assign to something like this does not seem to match the excitement expressed here.
I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
> they do not believe that big corporations in the US or Europe would actually use chinese models
This is exactly right. The only AI that matters from an investor perspective is the AI that is built into your phone, your car, ships with Windows and Mac, and built into legal and business tools like Lexus Nexus and Salesforce.
Everything else is hobby.
Also the concern that open weights models could contain dormant adversarial behavior (i.e. they are sleeper agents) is more popular amongst decision makers than one would think.
In that regard it was a clever marketing move by Anthropic to release their paper "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training" ....
to be honest anything built into my phone, car, ships with mac is a hobby
what's important is where big enterprise point their budgets. and there we agree, it's not to chinese models (as hopeful as I am that we can avoid a duopoly and great local models)
If they are open sourced, it means any random data center could host it domestically and AI becomes a utility.
There were pretty specific rumors about IPOs happening early this year, up to apparent insider bets on prediction platforms. Don't you think the apparent delays in IPOs could have something to do with the open weights/open source competition? The monetary conditions (interest rates,...) were better early in the year, arguably predictably so, so what would be your alternative explanation why they chose to postpone?
> I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
European business users (+ consumers) can access GLM-5.3 via Mistral.ai hosting. (with GDPR)
https://docs.mistral.ai/models/zai-glm-5-3
https://legal.mistral.ai/terms/get-started/
Can you explain how this removes their moat? Genuine question, I'm not sure I understand enough about LLMs to get how an open source inference engine undermines them.
Today the closed-source giants maintain a substantial cost-per-token advantage due to proprietary inference tricks like speculative decoding kernels, ultra-optimized FlashAttention derivatives, multi-GPU pipeline orchestration and dynamic prefix-caching algorithms.
I'm not sure whether the DeepSeek's open-sourced inference engine does all of these, but it appears that plenty of inference tricks have just been revealed for the open source community to learn from and start improving upon.
So... this makes it easier for someone in the position to run chatgpt class models as a service to get similar returns to openAI/anthropic, if I got you.
And I imagine this engine is by default mostly just deepseek compatible and inference engines are not universally applicable to all models?
It sounds significant for competition but I'm not sure how it sinks Anthropic/openAI. Presumably they still have their frontier models, the training data they're vacuuming from current users and access to hardware to serve at a ridiculous scale, right?
I think you’re correct, although suggesting that open AI is a trickle drown of AI companies is a bit naive.
But the key point is that something good enough always attracts more interest and contributions because a productive outcome is very realistic and tangible.
Open Weight models and competing open source inferences engines massively undermine monopolies; in theory at least.
See how cloud is easy to divest and “nationalise”.
What moat?
Well, whatever real or perceived moat Openai/anthropic had. OP claims this open source inference engine reduces their chances of high valuation, I'm assuming this is due to their in house engine being something they could use as differentiation before?
Yeah what moat is the right question to ask.
This will change nothing.
The real money for AI will come from enterprises, and applications that follow the law to the dot.
And little by little all those will be banned from using anything AI related with roots in China
It's the USA, you can just ban it.
https://public-inspection.federalregister.gov/2026-21447.pdf
Isn’t this a ban on it being used by the state, rather than people/civilians?
For now
Yeah, but they can't ban the rest of the world from using it :)
You could claim you're the more ethical of the two countries and hope people boycott it
ahahahah
That's right, they could
Thats like sticking your head in the sand :)
That’s a bingo
A nuclear bomb
"Based on this framework, we have achieved high-throughput, low-latency large-scale inference services for DeepSeek V4.1 Flash. Currently, the service throughput of a single cluster is 155 trillion tokens/day, and the overall revenue is $38 per card per hour (at off-peak pricing)"
Source: https://zhuanlan.zhihu.com/p/2095560295930508953
It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments. Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
Not to say it can't still be a bubble, just that there is legitimately profit to be had and perhaps some sustainable equilibrium in the future.
I can't read the original Chinese text, but if it's revenue, then you still can't say anything about profitability. Housing and powering one GPU should be way less than $38 per hour, so there should be a profit on inference. If that profit can cover training new models we don't know, but it would appear to not be the case.
For the past year or two I think it has generally been agreed that inference is profitable, but the LLM business as a whole is not.
Until the Chinese stuff came online I was completely convinced this is just a bubble waiting to pop and nothing will come of this tech due to its absurd costs. The Chinese stuff changed this picture massively. I'm daily driving DeepSeek 4.1 flash (at API prices) and every now and then I give Opus 5.5 a try and don't notice anything missing for me.
My workflow is one where I am heavily involved in the process and I don't like giving the agent too much autonomy + I need to follow along. Opus 5.5 speak is still cause aneurysms for my brain. It also has this vibe to its prose... like it's looking down on me. This is Prolly my bias speaking though.
With this new data at hand, I have to wonder how profitable would it be for the investors.
The cost of amortising a datcenter, the gpus, the engineering costs attached all to produce inference.
At how much percentage of its inference would it be profitable (if it does), how much returns would it make over the long term and what would be the profit of those inferences/ profit-revenue and how that compares to other industries.
Are there any statistics which show this. Also how efficient token inference has gotten over time?
https://inferencex.semianalysis.com/profit-estimator
Today. As new suppy comes online and demand levels off, that 38$ will shrink. Absent legal monopolies, the margin will shrink to a sliver more similar to any other service. That is the danger of open source models. They destroy monopolies and shrink margins.
Deepseeks strategy is that each project must break even within 10 months.
$38/hrs equals $320K in 10 months.
Assume they only have 100 clusters, and as much as 40% operational costs. That would get them about $20M in 10 months, which I suspect is more than what 4.1 flash training has cost them.
For the ones who are still wondering how so much profits, inference is a highly profitable domain: https://inferencex.semianalysis.com/profit-estimator
Accounting for initial acquisition cost and typical depreciation/failure rate? I assume they already consider colo price, electricity, etc
70%+ net profits margins are for rented infra, that means no major upfont expense. Also, these margins are against 60% utilization only.
You can well imagine if you own your infra and can make utilization over 90%, where these go.
And then, my link above is for K3 model, which, compared to ds flash models, is highly unoptimized.
If I had a few million lying around, I would have invested in inference.
Fascinating site. Thanks for linking. However note if you change to "open router" pricing in some cases even B200 which is ~2y old has negative margin (and AFAIK ppl are using longer amortization than 2y - I do not know how cost are calculated here though).
1. The link is for rented infra. That means the inference provider doesnt have to worry about deprecation.
2. Newer gen nvidia gpu's like GB300 are way more profitable.
Why isn't ordered by profit % per GPU? It' surely the most important indicator
$0.00051192 per M/tok ?
Assuming a 64 card cluster
Relevant: Deepseek reports its Revenue per GPU per hour at off peak prices (38$) [reddit link] https://www.reddit.com/r/DeepSeek/comments/1xanexq/deepseek_...
Also cool (from the translated screenshot since original is in Chineese https://zhuanlan.zhihu.com/p/2095560295930508953)
> the DeepSeek V4.1 Flash inference system has established three goals
> 1-..... 2-..... 3. Supports retaining KVCache for more than 72 hours
This is awesome
Interesting how a comment asking about the reasoning behind releasing this as open source is getting attacked, as if doing a good thing should not ever be questioned. I do wonder of China's motives behind this as well.
The government would not allow this open work if it did not benefit them somehow, or perhaps they feel that keeping it closed is not very important right now (maybe because they are somewhat behind the frontier).
Comparing a university student Torvalds and state actors open sourcing code is pretty naive...
You seem to severely overestimate the level of centralised command-and-control in China. It's not like every software company is going to the government for explicit sign-off of every open source release.
There is a pretty clear benefit to DeepSeek's reputation for all the other providers offering DeekSeek's open-weights model to have similar performance/benchmark scores.
In the past performance has varied wildly across inference providers, making this available should result in every inference provider implementing the same optimisations
This is based on a video seen on Facebook, so take it with a grain of salt.
AFAIK, BYD batteries are based on "bad" materials, with fabrication method perfected until they are competitive [enough(?)] with batteries made with the "good" materials.
The good stuff is patented, locked by the West. China workarounds it. It's not that it is "bad", is just copying West's strategy doesn't work for them.
I don’t think there’s got to be some deep conspiracy. Opening this is a pretty clear way to commoditize your competition. That’s an age-old strategy. The community may very well get this up to speed faster than if it remained closed, and DeepSeek / China can (probably) run circles around US labs from a hosting / service cost perspective.
Let’s hope this is just the beginning of bringing the drama queens (Dario and Sam) closer to planet Earth.
Curious how the performance compares to vLLM or SGLang!
They say it was inspired by both so I am wondering what the performance gain is.
For comparison, rental prices seem to be ~$2.75/ per H-100-hour [1]. Meanwhile, Deepseek says "38 USD per hour per card”. They haven’t specified which card as far as I can tell.
[1] https://www.ft.com/content/a1815bc5-b5d5-47ff-bbbc-e1a689929...
I also assume they will just slot in whatever cards they can get their hands on, so this figure could be an average.
It's harder than ever to get h100s and then you have to usually commit to a long time period upfront, but yeah this release should close some of the gap.
It's specified in the plain text:
--
Supported hardware:
NVIDIA SM100 GPUs
Huawei Ascend 950 NPUs
that doesnt say what cards the reference number used.
Notably missing their kv cache system
It's exceptionally optimized... 890 bytes per token. At standard 32K context, that's less than 30MB.
1. https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
2. https://www.mindstudio.ai/blog/deepseek-v4-1-flash-specs-arc...
yeah they mention:
> Required for a complete inference service (not included in this release):
> A KV cache system
> VE/Prefill/Decode disaggregation (most required engine functionality is already implemented)
> Load balancing across data-parallel replicas for both prefill and decode
Supported hardware: Huawei Ascend.
Deepseek moment.
They open source it because they don't need it anymore. Probably they are doing something better with Huawei
If that's the thing that makes $0.00X cached inputs possible, then this is huge!
https://x.com/lightseekorg/status/2112470406158561730
Deepseek released just enough to stir the pot but not enough to cook your own from scratch.
Wake me up when we can clone, point it at a dozen H100s we own, and get a working production service.
Anthropic and OpenAI will do just fine.
Can somebody explain like I'm five what this exactly is? How are providers like Fireworks or Novita providing inference today? Do they use some other inference engine software? If yes, what is the advantage of the DeepSeek inference engine?
For the most part each provider implemented their own inference engines. This release will help them all adopt the common set of optimisations to serve DeepSeek Flash optimally.
I think the reason DeepSeek does this is partly to neutralize competitors' technology, but ultimately the core reason might be that data is what matters.
Data and GPU compute capacity.
Chinese labs do not release "open weight" models because they think it is good for humanity. They do it because they are severely GPU compute constrained due to the Nvidia and ASML embargos. All available GPUs have to go toward training which leaves very little capacity to host inference at scale. So they release the models and encourage EU and USA 3rd parties to host them.
China will inevitably produce their own AI chips and then they will lock down their models just the same as everyone else.
The real loser in all this is obviously Europe, which has nothing competitive once the Chinese models are gone.
> The real loser in all this is obviously Europe, which has nothing competitive once the Chinese models are gone.
Those models are open source / open weights. Europeans can continue with Chinese models as a base same like Cursor used Kimi K2.5 a base for their composer 2.0 model or SWE-2 from cognition is post-trained Kimi K3
How to destroy the entire US economy without firing a single bullet
Not written in Rust? How dare you?
Has to be something other than Rust to be able to be "rewritten in Rust"
Given how much the models like React and JavaScript I guess we should be thankful its Python.
Their inference server/service (wrapper around this engine) could still be written in Rust, this is just the engine.
This is written in Rust: https://github.com/deepseek-ai/deepseek-recipe
This DeepSeek Recipe is also mentioned here: https://zhuanlan.zhihu.com/p/2095560295930508953