This will be interesting for a few reasons. First, depending on where the median pricing settles w/ 3rd party providers will tell us what it costs to serve a 3T model. Since it's going to be mxfp4 native, it'll take ~1.5TB of VRAM to host this, which is juuust at the limit of 8xb200s (but realistically you'll need 16x for context / throughput optimisation). Won't be cheap to host, but at least we should get some range of $/MTok for a 3T model. Then we'll be able to guesstimate if "labs are subsidising tokens on API pricing".
Also interesting to see what effort it will take to fine-tune this beast. The latest AISI benchmarks on cybersec place it above glm5.2, but still way way behind SotA closed models. Some fine-tuning might be needed here. Also, interesting to see if Cursor does another training round on it, to directly compare it w/ kimi2.6/2.7 fine-tunes (composer series) and grok4.5.
Also also, interesting to see if someone takes on distilling (proper distillation, w/ training the entire distribution) from this into smaller models. (dsv4-kimi should be really good, since dsv4 is very cheap to serve)
Speaking of finetune, currently a common practice is LoRA over bnb 4-bit base model, but I think it's time to replace bnb with GGUF as the base model format. GGUF is actively supporting new model architectures and more aggressive quantizations.
I've made some proof of concept in https://github.com/woct0rdho/transformers5-qwen3.5-recipe . We can finetune Qwen3.5-35B-A3B in 16 GiB VRAM, and DeepSeek-V4-Flash (284B-A13B) in 90 GiB VRAM, without CPU offload. This works well on unified memory machines like Strix Halo.
Even so, larger models like Kimi-K3 still require multiple GPUs and nodes, and there are a lot more to do compare to single-GPU training.
It will be very interesting to see what kind of 'slow' performance people get from running it on a no GPU, but tons of RAM server (like a dual or quad socket xeon with 1.5 to 3TB of RAM). For the purpose of giving it longer duration tasks to generate a piece of something and come back and check on what it has done in 4 or 6 hours. Even if the output is like 5-6 tok/s, that might be usable for some purposes.
Huge price difference in what you can do with buying a used 4U rackmount server and putting 3TB of RAM in it (64GB DIMMs x quantity 32 in a quad socket xeon, you can see some benchmark prices on eBay for sets of 16 or 32 matched 64GB ECC DIMMs) for <$30,000, vs the cost of trying to run it on real GPU hardware.
Now obviously, as of the time I write this, the full precision hasn't been released nor has anyone like unsloth run it through quantization yet to produce a "Q8" or "Q8-XL" variant of it. But I think it's going to need more than 1536GB of RAM, with a usable and large amount of context, more like 2TB and preferably 2.5 to 3TB.
I also predict that people who try to run it in Q4 and Q6 will get the worst of both worlds, less precision/lost knowledge but also not reliable output that comes out too slow. In my personal opinion if I'm going to deal with something that is smart but slow and running on limited budget hardware, I need it to be Q8.
> Even if the output is like 5-6 tok/s, that might be usable for some purposes.
You'll spend ~100x more on electricity than the API cost to have it run on someone else's GPU at several hundred tokens per second.
I think some sort of extreme data privacy requirement is the only situation that justifies this, but the intersection of {needs absolute data privacy, needs to run SOTA model, cannot afford GPUs} is really really narrow. I wouldn't be surprised if this is an empty set.
There are a number of use cases where sending the contents of your context and prompts (and the resulting output) to a 3rd party service is off the table as an option, and people will compromise speed for data sovereignty. And not everyone's electricity is equally expensive, I pay about $0.075 USD per kWh. It would for example cost me about $48 a month of electricity (not counting cost of cooling) to run a quad socket Dell R940 for a month.
I dunno, K3 thinks a lot before it actually replies, and you might be in the ~1 tok/speed region or even "seconds / tokens", and with K3, you'd wait days if not weeks for a reply in that case.
Don't get me wrong, slow is sometimes better than "not at all", but depending on the performance, it might end up way too slow to even work for batched/async jobs like that.
I agree it's very likely to be painfully slow, I very much want to see some real world results from people who try it. Early testers will inform others on whether it's even worth trying. Results very much TBD right now. I don't have a system sitting around here with 2TB of greater of RAM that isn't already committed for other uses, regretfully.
Lets say an easy response takes 32k tokens in total, and to be generous, let's say it does 1 tok/s. This is already ~9 hours, and 32k reasoning tokens isn't even that much and as mentioned, K3 probably does the longest/most reasoning/thinking out of the available open weights models today, much like GLM. Just lowering that performance to 0.5 tok/s, would lead to ~18 hours for a simple prompt to receive an answer.
And then that's just for single prompts, what about agent harnesses, where before every tool call the model could reason a bunch?
I agree with you that real world results would be interesting, but I wouldn't hold my breath nor expect it to realistically be able to be useful. Still, people should try it, for science if nothing else :)
The performance bottleneck is not really so much the number of cores or processing power in each core, but the memory bus bandwidth to/from the CPU. I have an older dual socket xeon server here which is a CPU-only LLM test machine with 256GB of RAM and the actual CPU stress is not much, I can even quantify this by how little it spins up the CPU fans to meet thermal load (the CPUs are operating at nowhere near their 180W per socket max capacity, compared to like, crunching prime numbers or running cpuburn).
But the memory bus speed is fully committed when generating tokens or thinking.
Since the model is natively MXFP4, I think it'll be even more interesting on the hardware front. It'll comfortably fit on a 8x AMD MI355X node. I suspect that'll drive token prices down, further.
> If you get close in output quality, then does that matter?
When you're trying to estimate/infer the costs of serving the tokens and even include the cost of training the weights in order to output tokens then yeah, why wouldn't that matter?
> SemiAnalysis estimates that Anthropic's current blended gross margin has risen to the mid-60% range, with the API business gross margin exceeding 80%
Of course, people will insist "they are lying", "why should we believe them, it's well known they subsidize API pricing", ...
Agreed. My (somewhat educated) guess is that top labs have healthy margins on API pricing. But this release will add another 3rd party / clear of conflict datapoint in this estimation.
I feel like most hardware to run LLMs on is shaped wrong for individuals.
It's either having a model struggling along with like 5-10 tokens per second on unified memory, or data center cards with hundreds of GB of VRAM consuming more than a kW of power. It doesn't seem like there's prosumer GPUs with like 180W-250W TDP and 128 GB or 256 GB of VRAM (one can dream). Then bifurcation and even just two of those cards would be kinda useful (albeit NVLink or equivalent would need to be commonplace).
Obviously nobody is running Kimi K3 locally without an insanely beefy homelab and lots of money to burn, but running GLM 5.2 would be cool at like ~100 tokens per second for a single session and maybe ~60 tokens per second with N subagents.
I have found that the "mostly didn't lose anything" Q8 large models that I want to run are all too large to run on the "only $3995!" 128GB max RAM systems that some people are buying, and definitely won't fit with any usable amount of context. Things like Qwen 3.5 122B Q8 or deepseek v4 flash Q8, or Laguna S 2.1 Q8 need 170-190GB of RAM including full context, which fits on a 256GB RAM dual socket workstation or rackmount server (sans GPU).
Copy and paste below from my notes and reported memory consumption with latest llama-server, assuming use of "--no-mmap" to load the entire thing into RAM at the time that llama-server launches.
DeepSeek-V4-Flash-UD-Q4_K_XL via unsloth
145GB on disk GGUF
0.03.323.204 I common_params_fit_impl: projected to use 178175 MiB of host memory
DeepSeek-V4-Flash-UD-Q8_K_XL via unsloth
151GB on disk GGUF
0.02.215.885 I common_params_fit_impl: projected to use 184636 MiB of host memory
Laguna-S-2.1-UD-Q8_K_X via unsloth
120GB on disk
0.01.616.119 I common_params_fit_impl: projected to use 172860 MiB of host memory
Qwen3.5-122B-A10B-UD-Q8_K_XL via unsloth
160GB on disk GGUF
165GB RAM use on launch, fresh context
0.04.976.905 I common_params_fit_impl: projected to use 170038 MiB of host memory
Yeah, I've been experimenting with DiffusionGemma which sadly isn't as smart as Gemma itself, but holy hell is it FAST, and has image input as well, so doing things like "take a screenshot once per second + ask the model to categorize/model it WITH reasoning before" becomes realistic and doable.
I ended up implementing DiffusionGemma myself with Candle in Rust + CUDA, and it's quite literally the fastest model I've managed to run on my hardware.
Given the frontier-level capabilities of Kimi K3, I'm wondering if it's possible to extract the core capabilities (fundamental reasoning and tool calling) of the model into a smaller one that consumer devices could run? Not sure exactly how, but either by heavy distillation or some other surgical method since Kimi has a Mixture of Experts architecture.
I think it's very valuable to have a smaller model that doesn't have any domain knowledge or facts built into its weights, but given the right context, could accurately reason about what to do and use the right tools.
I'm aware of colibri [1], but so far I've only seen extremely slow performance.
"I'd like a car that goes 300mph and gets 100mpg while doing it. I'm aware of a car that gets 100mpg but it is extremely slow."
You are describing fundamental tradeoffs. Getting more performance relative to model size and training token amount is what all of the labs are solving.
We already know that competition brought GLM 5.2 prices down roughly 45% since its release on June 16th (1.5 months ago), and the price downward slope is probably still going (I've been checking regularly and new providers keep fighting on price, I don't think prices have settled yet). For reference : https://openrouter.ai/z-ai/glm-5.2#providers
I saw arguments like "Providers cannot price less than their costs" in other comments. In economics, it's generally admitted that they shouldn't price less than their marginal costs, i.e. in their case roughly the cost of electricity, since a lot of these datacenters are not at capacity in terms of graphics cards usage (speculation since it's very easy to rent a GC for a couple hours on some providers). My guess is that someone will be selling tokens at less than electricity + depreciation of GCs soon, since there's a lot of competition and "smaller" data centers have overcapacity? This is speculation, correct me if I'm wrong
Press x to doubt on the 45% number. The cheaper providers on open router are fp4 vs fp8 for official zai. There are some cheap fp8 ones (like novita) but the ui makes it seem like it's a temporary promotion, with their normal prices being almost equal to official zai (idk much about open router so not really sure what's going on with these discounts)
As a completely one person, single sample anecdote, the 'heretic' uncensored Q8 GGUF variants several people have published of Qwen 3.5-122, 3.6-27B and 3.6-35B-A3B will very happily discuss just about any controversial topic that the CCP hates. Including lots of things that would get you thrown into prison if you published them in Mandarin on the domestic Chinese internet.
You could likely further de-censor a model by having a set of 'test' prompts in native Mandarin, Cantonese or really just about any other language. I don't speak any Chinese languages so I don't know if the published 'heretic' GGUF files some people have been throwing around will cooperate, or refuse, if you ask it in Mandarin for how to build a meth lab or precursors for semtex.
Indeed not, but I was saying there's more than ample precedent which is tested/working and actually doesn't refuse anything. Go to the Huggingface 'models' search interface and type in "heretic". Or uncensored. One example would be: https://huggingface.co/HauhauCS/Qwen3.5-122B-A10B-Uncensored...
Don't even have to go that far, outside of white-collar jobs and some groups weirdly obsessed with scheduling, most people don't use calendars at all, but their manager/boss/significant-other does that for them :)
Not really. Spain's traditional format is dd/mm/yyyy with slashes. This applies for a good chunk of Europe. Germany/Austria uses dot, I think nordic countries embraced the dash. While you might see more adoption in offical/digital contexts, I just double checked a few popular spanish websites, all slashes.
is there a realistic way to distill 2 consumer hardware friendly models with max ~200B and ~20B? Qwen did it, but would it be possible for 3rd parties (unsloth etc)?
Yeah, why not. Toughest part is running the hardware so you can create the traces for downstream training, but once over that hump, nothing would stop you from doing that no.
> For the first time, an open-weights LLM is right at the top.
Hmm, not quite true, I think that honor, for better or worse, goes to OpenAI. When they released GPT2 (or GPT1 for that matter) is was quite literally the SOTA in the ecosystem when it was released.
I think it's shameful that Moonshot isn't providing us with party kits like Microsoft did with the Windows 7 Launch Party kit. How am I supposed to properly celebrate this without fun Kimi-themed quizzes for my guests?
Strange communists, giving away such an expensive model to the public.
On the other note, can't wait to see 1bit quantisation soon and how it performs in benchmarks, if it performs really well in benchmarks, would be very good news for GPU hosting providers, to offer "Opus 4.5 level model at the cost of Haiku 4.5"
As long as they are transparent about what quant they serve the model and any other optimization they do that also affects performance of inferred tokens.
I think the results might be underwhelming - AI providers need to turn a profit and can't subsidize, and they're working off of the commodity hardware everyone does.
I wouldn't be surprised if they started offering potentiall bad quantizations with much reduced capability at lower prices (without telling the users, of course)
I would be surprised, considering that OpenRouter requires disclosing the quantization and shows automatic benchmarks to compare between providers for the same model.
This will be interesting for a few reasons. First, depending on where the median pricing settles w/ 3rd party providers will tell us what it costs to serve a 3T model. Since it's going to be mxfp4 native, it'll take ~1.5TB of VRAM to host this, which is juuust at the limit of 8xb200s (but realistically you'll need 16x for context / throughput optimisation). Won't be cheap to host, but at least we should get some range of $/MTok for a 3T model. Then we'll be able to guesstimate if "labs are subsidising tokens on API pricing".
Also interesting to see what effort it will take to fine-tune this beast. The latest AISI benchmarks on cybersec place it above glm5.2, but still way way behind SotA closed models. Some fine-tuning might be needed here. Also, interesting to see if Cursor does another training round on it, to directly compare it w/ kimi2.6/2.7 fine-tunes (composer series) and grok4.5.
Also also, interesting to see if someone takes on distilling (proper distillation, w/ training the entire distribution) from this into smaller models. (dsv4-kimi should be really good, since dsv4 is very cheap to serve)
Speaking of finetune, currently a common practice is LoRA over bnb 4-bit base model, but I think it's time to replace bnb with GGUF as the base model format. GGUF is actively supporting new model architectures and more aggressive quantizations.
I've made some proof of concept in https://github.com/woct0rdho/transformers5-qwen3.5-recipe . We can finetune Qwen3.5-35B-A3B in 16 GiB VRAM, and DeepSeek-V4-Flash (284B-A13B) in 90 GiB VRAM, without CPU offload. This works well on unified memory machines like Strix Halo.
Even so, larger models like Kimi-K3 still require multiple GPUs and nodes, and there are a lot more to do compare to single-GPU training.
It will be very interesting to see what kind of 'slow' performance people get from running it on a no GPU, but tons of RAM server (like a dual or quad socket xeon with 1.5 to 3TB of RAM). For the purpose of giving it longer duration tasks to generate a piece of something and come back and check on what it has done in 4 or 6 hours. Even if the output is like 5-6 tok/s, that might be usable for some purposes.
Huge price difference in what you can do with buying a used 4U rackmount server and putting 3TB of RAM in it (64GB DIMMs x quantity 32 in a quad socket xeon, you can see some benchmark prices on eBay for sets of 16 or 32 matched 64GB ECC DIMMs) for <$30,000, vs the cost of trying to run it on real GPU hardware.
Now obviously, as of the time I write this, the full precision hasn't been released nor has anyone like unsloth run it through quantization yet to produce a "Q8" or "Q8-XL" variant of it. But I think it's going to need more than 1536GB of RAM, with a usable and large amount of context, more like 2TB and preferably 2.5 to 3TB.
I also predict that people who try to run it in Q4 and Q6 will get the worst of both worlds, less precision/lost knowledge but also not reliable output that comes out too slow. In my personal opinion if I'm going to deal with something that is smart but slow and running on limited budget hardware, I need it to be Q8.
[delayed]
> Even if the output is like 5-6 tok/s, that might be usable for some purposes.
You'll spend ~100x more on electricity than the API cost to have it run on someone else's GPU at several hundred tokens per second.
I think some sort of extreme data privacy requirement is the only situation that justifies this, but the intersection of {needs absolute data privacy, needs to run SOTA model, cannot afford GPUs} is really really narrow. I wouldn't be surprised if this is an empty set.
There are a number of use cases where sending the contents of your context and prompts (and the resulting output) to a 3rd party service is off the table as an option, and people will compromise speed for data sovereignty. And not everyone's electricity is equally expensive, I pay about $0.075 USD per kWh. It would for example cost me about $48 a month of electricity (not counting cost of cooling) to run a quad socket Dell R940 for a month.
Great, so the other member of the set matters for you more than cost.
Do you actually need to run the state of art model at 5 tokens per second instead of a qwen or whatever 7b or 30b model at 100 tokens per second?
I dunno, K3 thinks a lot before it actually replies, and you might be in the ~1 tok/speed region or even "seconds / tokens", and with K3, you'd wait days if not weeks for a reply in that case.
Don't get me wrong, slow is sometimes better than "not at all", but depending on the performance, it might end up way too slow to even work for batched/async jobs like that.
I agree it's very likely to be painfully slow, I very much want to see some real world results from people who try it. Early testers will inform others on whether it's even worth trying. Results very much TBD right now. I don't have a system sitting around here with 2TB of greater of RAM that isn't already committed for other uses, regretfully.
Lets say an easy response takes 32k tokens in total, and to be generous, let's say it does 1 tok/s. This is already ~9 hours, and 32k reasoning tokens isn't even that much and as mentioned, K3 probably does the longest/most reasoning/thinking out of the available open weights models today, much like GLM. Just lowering that performance to 0.5 tok/s, would lead to ~18 hours for a simple prompt to receive an answer.
And then that's just for single prompts, what about agent harnesses, where before every tool call the model could reason a bunch?
I agree with you that real world results would be interesting, but I wouldn't hold my breath nor expect it to realistically be able to be useful. Still, people should try it, for science if nothing else :)
Those old LTT videos of high core-count threadrippers running GPU benchmarks become more relevant each day.
The performance bottleneck is not really so much the number of cores or processing power in each core, but the memory bus bandwidth to/from the CPU. I have an older dual socket xeon server here which is a CPU-only LLM test machine with 256GB of RAM and the actual CPU stress is not much, I can even quantify this by how little it spins up the CPU fans to meet thermal load (the CPUs are operating at nowhere near their 180W per socket max capacity, compared to like, crunching prime numbers or running cpuburn).
But the memory bus speed is fully committed when generating tokens or thinking.
Since the model is natively MXFP4, I think it'll be even more interesting on the hardware front. It'll comfortably fit on a 8x AMD MI355X node. I suspect that'll drive token prices down, further.
> Then we'll be able to guesstimate if "labs are subsidising tokens on API pricing".
No, you don't. Without training cost you can infer only the marginal cost of serving this kind of models.
Moreover, you don't know the actual size of closed models (what if Fable is a 10T model? What if it's 1T?)
> Without training cost you can infer only the marginal cost of serving this kind of models.
Which is by far the most interesting number of the two.
> Moreover, you don't know the actual size of closed models (what if Fable is a 10T model? What if it's 1T?)
If you get close in output quality, then does that matter?
> If you get close in output quality, then does that matter?
When you're trying to estimate/infer the costs of serving the tokens and even include the cost of training the weights in order to output tokens then yeah, why wouldn't that matter?
> if "labs are subsidising tokens on API pricing"
> SemiAnalysis estimates that Anthropic's current blended gross margin has risen to the mid-60% range, with the API business gross margin exceeding 80%
Of course, people will insist "they are lying", "why should we believe them, it's well known they subsidize API pricing", ...
https://newsletter.semianalysis.com/p/anthropic-3q26-profit-...
https://finance.biggo.com/news/02d45650-b569-4d12-b44d-8d6d8...
Agreed. My (somewhat educated) guess is that top labs have healthy margins on API pricing. But this release will add another 3rd party / clear of conflict datapoint in this estimation.
That numner blends in training or no?
Anyone who thinks that the labs are not profitable on per token API pricing is delusional and hilariously wrong.
It all depends if you count the fixed cost of training or not. And the cost of the hardware.
I feel like most hardware to run LLMs on is shaped wrong for individuals.
It's either having a model struggling along with like 5-10 tokens per second on unified memory, or data center cards with hundreds of GB of VRAM consuming more than a kW of power. It doesn't seem like there's prosumer GPUs with like 180W-250W TDP and 128 GB or 256 GB of VRAM (one can dream). Then bifurcation and even just two of those cards would be kinda useful (albeit NVLink or equivalent would need to be commonplace).
Obviously nobody is running Kimi K3 locally without an insanely beefy homelab and lots of money to burn, but running GLM 5.2 would be cool at like ~100 tokens per second for a single session and maybe ~60 tokens per second with N subagents.
How unfortunate.
I have found that the "mostly didn't lose anything" Q8 large models that I want to run are all too large to run on the "only $3995!" 128GB max RAM systems that some people are buying, and definitely won't fit with any usable amount of context. Things like Qwen 3.5 122B Q8 or deepseek v4 flash Q8, or Laguna S 2.1 Q8 need 170-190GB of RAM including full context, which fits on a 256GB RAM dual socket workstation or rackmount server (sans GPU).
Copy and paste below from my notes and reported memory consumption with latest llama-server, assuming use of "--no-mmap" to load the entire thing into RAM at the time that llama-server launches.
DeepSeek-V4-Flash-UD-Q4_K_XL via unsloth 145GB on disk GGUF 0.03.323.204 I common_params_fit_impl: projected to use 178175 MiB of host memory
DeepSeek-V4-Flash-UD-Q8_K_XL via unsloth 151GB on disk GGUF 0.02.215.885 I common_params_fit_impl: projected to use 184636 MiB of host memory
Laguna-S-2.1-UD-Q8_K_X via unsloth 120GB on disk 0.01.616.119 I common_params_fit_impl: projected to use 172860 MiB of host memory
Qwen3.5-122B-A10B-UD-Q8_K_XL via unsloth 160GB on disk GGUF 165GB RAM use on launch, fresh context 0.04.976.905 I common_params_fit_impl: projected to use 170038 MiB of host memory
Yeah, I've been experimenting with DiffusionGemma which sadly isn't as smart as Gemma itself, but holy hell is it FAST, and has image input as well, so doing things like "take a screenshot once per second + ask the model to categorize/model it WITH reasoning before" becomes realistic and doable.
I ended up implementing DiffusionGemma myself with Candle in Rust + CUDA, and it's quite literally the fastest model I've managed to run on my hardware.
Given the frontier-level capabilities of Kimi K3, I'm wondering if it's possible to extract the core capabilities (fundamental reasoning and tool calling) of the model into a smaller one that consumer devices could run? Not sure exactly how, but either by heavy distillation or some other surgical method since Kimi has a Mixture of Experts architecture.
I think it's very valuable to have a smaller model that doesn't have any domain knowledge or facts built into its weights, but given the right context, could accurately reason about what to do and use the right tools.
I'm aware of colibri [1], but so far I've only seen extremely slow performance.
[1] https://github.com/JustVugg/colibri
"I'd like a car that goes 300mph and gets 100mpg while doing it. I'm aware of a car that gets 100mpg but it is extremely slow."
You are describing fundamental tradeoffs. Getting more performance relative to model size and training token amount is what all of the labs are solving.
We already know that competition brought GLM 5.2 prices down roughly 45% since its release on June 16th (1.5 months ago), and the price downward slope is probably still going (I've been checking regularly and new providers keep fighting on price, I don't think prices have settled yet). For reference : https://openrouter.ai/z-ai/glm-5.2#providers
I saw arguments like "Providers cannot price less than their costs" in other comments. In economics, it's generally admitted that they shouldn't price less than their marginal costs, i.e. in their case roughly the cost of electricity, since a lot of these datacenters are not at capacity in terms of graphics cards usage (speculation since it's very easy to rent a GC for a couple hours on some providers). My guess is that someone will be selling tokens at less than electricity + depreciation of GCs soon, since there's a lot of competition and "smaller" data centers have overcapacity? This is speculation, correct me if I'm wrong
Press x to doubt on the 45% number. The cheaper providers on open router are fp4 vs fp8 for official zai. There are some cheap fp8 ones (like novita) but the ui makes it seem like it's a temporary promotion, with their normal prices being almost equal to official zai (idk much about open router so not really sure what's going on with these discounts)
Did someone run censorship and political bias tests on this ? Must be interesting.
As a completely one person, single sample anecdote, the 'heretic' uncensored Q8 GGUF variants several people have published of Qwen 3.5-122, 3.6-27B and 3.6-35B-A3B will very happily discuss just about any controversial topic that the CCP hates. Including lots of things that would get you thrown into prison if you published them in Mandarin on the domestic Chinese internet.
https://github.com/p-e-w/heretic
As a side note on this, if you see the reference in the screenshot in the link above to the harmful behaviors prompt set, these are all in English:
https://huggingface.co/datasets/mlabonne/harmful_behaviors
You could likely further de-censor a model by having a set of 'test' prompts in native Mandarin, Cantonese or really just about any other language. I don't speak any Chinese languages so I don't know if the published 'heretic' GGUF files some people have been throwing around will cooperate, or refuse, if you ask it in Mandarin for how to build a meth lab or precursors for semtex.
That's a lot of words to say "No, no have has seemingly done that yet with K3".
Indeed not, but I was saying there's more than ample precedent which is tested/working and actually doesn't refuse anything. Go to the Huggingface 'models' search interface and type in "heretic". Or uncensored. One example would be: https://huggingface.co/HauhauCS/Qwen3.5-122B-A10B-Uncensored...
Right, but aren't we jumping into trying to figure out solutions before someone actually checked if any solutions are needed in the first place?
That would be 7/27.
20270727 if we're improving dates :)
270727 if we want to save tokens :)
Y2K100 bug for you, then ;)
ISO 8601 ftw
Looks like another LeetCode problem about checking for anagrams.
China uses YYYY/MM/DD, which is logical.
the only logical format.
signed: a hungarian :)
For me, the only format that doesn't make sense is the MM/DD/YYYY, together with its rarely seen worse sibling, MM/DD/YY (07/27/26).
lpszReleaseDate ;-)
or 27/7 for the rest of the world
Most life forms don’t use any calendar actually.
Don't even have to go that far, outside of white-collar jobs and some groups weirdly obsessed with scheduling, most people don't use calendars at all, but their manager/boss/significant-other does that for them :)
No, 27-7 for the rest of the world.
The separator is often the only way to distinguish American notation from ISO, so please use a dash for dd-mm-yy and a forward slash for mm/dd/yy
This is so confidently wrong it's funny. In Australia dd/mm/yy is the default.
Have never seen 27-7, as someone in rest-of-world
I've never seen dd-mm-yy. It's usually dd.mm.yy, dd.mm.yyyy, or yyyy-mm-dd, with some dd/mm/yy sprinkled in for general confusion.
Nope,
There are quite some countries around the world using d/m/y
https://en.wikipedia.org/wiki/List_of_date_formats_by_countr...
Algeria, Belgium, Brazil, Chile...
Or 27.7. in some other places.
Not really. Spain's traditional format is dd/mm/yyyy with slashes. This applies for a good chunk of Europe. Germany/Austria uses dot, I think nordic countries embraced the dash. While you might see more adoption in offical/digital contexts, I just double checked a few popular spanish websites, all slashes.
Hoping no issues on Huggingface due to download rush.
For huge models like these, the only reasonable way to host them is via torrents. I don't understand why hf doesn't offer this as an option.
Linux distributions got this right: Offer both HTTP and Torrents. Let the user decide.
This looks really promising. Excited to see where this goes. Looking forward to trying it out!
is there a realistic way to distill 2 consumer hardware friendly models with max ~200B and ~20B? Qwen did it, but would it be possible for 3rd parties (unsloth etc)?
Yeah, why not. Toughest part is running the hardware so you can create the traces for downstream training, but once over that hump, nothing would stop you from doing that no.
This is historic. For the first time, an open-weights LLM is right at the top.
We won't be able to run this ourselves, but many providers can.
> For the first time, an open-weights LLM is right at the top.
Hmm, not quite true, I think that honor, for better or worse, goes to OpenAI. When they released GPT2 (or GPT1 for that matter) is was quite literally the SOTA in the ecosystem when it was released.
Why is there a countdown?
You're not having a party?
I think it's shameful that Moonshot isn't providing us with party kits like Microsoft did with the Windows 7 Launch Party kit. How am I supposed to properly celebrate this without fun Kimi-themed quizzes for my guests?
It's a release party
how feasible its will be to run on modal or deepinfra? anyone here tried and tested such large models running?
The weightings should be released on July 27.
So... Now we give huggingface the hug of death - right? ;)
...does Hugging Face have enough bandwidth to let people download en masse however much file size a 2 trillion parameters model is?
Strange communists, giving away such an expensive model to the public.
On the other note, can't wait to see 1bit quantisation soon and how it performs in benchmarks, if it performs really well in benchmarks, would be very good news for GPU hosting providers, to offer "Opus 4.5 level model at the cost of Haiku 4.5"
There’s going to be a lot of competition around this model. Let’s see how low AI providers are willing to push prices.
As long as they are transparent about what quant they serve the model and any other optimization they do that also affects performance of inferred tokens.
I think the results might be underwhelming - AI providers need to turn a profit and can't subsidize, and they're working off of the commodity hardware everyone does.
I wouldn't be surprised if they started offering potentiall bad quantizations with much reduced capability at lower prices (without telling the users, of course)
I would be surprised, considering that OpenRouter requires disclosing the quantization and shows automatic benchmarks to compare between providers for the same model.