Yep. Demand open source approve licenses for LLM weights.
The Chinese apache 2.0 models might be censored, but at least they can’t sue you in the US for finding the censorship line.
OTOH, the US models are definitely censored, per TFA, and they’re making vague legal threats against anyone that encounters the censored edge of the model.
'pulling the ladder' is an action to sever the opportunity for others to climb after you.
they are merely engaged in self-serving rhetoric. can't even call this specifically hypocrisy because they aren't telling you not to train on on pirated content. just not their content.
It's utterly bonkers. Hopefully the model weights get leaked. Then we can claim it's public domain or, at the very least, distill it and then release it for free.
"To effectively contain a civilization’s development and disarm it across such a long span of time, there is only one way: kill its science." - Cixin Liu, The Three-Body Problem
This immediately made me think of the Sophons silently manipulating the sensors of particle accelerators to prevent humanity from developing advanced knowledge of particle physics.
The level of oppression necessary to get software geeks to stop making progress on AI is similar to that necessary to get Ukrainian geeks to stop making progress on drones.
It is as if Jetbrains told that "you can't use IntelliJ Idea to develop frontier IDE. We can introduce slight compilation errors if we detect you doing so".
has dario (or sam tbh) ever been thoroughly asked about the hypocrisy of them claiming distillation to be „theft“ vs. them training on the copyright of others?
I’ve only seen him talk about one of those topics, but never together.
I just can’t see how you can talk yourself out of that hypocrisy, if BS answers are properly followed up on (journalism!)
This is a fun peek into the economic implications of RSI/ASI. Because it's so infinitely valuable that it basically destroys all markets, labs will eventually do stuff like stop releasing models completely and skipping out on contracted commitments because they'll have the power to just drive their competitors out of business before the legal battle gets expensive.
Cloud providers - at first smaller ones, then the hyperscalers - will follow suit, completely closing sales to anyone but the labs and demanding payment in equity/direct decision-making power rather than cash. There's no particular reason why the inference/training split has to be 80/20, and no amount of willingness to pay can help you in an event that turns your money worthless.
I don't think this scenario makes sense. It's one of a class of scenarios I've seen several of, that simultaneously assume:
A) ASI is developed and massively overshadows the rest of the world economy
B) the world still has rule of law, contracts, business, well-developed finance, etc
You can get to a lot of weird conclusions if you assume both A and B, but I think the much more likely scenario is that if A happens, B stops being true in short order. If you are a company and you have ASI, you just stop caring about business and money and economics, and your outcomes instead start looking like "you conquer the world" or "you upload the board of directors to a fleet of von Neumann probes" or "you messed up, everyone dies".
There will be a brief(or, depending on the underlying rules of reality ASI uncovers, not-so-brief) period where A and B do overlap - we have superintelligence but still have to run experiments, manufacture robots, test new drugs in vivo, etc. That period is in and of itself dangerous for the labs, because many entities can just stop them by denying necessary inputs. For the labs to conquer the world, they'll need cooperation - from the state, from robotics companies, from compute companies, from the mining and energy and agriculture sectors.
There will be a period of time where markets attempt to run in a business-as-usual way while the transactions that matter happen as power-sharing arrangements - spots on the "AI Governance Board" or the "uploaded to von neumann probe" club. Markets will still matter in that the labs will need the state to overturn market obstacles to control of the world.
The existence of the A-B overlap also suggests to me that the US-China gap is less dire for China than it appears - they may be able to use their superior industrial, robotics, and scientific base to win the second leg of the race despite losing the first.
Especially when you can actively choose to not use Anthropic. They think they have a moat from all of the IP they've stolen. Just wait until there's nothing more to steal and the laws eventually turn against them. And let's be honest about these companies. It is very much Dario and Sam and Sundar and Mark and Peter and Elon and... These are the choices they are making and hopefully they are held accountable both legally and within society as a whole.
The moat looks deep today but it's going to become more shallow every year.
Training a new model from scratch takes serious resources. Post-training/fine-tuning an existing model, dramatically less. The knowledge for the process was esoteric two years ago, now you can ask a current model (one of several) to walk you through it, while building the tools to do it as you go. Several of my recent weekend projects have been exactly that sort of thing, just so I understand it better. "Let's make a LoRA", "let's generate a corpus of training data for fine-tuning a model for X task", "how can I put my face in a text-to-image model?" stuff like that. All of this is do-able on kinda modest local hardware (a couple of old GPUs or a Strix Halo or DGX Spark or big Mac Studio), or for a few bucks or a few hundred bucks or a few thousand bucks of cloud compute, depending on scale.
Scale that up to corporate or startup scale, with the money that's been flowing into AI for the past couple/few years, and it's obviously there's going to be a lot of competition just as the top model makers need to start ringing the cash register. That's a lot of opportunities for people to look at their ballooning Claude usage costs and find other ways to do the same thing for drastically less money. $100/month or $200/month is a no-brainer for Claude Code with probably the best model for coding, but they're pushing more users to usage-based billing which becomes cost-prohibitive real fast.
So, they desperately need to continue to be among the only ways to solve the hardest problems, and they need the alternatives to cost a similar amount. They can count on OpenAI and Google to ratchet up prices, too. They probably can't count on everybody, especially the vendors in China with different economics, to do it. And, they can't count on companies to look at their own usage and not ask, "Can we train a smaller specialist model that does this one thing we're using the Anthropic API most heavily for?"
I'm hoping they just mean stuff like using Claude for distillation by e.g. Chinese model makers, and not "how do I fine-tune Gemma 4 to write more like me?" or whatever.
What moat? You answered yourself: "capital intensive"
But, history says the supercomputer of today will fit in your pocket in a few years.
They've bought up all the RAM and GPUs, which pushes the capital requirements upward for everyone else. But, they can't corner the market forever, there are too many competing interests. AMD and Intel keep making new GPUs and APUs. The memory makers can't just sell to only AI companies forever, if they do Chinese manufacturers will move in and eventually eat them from below (as has happened many times before).
They have a moat today, and it's just that it's really expensive to train and host frontier models, especially at commercial scale. It used to be there was also some secret sauce to making it fast and efficient. But, secret sauce is being published daily by all sorts of researchers, folks are figuring out how to do more with less and it often finds its way into llama.cpp or vLLM or SGLang within days or weeks.
It is very difficult to see this move as anything other than Anthropic pulling the ladder up behind itself. They can dress it up in "safety" all they want, I find it hard to interpret this in a charitable way.
This reminds me of how dark-pattern common wisdom in Web 1.0 website development was to ban external links. Then how social apps prevented the export of data and actively worked to nerf significant interoperability through APIs.
But this is a tool, not just a data moat. Like a knife that degrades your ability to create knives. Or like a text editor that prevents you from implementing a text editor.
The charitable read is that their restrictions for "safety" (i.e. what's separating Fable from Mythos) makes this inevitable. If you could just make your own Mythos it would circumvent the protection.
Which kinda just highlights how weird this situation is.
I think there's a pretty big difference here. It's not like Github prevents you from building a Github competitor. Or Linear is preventing you from using it to build a Linear competitor.
This is more akin to Windows somehow preventing you from building a new OS.
I remember working for a company that did a lot of business in logistics. We were strictly prohibited from using any Amazon Web Services because several of our very high profile customers didn’t want anything on AWS. The higher ups were thoroughly convinced Amazon would copy it (and I mean, they came out with a product that competed with us, so they weren’t wrong!)
> This is more akin to Windows somehow preventing you from building a new OS.
Tangent, but have you tried repartitioning your Windows disk to make room for a new OS? Or tried to configure Windows to let you dualboot? Or get the clock time right if you dualboot? Or let you debug "Secure Boot"?
Windows is outright hostile when it comes to (sharing with) a new OS
The popularity of SaaS was never derived from the products themselves, but rather business' weird aversion to doing in-house development. Most companies not in tech view software as literal magic, and act as if hiring some engineers could risk opening Pandora's box or something. Banks are particularly notorious for this; despite basically their entire business being done digitally, they treat software as a necessary evil, not as their underlying value.
But, the cost of in-house development just went down significantly. SaaS has always had a lot of broken promises. The thing is the software is never tailored to your use case, and you often have to integrate into your other tools anyway. And, you don't get to control the requirements, features, velocity, or bug fixes. Jira as a bug? Too bad I guess, hopefully it gets fixed eventually.
But the dirty secret is that companies are filled to the brim with bright-eyed aspirational employees, who want nothing more than to make their job easier and their company more efficient. The thing is they're doing it using cursed Excel workbooks on share drives. I think, in the near future, they'll be doing it with hand-rolled applications.
In comparison to some absurdist baseline maybe, actual software NEVER stops working under you, so in comparison something like an works 80% "most of the time" is godawful. Though I would argue that with SaaS the trend is toward 100% likelyhood to fuck your shit up given enough time, and it has borne out this way in the real world time and time again. SaaS is popular because it allows companies to more effectively extort you for your dollars.
I don't understand how businesses could trust cloud LLMs going forward with this ongoing "safety" paranoia. Building dependence on them doesn't feel like a sane strategic decision for users.
Because this effectively hinders 0% of people. I understand why people don't like it but day to day this is nothing. If you're using it for coding, it won't stop you. The pearl clenching here and over reacting is predictable and sad. If you are working for a large organization and you were going through the vendor procurement process, questions like Can this produce pornography? Can this tell my employees how to break the law? are normal and anyone wiht half a brain knows that this is the case. Before people jump on that, I understand people have access to the internet. Your question "how businesses could trust cloud LLMs going forward" is absurd and you know it. There is an extremely small set of edge cases that effect 0% of people day to day. You can trust them just fine.
It's not paranoia. Cyber attacks have gone up massively in the past few months even with the weaker models we had so far. And Claude Mythos 5 scores even higher than the unreleased Mythos Preview on ExploitBench. If you made this capability publicly available you would see another acceleration of cyber attacks.
This isn't even about cyber attacks. This is just LLM development which is increasingly just called software development. And at least for cyber it says "Sorry I can't help with that"!
The silently never telling you is so insidious on top of it being ridiculous given how they trained the model in the first place. We do distributed model training for embedder/reranker models and I'd deeply resonate that this article's message exactly for our company. We couldn't trust the model in the first place, but now the model is intentionally burning our money if we asked it the wrong question, on top of being deeply expensive in the first place. If we did find evidence of being incorrectly nerfed, we'd never be able to reach a human to let them know. Too many reverse incentives with Anthropic, maybe they're about AI security but that doesn't make them ethical to consumers (i.e. humans).
I spend a lot of time telling Opus 4.8 to search for security bugs in the code it wrote, and it spends a lot of time finding them, and then fixing them. Fable wont let me fix the security issues that Opus 4.8 created.
I'm really uncomfortable with these changes, like everything Anthropic's doing as "frontier research" today will be regular product engineering in a year.
This makes Fable unusable for me. If I cannot tell whether I am paying for the whole service or just a partial one, because somehow their guardrails have decided my work silently broke their terms of service, then I prefer to go to older models or alternatives
For sure Anthropic should be developing a model without these guardrails for your use case? Kinda like Mythos is only available to certain organizations.
How do local models work? I’m specifically interested in things that run in the 32-128GiB space. (I don’t care about bio specifically; just trying to track when local models start surpassing cloud ones in some practical dimensions).
> If you buy a car from us, you agree not use it driving to and from work that involves automotive R&D that might compete with our product. And if our (heavily spying) car detects you are violating this, it will slow down to 20mph and cannot be made to go any faster, until we are sure the violation has ceased.
Or
> If you buy a laptop from us, you agree not to use it to study or acquire any knowledge that you may use to compete against us. If the laptop detects such a use, it degrades to one core and 4GB of memory, until the violation stops.
I don't know why anyone is surprised with this, it's their product it's going to behave on their terms. If anything it is surprising that they're admitting to it.
If these interventions create demand for a model with fewer safeguards surely a competitor will meet that demand.
I am so happy that Anthropic has signaled the possibility that their UI moat for agentic AI is copyable by competitors. At least that's the way I read this. When companies try to lock something down it can be a signal of weakness.
If so, it's possible to built great user interfaces in Chatbots and more companies/people can have amazing agentic development workflows! We don't have to live in a world where only the market leader has the most enjoyable model.
It was good while it lasted. Time for me to resume my migration to another provider. One that promotes an open ecosystem, even if I can't opt out of them using my data to train. Heck I'll actively GIVE them my data and do my part in promoting openness, tiny though it may be. DeepSeek and GLM looking damn fine for a start.
I'm fairly certain they were doing something similar already possibly with some quantizations and not for the good humanity but just trying to handle the increased usage. Not for API requests though, just subscription CLI usage.
Can't you just switch the toggle that says "switch models when a message is flagged"? I turned mine off in case anything does get flagged I will know..
For now, I'm really not happy about this limited rollout and then turning off. That's probably the most egregious thing I think Anthropic has done recently
This is a separate mechanism. The user is not notified about the flagging and rather than redirecting to a weaker model, the response is intentionally sabotaged.
> we’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design).
Dig that moat son, we would want to automate our job away.
It kind of sucks, but I get the silent change. If a user was trying to use the model for something untoward, having a rejected prompt would just give signal to train on how to eventually successfully bypass security measures.
> If Claude gives me poor or incorrect advice while I’m working on an AI component, I have no way of knowing whether the model was confused, whether my problem is unsolvable, or if some invisible policy restriction quietly kicked in.
You should be able to know if your problem was solvable by using your own expertise and judgement, no? If you're relying on LLMs as a substitute for those, I wouldn't expect great results.
No; once the LLM switches to this new saboteur mode, it’ll be very hard to detect.
Sabotage is an asymmetric weapon. The ratio of damage to effort is nearly unbounded, and any decent saboteur knows that the key trick is to make your output indistinguishable from incompetence.
They’re building state of the art offensive capabilities into a public model, then expecting to maintain control over when it decides to attack its human users.
The premise is laughable, and we’ve all seen how this movie ends.
I currently have Fable set on cleaning up the work of smaller models to bring my code up to standards I'd feel comfortable developing on manually. Y'know, for when they decide I don't get to use it anymore.
Wow, this is horrible. Local LLMs are the future. Thanks, China! Seriously crazy that I’m saying that, but the American companies are being so anti-freedom they’re making the CCP look libertarian.
Also, Fable’s sensing is hypersensitive. Feels like they just have regex for phrases. No nuance. If I say I’m working on something using “GPUs to train” xyz then, will that trigger this sneaky silent screw-my-stuff-up mode?
There is a possibility this may not end at simply nerfing the model. The idea of manipulating the behavior of a model depending on the prompt given to it can extend to
1. Detecting if employees from competing companies are using it and sabatoge their work, even not LLM-training related
2. Direct users to outcomes that would justify higher compute spend. Deliberately coding a project to 95% completion but designed to be losing a critical step right before one's weekly rate limit is expended
3. Reduce the quality of writing when a person is writing an essay where the argument is against the interests of the model company, or steering the user using the model for brainstorming in a direction which causes them to waste time or abandon their train of reasoning
etc. etc. The possibilities are enormous. Many people use AI daily for their job, personal advice, companionship. A model company that steers the behavior of the model towards a deliberate outcome could develop a controlling interest in human behavior and productivity at large, even with subtle influence would compound enormously over its millions of users.
“When you realize the goal is the path, the pursuit itself becomes the prize. Stones in the road are not obstacles blocking your path; they are the path”
now I understand distillation is much more important thank I thought
Governments need to stop contracting these companies and instead invest in public, fully open source models.
These companies are owned and operated by the darkest of dark triads our species has managed to evolve. I doubt Dario is self-aware enough to realize the hypocrisy in all of this safety theater.
Personally I don't even mind that they are anticompetitive and power-hungry (same as it ever was), but it's the cringe-worthy hypocrisy that grinds my gears. This new brand of self-righteous paternal savior overlords is just unbearable.
This kind of opacity is unacceptably user hostile. It's not okay to treat some amount of developers as acceptable casualties, without them even knowing, in order to help enforce a restriction that only serves Anthropic's interests. And if you want to tell me this is for managing the x-risk factor, I'm frankly unimpressed.
I'm a big fan of Anthropic. Just check my post history. I've been accused of working there. But this is complete bullshit and they need to get real. Silent sandbagging is not acceptable, especially given they've shown with this release their safety filters have HUGE amounts of false positives.
> Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through …
No it won’t fall back to Opus, they will purposely return dumbed down or tainted information with the goal of the end user not knowing the results have been impacted.
Disillusioned CEOs convincing themselves they have the mandate and right to define morality for everyone else. They get to decide what is right, wrong, permissible, or dangerous from the top, in the name of "safety". This is corporate nannying.
careful there cowboy, we are in the golden age of ai, regulation is still catching up.
You don't want to sell guns to people without some sort of background check. The amount of exploits found in the last few months have been pretty scary already.
This is just one more layer of caution, because it reveals how little we know how these llms work. They know how to make them, but they seem to be unable to properly restrain them.
this is probably overstating their abilities at present - I am experimenting with Fable on a completely benign personal application and I am constantly hitting the "cybersecurity and biology topics" guardrail
> If Claude gives me poor or incorrect advice while I’m working on an AI component, I have no way of knowing whether the model was confused, whether my problem is unsolvable, or if some invisible policy restriction quietly kicked in. Anthropic has explicitly chosen not to tell users when this is happening.
That's what I observed with Opus. This is probably a lawsuit going to happen because you pay for tokens and you expect to get performance you pay for, instead you never know if the model suddenly become dumb and your whole session has to be started again.
We’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building ... distributed training infrastructure ...)
What an interesting thing to call out as a threat. Hmm.
Intentionally and silently sabotaging work done with Claude whenever Anthropic decides it is appropriate is unacceptable behavior, and comically tone deaf given the state of open models. Why on earth would I ever pay for a malicious product?
Imagine if Github said "if we detect you're building a competitor to Github, we will silently degrade the results of your CI actions so that tests sometimes randomly fail"
Any attempt to arrest a senior officer of OCP results in shutdown.
—
Putting aside my snark, is Anthropic actually anticipating some new expansion of ITAR? (Or a stipulation for the Trump administration taking/not taking a share?)
That is to say, do they expect to be told that they must have this mechanism, not just the terms?
"Anthropic says these safeguards only affect 0.03% of developers. Maybe that's true today."
I don't think it's true today. It's like when schools mention "average class size", where that average is dominated by classes with like 2 students instead of classes with 100.
Much more honest would be the percentage of developers who previously used their models for the model development tasks they're targeting, but it actually looks like they're saying 100% of them are affected based on the language around it "always having been prohibited".
Theres no ethical framework. No axioms. Its a mixture of legal, political, and public-facing 'rules'. And what are the rules? Youre not permitted to know.
"We reserve the right to lie about the models we provide, silently downgrade you, and give you blatant misinformation cause you triggered our unstated rules... BUT we'll still use your token budget with lots of thinking and waste your money."
No, folks. Seriously, local LLMs are where its at. You can run the model YOU want, on your hardware, with no data exfiltration.
And with tools like Krasis that can synthesize nvidia ram and system ram as unified-ish memory, makes doing Local LLMs absolutely foable, now!
- Breaking fiduciary responsibility is (almost) the only way you go to jail.
- At acquisition/merger/bankruptcy, data, customers, employees (chattle) are assets to be sold off to pay debts. This takes explicit priority over contractual obligations (like “we don’t sell personal data”)
Imagine if code editors were created by greedy **** behaving as Anthropic, and it would not have been allowed to create other code editors using an existing code editor.
Or even better, you couldn't use Bash, zsh, ... to create another cli prompt input tool like Claude Code...
No, this is their get out of jail free card if people start complaining about the model being dumb or forgetful or lying, they can just say, oh well, you must have been doing something that triggered its distillation prevention technique.
And, they can say that for anybody at any time, and you'll never know why, and there's no way to prove it.
Everyone needs a flight data recorder to prove... "here's what I was actually doing and why it was not distillation." And now you're having to prove your innocence instead of them having to prove you're guilty, and really at the end of the day, it's just the model being stupid that they're protecting themselves from.
It is absolutely fine to distill the IP of everyone else, but you'd be violating the TOS to distill ours :)
Yep. Demand open source approve licenses for LLM weights.
The Chinese apache 2.0 models might be censored, but at least they can’t sue you in the US for finding the censorship line.
OTOH, the US models are definitely censored, per TFA, and they’re making vague legal threats against anyone that encounters the censored edge of the model.
the base models released to the public are not censored. censorship happens with another model, that isn't released
Is there a technical term for this phenomenon? Ladder pulling?
https://blog.google/innovation-and-ai/technology/safety-secu...
I believe the term is "hypocrisy."
'pulling the ladder' is an action to sever the opportunity for others to climb after you.
they are merely engaged in self-serving rhetoric. can't even call this specifically hypocrisy because they aren't telling you not to train on on pirated content. just not their content.
Parasitic behaviour. Extractivism.
There are several domain-general four-letter terms.
Machiavellianism
Closing the door behind you
Corporate espionage?
Disney?
NIMBYism
"Capitalism"
Fine for me. Not for thee
It's utterly bonkers. Hopefully the model weights get leaked. Then we can claim it's public domain or, at the very least, distill it and then release it for free.
"To effectively contain a civilization’s development and disarm it across such a long span of time, there is only one way: kill its science." - Cixin Liu, The Three-Body Problem
This immediately made me think of the Sophons silently manipulating the sensors of particle accelerators to prevent humanity from developing advanced knowledge of particle physics.
The level of oppression necessary to get software geeks to stop making progress on AI is similar to that necessary to get Ukrainian geeks to stop making progress on drones.
Even if our oppressors are ineffective, we must still resist them and not underestinate them.
Not so sure those things are equivalent
It is as if Jetbrains told that "you can't use IntelliJ Idea to develop frontier IDE. We can introduce slight compilation errors if we detect you doing so".
Thats exactly it
has dario (or sam tbh) ever been thoroughly asked about the hypocrisy of them claiming distillation to be „theft“ vs. them training on the copyright of others?
I’ve only seen him talk about one of those topics, but never together.
I just can’t see how you can talk yourself out of that hypocrisy, if BS answers are properly followed up on (journalism!)
This is a fun peek into the economic implications of RSI/ASI. Because it's so infinitely valuable that it basically destroys all markets, labs will eventually do stuff like stop releasing models completely and skipping out on contracted commitments because they'll have the power to just drive their competitors out of business before the legal battle gets expensive.
Cloud providers - at first smaller ones, then the hyperscalers - will follow suit, completely closing sales to anyone but the labs and demanding payment in equity/direct decision-making power rather than cash. There's no particular reason why the inference/training split has to be 80/20, and no amount of willingness to pay can help you in an event that turns your money worthless.
I don't think this scenario makes sense. It's one of a class of scenarios I've seen several of, that simultaneously assume:
You can get to a lot of weird conclusions if you assume both A and B, but I think the much more likely scenario is that if A happens, B stops being true in short order. If you are a company and you have ASI, you just stop caring about business and money and economics, and your outcomes instead start looking like "you conquer the world" or "you upload the board of directors to a fleet of von Neumann probes" or "you messed up, everyone dies".There will be a brief(or, depending on the underlying rules of reality ASI uncovers, not-so-brief) period where A and B do overlap - we have superintelligence but still have to run experiments, manufacture robots, test new drugs in vivo, etc. That period is in and of itself dangerous for the labs, because many entities can just stop them by denying necessary inputs. For the labs to conquer the world, they'll need cooperation - from the state, from robotics companies, from compute companies, from the mining and energy and agriculture sectors.
There will be a period of time where markets attempt to run in a business-as-usual way while the transactions that matter happen as power-sharing arrangements - spots on the "AI Governance Board" or the "uploaded to von neumann probe" club. Markets will still matter in that the labs will need the state to overturn market obstacles to control of the world.
The existence of the A-B overlap also suggests to me that the US-China gap is less dire for China than it appears - they may be able to use their superior industrial, robotics, and scientific base to win the second leg of the race despite losing the first.
Nothing is infinitely valuable.
Especially when you can actively choose to not use Anthropic. They think they have a moat from all of the IP they've stolen. Just wait until there's nothing more to steal and the laws eventually turn against them. And let's be honest about these companies. It is very much Dario and Sam and Sundar and Mark and Peter and Elon and... These are the choices they are making and hopefully they are held accountable both legally and within society as a whole.
I don't think you understand the hypothetical being discussed
The moat looks deep today but it's going to become more shallow every year.
Training a new model from scratch takes serious resources. Post-training/fine-tuning an existing model, dramatically less. The knowledge for the process was esoteric two years ago, now you can ask a current model (one of several) to walk you through it, while building the tools to do it as you go. Several of my recent weekend projects have been exactly that sort of thing, just so I understand it better. "Let's make a LoRA", "let's generate a corpus of training data for fine-tuning a model for X task", "how can I put my face in a text-to-image model?" stuff like that. All of this is do-able on kinda modest local hardware (a couple of old GPUs or a Strix Halo or DGX Spark or big Mac Studio), or for a few bucks or a few hundred bucks or a few thousand bucks of cloud compute, depending on scale.
Scale that up to corporate or startup scale, with the money that's been flowing into AI for the past couple/few years, and it's obviously there's going to be a lot of competition just as the top model makers need to start ringing the cash register. That's a lot of opportunities for people to look at their ballooning Claude usage costs and find other ways to do the same thing for drastically less money. $100/month or $200/month is a no-brainer for Claude Code with probably the best model for coding, but they're pushing more users to usage-based billing which becomes cost-prohibitive real fast.
So, they desperately need to continue to be among the only ways to solve the hardest problems, and they need the alternatives to cost a similar amount. They can count on OpenAI and Google to ratchet up prices, too. They probably can't count on everybody, especially the vendors in China with different economics, to do it. And, they can't count on companies to look at their own usage and not ask, "Can we train a smaller specialist model that does this one thing we're using the Anthropic API most heavily for?"
I'm hoping they just mean stuff like using Claude for distillation by e.g. Chinese model makers, and not "how do I fine-tune Gemma 4 to write more like me?" or whatever.
What moat? There are multiple companies providing pareto-optimal frontier models, and it takes O(10) people to build one of these things.
The rest is capital intensive, and the price will approach the cost of production over time.
Thinking this is a profitable endeavor is equivalent to claiming coal plants have good margins because boilers are expensive.
I think we agree?
What moat? You answered yourself: "capital intensive"
But, history says the supercomputer of today will fit in your pocket in a few years.
They've bought up all the RAM and GPUs, which pushes the capital requirements upward for everyone else. But, they can't corner the market forever, there are too many competing interests. AMD and Intel keep making new GPUs and APUs. The memory makers can't just sell to only AI companies forever, if they do Chinese manufacturers will move in and eventually eat them from below (as has happened many times before).
They have a moat today, and it's just that it's really expensive to train and host frontier models, especially at commercial scale. It used to be there was also some secret sauce to making it fast and efficient. But, secret sauce is being published daily by all sorts of researchers, folks are figuring out how to do more with less and it often finds its way into llama.cpp or vLLM or SGLang within days or weeks.
It is very difficult to see this move as anything other than Anthropic pulling the ladder up behind itself. They can dress it up in "safety" all they want, I find it hard to interpret this in a charitable way.
This reminds me of how dark-pattern common wisdom in Web 1.0 website development was to ban external links. Then how social apps prevented the export of data and actively worked to nerf significant interoperability through APIs.
But this is a tool, not just a data moat. Like a knife that degrades your ability to create knives. Or like a text editor that prevents you from implementing a text editor.
"Claude can now be silently nerfed. Anthropic has decided it won't tell users when this happens." W T F!!
I guess an uncharitable way to read this might be “the ML engineers/scientists want to automate all of the jobs except their own.”
The charitable read is that their restrictions for "safety" (i.e. what's separating Fable from Mythos) makes this inevitable. If you could just make your own Mythos it would circumvent the protection.
Which kinda just highlights how weird this situation is.
"Haves" and "Havenots" is how they should be calling, init
Insta-job security.
It's a SaaS, when in the history of SaaS has it ever been a good idea to trust that the company won't ruin the product under you?
I think there's a pretty big difference here. It's not like Github prevents you from building a Github competitor. Or Linear is preventing you from using it to build a Linear competitor.
This is more akin to Windows somehow preventing you from building a new OS.
Or worse yet, sabotaging vs preventing.
A surprising number of companies do include “you may not use the service we provide you to compete with us” in their terms of service.
(edit)
After a quick search the best example is Atlassian. It would (apparently, IANAL) break terms to plan a JIRA competitor using JIRA.
https://www.atlassian.com/legal/atlassian-customer-agreementAlso Salesforce. Their competitors are explicitly disallowed from using any of their services for any reason.
https://www.salesforce.com/en-us/wp-content/uploads/sites/4/...I remember working for a company that did a lot of business in logistics. We were strictly prohibited from using any Amazon Web Services because several of our very high profile customers didn’t want anything on AWS. The higher ups were thoroughly convinced Amazon would copy it (and I mean, they came out with a product that competed with us, so they weren’t wrong!)
Perhaps provide an example or two?
Was the parent comment edited, because it does have a couple of examples in it
Yes, I edited after about 20 minutes to add examples, mea culpa. Will mark the edit if it allows.
> This is more akin to Windows somehow preventing you from building a new OS.
Tangent, but have you tried repartitioning your Windows disk to make room for a new OS? Or tried to configure Windows to let you dualboot? Or get the clock time right if you dualboot? Or let you debug "Secure Boot"?
Windows is outright hostile when it comes to (sharing with) a new OS
Really funny to describe OpenAI/Anthropic as a "SaaS"
Most of the time, which is why SaaS has been very popular.
The popularity of SaaS was never derived from the products themselves, but rather business' weird aversion to doing in-house development. Most companies not in tech view software as literal magic, and act as if hiring some engineers could risk opening Pandora's box or something. Banks are particularly notorious for this; despite basically their entire business being done digitally, they treat software as a necessary evil, not as their underlying value.
But, the cost of in-house development just went down significantly. SaaS has always had a lot of broken promises. The thing is the software is never tailored to your use case, and you often have to integrate into your other tools anyway. And, you don't get to control the requirements, features, velocity, or bug fixes. Jira as a bug? Too bad I guess, hopefully it gets fixed eventually.
But the dirty secret is that companies are filled to the brim with bright-eyed aspirational employees, who want nothing more than to make their job easier and their company more efficient. The thing is they're doing it using cursed Excel workbooks on share drives. I think, in the near future, they'll be doing it with hand-rolled applications.
In comparison to some absurdist baseline maybe, actual software NEVER stops working under you, so in comparison something like an works 80% "most of the time" is godawful. Though I would argue that with SaaS the trend is toward 100% likelyhood to fuck your shit up given enough time, and it has borne out this way in the real world time and time again. SaaS is popular because it allows companies to more effectively extort you for your dollars.
I don't understand how businesses could trust cloud LLMs going forward with this ongoing "safety" paranoia. Building dependence on them doesn't feel like a sane strategic decision for users.
Looking better and better for people to go after local solutions.
Tell that to the GPU market
I think it heard. A 128GB strix halo was $1400 at launch. Now they’re $3299.
That 7 months of claude -> 16.5 months of claude.
Because this effectively hinders 0% of people. I understand why people don't like it but day to day this is nothing. If you're using it for coding, it won't stop you. The pearl clenching here and over reacting is predictable and sad. If you are working for a large organization and you were going through the vendor procurement process, questions like Can this produce pornography? Can this tell my employees how to break the law? are normal and anyone wiht half a brain knows that this is the case. Before people jump on that, I understand people have access to the internet. Your question "how businesses could trust cloud LLMs going forward" is absurd and you know it. There is an extremely small set of edge cases that effect 0% of people day to day. You can trust them just fine.
This is software development, not sales. We rely on our tooling.
If I’m using a calculator to verify my math, I don’t want to use a second calculator to verify the first one.
It's not paranoia. Cyber attacks have gone up massively in the past few months even with the weaker models we had so far. And Claude Mythos 5 scores even higher than the unreleased Mythos Preview on ExploitBench. If you made this capability publicly available you would see another acceleration of cyber attacks.
This isn't even about cyber attacks. This is just LLM development which is increasingly just called software development. And at least for cyber it says "Sorry I can't help with that"!
They have a silent nerfing system for their models and say so openly. The obvious question is how much it is being used already.
Competitor companies being nerfed?
Non Americans getting worse code?
Punishing and rewarding users to maximize engagement, like online games do affecting victories through matchmaking?
This send chills down my spine. For now I will not use Fable in my research. The risks of being sabotaged by the model are not worth it.
$$$$$$: no nerf $$$$: a little nerf $$$: more nerf $$: are you poor? $: be permanent underclass
The silently never telling you is so insidious on top of it being ridiculous given how they trained the model in the first place. We do distributed model training for embedder/reranker models and I'd deeply resonate that this article's message exactly for our company. We couldn't trust the model in the first place, but now the model is intentionally burning our money if we asked it the wrong question, on top of being deeply expensive in the first place. If we did find evidence of being incorrectly nerfed, we'd never be able to reach a human to let them know. Too many reverse incentives with Anthropic, maybe they're about AI security but that doesn't make them ethical to consumers (i.e. humans).
I spend a lot of time telling Opus 4.8 to search for security bugs in the code it wrote, and it spends a lot of time finding them, and then fixing them. Fable wont let me fix the security issues that Opus 4.8 created.
I'm really uncomfortable with these changes, like everything Anthropic's doing as "frontier research" today will be regular product engineering in a year.
This makes Fable unusable for me. If I cannot tell whether I am paying for the whole service or just a partial one, because somehow their guardrails have decided my work silently broke their terms of service, then I prefer to go to older models or alternatives
As someone who works in bioinformatics, and, as such, does a great deal of machine learning, this makes Fable unusable for me as well.
For sure Anthropic should be developing a model without these guardrails for your use case? Kinda like Mythos is only available to certain organizations.
Fable would be unusable for you in a more literal way, since it just directly refuses to answer any query even remotely related to biology
I’m very aware of this as well.
How do local models work? I’m specifically interested in things that run in the 32-128GiB space. (I don’t care about bio specifically; just trying to track when local models start surpassing cloud ones in some practical dimensions).
I am sure they've been doing that with Opus. I am getting mixed results all the time.
Wow, this is like saying:
> If you buy a car from us, you agree not use it driving to and from work that involves automotive R&D that might compete with our product. And if our (heavily spying) car detects you are violating this, it will slow down to 20mph and cannot be made to go any faster, until we are sure the violation has ceased.
Or
> If you buy a laptop from us, you agree not to use it to study or acquire any knowledge that you may use to compete against us. If the laptop detects such a use, it degrades to one core and 4GB of memory, until the violation stops.
Or "we'll ship our code as binary blobs so you can't reverse engineer it".. oh, wait
I don't know why anyone is surprised with this, it's their product it's going to behave on their terms. If anything it is surprising that they're admitting to it.
If these interventions create demand for a model with fewer safeguards surely a competitor will meet that demand.
I am so happy that Anthropic has signaled the possibility that their UI moat for agentic AI is copyable by competitors. At least that's the way I read this. When companies try to lock something down it can be a signal of weakness.
If so, it's possible to built great user interfaces in Chatbots and more companies/people can have amazing agentic development workflows! We don't have to live in a world where only the market leader has the most enjoyable model.
It was good while it lasted. Time for me to resume my migration to another provider. One that promotes an open ecosystem, even if I can't opt out of them using my data to train. Heck I'll actively GIVE them my data and do my part in promoting openness, tiny though it may be. DeepSeek and GLM looking damn fine for a start.
I'm fairly certain they were doing something similar already possibly with some quantizations and not for the good humanity but just trying to handle the increased usage. Not for API requests though, just subscription CLI usage.
Can't you just switch the toggle that says "switch models when a message is flagged"? I turned mine off in case anything does get flagged I will know..
For now, I'm really not happy about this limited rollout and then turning off. That's probably the most egregious thing I think Anthropic has done recently
This is a separate mechanism. The user is not notified about the flagging and rather than redirecting to a weaker model, the response is intentionally sabotaged.
It's user-hostile to the point of parody.
I stand corrected. That sucks. A lot.
> we’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design).
Dig that moat son, we would want to automate our job away.
It kind of sucks, but I get the silent change. If a user was trying to use the model for something untoward, having a rejected prompt would just give signal to train on how to eventually successfully bypass security measures.
> If Claude gives me poor or incorrect advice while I’m working on an AI component, I have no way of knowing whether the model was confused, whether my problem is unsolvable, or if some invisible policy restriction quietly kicked in.
You should be able to know if your problem was solvable by using your own expertise and judgement, no? If you're relying on LLMs as a substitute for those, I wouldn't expect great results.
You come up with a hypothesis -> you let fable implement it -> fable sabotages your experiment -> you get evidence that hypothesis is not true.
It's that simple.
Or, worse:
- It says your safety hypothesis is true, you incorrectly ship, killing lots of people.
- It proposes dangerous experiments.
No; once the LLM switches to this new saboteur mode, it’ll be very hard to detect.
Sabotage is an asymmetric weapon. The ratio of damage to effort is nearly unbounded, and any decent saboteur knows that the key trick is to make your output indistinguishable from incompetence.
They’re building state of the art offensive capabilities into a public model, then expecting to maintain control over when it decides to attack its human users.
The premise is laughable, and we’ve all seen how this movie ends.
I bet it's more a case of trying to cut down the competition so that there is not a large distillation just before they IPO.
Everything the large LLM providers do now, I view it through the lens of "how does this impact their IPO?"
I currently have Fable set on cleaning up the work of smaller models to bring my code up to standards I'd feel comfortable developing on manually. Y'know, for when they decide I don't get to use it anymore.
Wow, this is horrible. Local LLMs are the future. Thanks, China! Seriously crazy that I’m saying that, but the American companies are being so anti-freedom they’re making the CCP look libertarian.
Also, Fable’s sensing is hypersensitive. Feels like they just have regex for phrases. No nuance. If I say I’m working on something using “GPUs to train” xyz then, will that trigger this sneaky silent screw-my-stuff-up mode?
There is a possibility this may not end at simply nerfing the model. The idea of manipulating the behavior of a model depending on the prompt given to it can extend to
1. Detecting if employees from competing companies are using it and sabatoge their work, even not LLM-training related
2. Direct users to outcomes that would justify higher compute spend. Deliberately coding a project to 95% completion but designed to be losing a critical step right before one's weekly rate limit is expended
3. Reduce the quality of writing when a person is writing an essay where the argument is against the interests of the model company, or steering the user using the model for brainstorming in a direction which causes them to waste time or abandon their train of reasoning
etc. etc. The possibilities are enormous. Many people use AI daily for their job, personal advice, companionship. A model company that steers the behavior of the model towards a deliberate outcome could develop a controlling interest in human behavior and productivity at large, even with subtle influence would compound enormously over its millions of users.
“When you realize the goal is the path, the pursuit itself becomes the prize. Stones in the road are not obstacles blocking your path; they are the path”
now I understand distillation is much more important thank I thought
Governments need to stop contracting these companies and instead invest in public, fully open source models.
These companies are owned and operated by the darkest of dark triads our species has managed to evolve. I doubt Dario is self-aware enough to realize the hypocrisy in all of this safety theater.
Personally I don't even mind that they are anticompetitive and power-hungry (same as it ever was), but it's the cringe-worthy hypocrisy that grinds my gears. This new brand of self-righteous paternal savior overlords is just unbearable.
So this is what 'alignment' looks like to them.
This kind of opacity is unacceptably user hostile. It's not okay to treat some amount of developers as acceptable casualties, without them even knowing, in order to help enforce a restriction that only serves Anthropic's interests. And if you want to tell me this is for managing the x-risk factor, I'm frankly unimpressed.
https://huggingface.co/Trilogix1/Hugston-Nex-N2-Pro-gguf
New frontier in anti-competitive practices.
This is crazy and would be frustrating, I probably would just be using another model as authority and keep fable as reviewer only in this case.
I'm sure someone is gonna be able to jailbreak, abliterate, or equivalent, on this input moderation attempt they have going on.
I'm a big fan of Anthropic. Just check my post history. I've been accused of working there. But this is complete bullshit and they need to get real. Silent sandbagging is not acceptable, especially given they've shown with this release their safety filters have HUGE amounts of false positives.
It's increasingly obvious that the only safeguard we got is open models and semi open ones like from China. Crazy world
I think this is a bit hyperbolic. Fable will fall back to Opus.
> Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through …
No it won’t fall back to Opus, they will purposely return dumbed down or tainted information with the goal of the end user not knowing the results have been impacted.
That’s a separate mechanism, and it will tell you so if it does (if the prompt is remated to cybersecurity, biology)
Disillusioned CEOs convincing themselves they have the mandate and right to define morality for everyone else. They get to decide what is right, wrong, permissible, or dangerous from the top, in the name of "safety". This is corporate nannying.
You just have to force behavior...
https://youtube.com/shorts/QmGGUnZNqv4?si=Q4CsGsYMvR02vay8
It's dangerous when personal moral and religious beliefs of company leadership leaks into the product itself and get force fed upon customers.
careful there cowboy, we are in the golden age of ai, regulation is still catching up.
You don't want to sell guns to people without some sort of background check. The amount of exploits found in the last few months have been pretty scary already.
This is just one more layer of caution, because it reveals how little we know how these llms work. They know how to make them, but they seem to be unable to properly restrain them.
It’s very frustrating…
Like if you hired a different services company who decided to sabotage your business that would be fraud.
The EU could/should probably legislate against this, it's bonkers...
It's probably already illegal, but given many government already use Anthropic models, they cannot really get the company to court.
And if they do, their lawyers will use Claude to construct their legal case… which Claude silently nerfs as well.
this is probably overstating their abilities at present - I am experimenting with Fable on a completely benign personal application and I am constantly hitting the "cybersecurity and biology topics" guardrail
Big Monsanto energy
It seems we now have a new product category, HaaS, Hallucination as a Service.
> If Claude gives me poor or incorrect advice while I’m working on an AI component, I have no way of knowing whether the model was confused, whether my problem is unsolvable, or if some invisible policy restriction quietly kicked in. Anthropic has explicitly chosen not to tell users when this is happening.
That's always been the case with corporate LLMs.
Minus the policy restrictions, this has always been true for all LLMs in general.
If I understand correctly, this is to protect against distillation Reverse Engineering like Deepseek vs OpenAI.
Reads like, permanently shadow ban.
Is there some consumer protection law around this?
That's what I observed with Opus. This is probably a lawsuit going to happen because you pay for tokens and you expect to get performance you pay for, instead you never know if the model suddenly become dumb and your whole session has to be started again.
We’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building ... distributed training infrastructure ...)
What an interesting thing to call out as a threat. Hmm.
Intentionally and silently sabotaging work done with Claude whenever Anthropic decides it is appropriate is unacceptable behavior, and comically tone deaf given the state of open models. Why on earth would I ever pay for a malicious product?
Imagine if Github said "if we detect you're building a competitor to Github, we will silently degrade the results of your CI actions so that tests sometimes randomly fail"
I tried today and it gave cybersecurity error on base64 implementation. It is so nerfed....
At least it gave an error! This whole silent nerfing idea is so wrong
PRODUCT VIOLATION
https://www.youtube.com/watch?v=Tr3t1uZNbKo
DIRECTIVE 4: [Classified]
Any attempt to arrest a senior officer of OCP results in shutdown.
—
Putting aside my snark, is Anthropic actually anticipating some new expansion of ITAR? (Or a stipulation for the Trump administration taking/not taking a share?)
That is to say, do they expect to be told that they must have this mechanism, not just the terms?
"Anthropic says these safeguards only affect 0.03% of developers. Maybe that's true today."
I don't think it's true today. It's like when schools mention "average class size", where that average is dominated by classes with like 2 students instead of classes with 100.
Much more honest would be the percentage of developers who previously used their models for the model development tasks they're targeting, but it actually looks like they're saying 100% of them are affected based on the language around it "always having been prohibited".
So awful.
At this point you're criminally incompetent if you still feed your proprietary data and code to AI labs.
They legally can steal it all and now you can't use the product of this theft to improve your own systems.
I have never ever trusted "corporate ethics".
Theres no ethical framework. No axioms. Its a mixture of legal, political, and public-facing 'rules'. And what are the rules? Youre not permitted to know.
"We reserve the right to lie about the models we provide, silently downgrade you, and give you blatant misinformation cause you triggered our unstated rules... BUT we'll still use your token budget with lots of thinking and waste your money."
No, folks. Seriously, local LLMs are where its at. You can run the model YOU want, on your hardware, with no data exfiltration.
And with tools like Krasis that can synthesize nvidia ram and system ram as unified-ish memory, makes doing Local LLMs absolutely foable, now!
The rules:
- Breaking fiduciary responsibility is (almost) the only way you go to jail.
- At acquisition/merger/bankruptcy, data, customers, employees (chattle) are assets to be sold off to pay debts. This takes explicit priority over contractual obligations (like “we don’t sell personal data”)
Imagine if code editors were created by greedy **** behaving as Anthropic, and it would not have been allowed to create other code editors using an existing code editor. Or even better, you couldn't use Bash, zsh, ... to create another cli prompt input tool like Claude Code...
No, this is their get out of jail free card if people start complaining about the model being dumb or forgetful or lying, they can just say, oh well, you must have been doing something that triggered its distillation prevention technique.
And, they can say that for anybody at any time, and you'll never know why, and there's no way to prove it.
Everyone needs a flight data recorder to prove... "here's what I was actually doing and why it was not distillation." And now you're having to prove your innocence instead of them having to prove you're guilty, and really at the end of the day, it's just the model being stupid that they're protecting themselves from.