Polymarket says 80% chance of fable 5.2 release over the next 8 days, it feels pretty consistent to me with other release periods, where the existing models feel "dumbed down".
Livenerf is also using a subscription to run, not API, so it's apparently not a subscription thing. Maybe if you're unlucky you're served with a quantized model until they release fable 5.2 ?
I doubt Anthropic are quantizing their own models. Since effort levels are essentially a compute budget I wonder if they’re using this behind the scenes to dial it down when getting close to capacity. Thinking blocks are hidden so we wouldn’t be able to tell right?
From Anthropic's POV it would be better to sneakily nerf the effort level and have some people whine on reddit, than hit capacity and have potentially thousands of customers unable to use the service they're paying for.
It seems like pre-IPO companies need to balance "cost" vs. "optics" and it's one of the metrics they optimize.
So, launch at an "exceptional" state and then cut costs after some time following the release, then repeat to make it appear like lots of advancement is happening.
I have zero evidence. But anecdotally (and working with Claude every day) I see wild swings in Opus/Fables abilities day to day. It really does feel like they are twiddling knobs behind the scenes and hoping no one notices.
- They put the shock collar on it [Cyber] Your session has been paused - meaning I have to start a new session. That is also on Opus 4.8 now, just not quite as sensitive as on 5.5. (Defensive coding, defensive system configuration) In my opinion Claude is not permitted to write secure code or analyze ones existing code for bugs that may be security related. Instead of the shock collar, just report anything illegal I ask Claude to do to Palantir and other related platforms like everyone else does.
- They downgraded my 20x account to a 10x, so I downgraded it to the lowest one. I would have paid for Mythos if they would have permitted me to verify myself.
- I keep hitting context memory limits and Claude starts acting like it has dementia, accusing me of doing things that it did. Claude is not proactively compacting when it should, especially on max. It says max does not matter but in my experience it very much does.
I have not logged in for a couple weeks so I have no idea if the model itself has been nerfed. The feeling I got is that they want to focus on enterprise customers and would prefer individuals went away so I cleared out my data and moved my projects elsewhere that does not block me for wanting to secure my code. Perhaps that is their way of sending more individuals away? Their message has been received loud and clear.
Claude's security restrictions seem more like a CYA move than anything related to security. In particular, restrictions on analyzing artifacts uploaded or supplied by public or legitimate authenticated access to erxternal sites seem counterproductive as far as security is concerned.
I find the verification requirements particularly puzzling: low level requirements that amount to "use competing LLMs to spam reports to build cred", high level requirements that preclude detailed analysis of anything that flys below the radar of large actors.
Polymarket says 80% chance of fable 5.2 release over the next 8 days, it feels pretty consistent to me with other release periods, where the existing models feel "dumbed down".
funnily livenerf is showing the model to be slightly more intelligent these days: https://github.com/ninjahawk/livenerf
Livenerf is also using a subscription to run, not API, so it's apparently not a subscription thing. Maybe if you're unlucky you're served with a quantized model until they release fable 5.2 ?
You have to assume they detect benchmarking / testing and feed those clients consistent models.
I doubt Anthropic are quantizing their own models. Since effort levels are essentially a compute budget I wonder if they’re using this behind the scenes to dial it down when getting close to capacity. Thinking blocks are hidden so we wouldn’t be able to tell right?
From Anthropic's POV it would be better to sneakily nerf the effort level and have some people whine on reddit, than hit capacity and have potentially thousands of customers unable to use the service they're paying for.
> We trust them that what is behind the string "opus-5-5" is consistent and what we signed up for.
Do you run evals as a matter of course? Everyone I know that does shows high levels of variance across them under the best case.
That’s not to say that the labs don’t downgrade models at times, just that without any measurement it’s a non-interesting assertion.
It seems like pre-IPO companies need to balance "cost" vs. "optics" and it's one of the metrics they optimize.
So, launch at an "exceptional" state and then cut costs after some time following the release, then repeat to make it appear like lots of advancement is happening.
I have zero evidence. But anecdotally (and working with Claude every day) I see wild swings in Opus/Fables abilities day to day. It really does feel like they are twiddling knobs behind the scenes and hoping no one notices.
Why not use a distributed benchmark? Tokens are shared across volunteers; everyone gets informed of the nerf simultaneously. No one need be surprised.
I guess a new model release is right around the corner
For me Claude was nerfed when:
- They put the shock collar on it [Cyber] Your session has been paused - meaning I have to start a new session. That is also on Opus 4.8 now, just not quite as sensitive as on 5.5. (Defensive coding, defensive system configuration) In my opinion Claude is not permitted to write secure code or analyze ones existing code for bugs that may be security related. Instead of the shock collar, just report anything illegal I ask Claude to do to Palantir and other related platforms like everyone else does.
- They downgraded my 20x account to a 10x, so I downgraded it to the lowest one. I would have paid for Mythos if they would have permitted me to verify myself.
- I keep hitting context memory limits and Claude starts acting like it has dementia, accusing me of doing things that it did. Claude is not proactively compacting when it should, especially on max. It says max does not matter but in my experience it very much does.
I have not logged in for a couple weeks so I have no idea if the model itself has been nerfed. The feeling I got is that they want to focus on enterprise customers and would prefer individuals went away so I cleared out my data and moved my projects elsewhere that does not block me for wanting to secure my code. Perhaps that is their way of sending more individuals away? Their message has been received loud and clear.
Claude's security restrictions seem more like a CYA move than anything related to security. In particular, restrictions on analyzing artifacts uploaded or supplied by public or legitimate authenticated access to erxternal sites seem counterproductive as far as security is concerned.
I find the verification requirements particularly puzzling: low level requirements that amount to "use competing LLMs to spam reports to build cred", high level requirements that preclude detailed analysis of anything that flys below the radar of large actors.
I've noticed this too, since about Wednesday.
Oh man, can't wait for open-source to catch up...
I noticed the same at work using Opus 5.5 on medium which was the sweet spot. It’s now just not great.