Decider v1.3 and Decider v.1.2 seem to have same benchmarks. However, V1.3 flash beats V1.2 flash by 4 points.
I am not particulary excited about this pattern/trend of having flash versions for decision models. The whole point of Decision models was to deliver faster ouput for high volume requests at much cheaper cost than Reasoning models. Since most reasoning models can and probably do outperform decision models already in accuracy but they are a lot slower and are pricier. Reasoning Models can already return structured ouput.
So your flagship model must be fastest and most accurate.
Unless it can handle some different form of input other than just image and text, I don't see the point of having both flagship and flash version for the same model.
It performs well on my local benchmark, and is priced competitively. Good timing too, considering that inception Labs is discontinuing their Mercury models API, which I was previously using. It seems plausible that the bidirectional attention performs better for decision models.
You seem to be confusing market capture for novelty. This almost certainly is not intended to be taken as some greenfield thing from perplexity, same goes for google
Well congrats.
Here are some thoughts not facts:
On AA Decision Index.
Decider v1.3 and Decider v.1.2 seem to have same benchmarks. However, V1.3 flash beats V1.2 flash by 4 points.
I am not particulary excited about this pattern/trend of having flash versions for decision models. The whole point of Decision models was to deliver faster ouput for high volume requests at much cheaper cost than Reasoning models. Since most reasoning models can and probably do outperform decision models already in accuracy but they are a lot slower and are pricier. Reasoning Models can already return structured ouput.
So your flagship model must be fastest and most accurate.
Unless it can handle some different form of input other than just image and text, I don't see the point of having both flagship and flash version for the same model.
Any ideas what decider models are actually good? There does not seem to be a strong competitive benchmark culture (as with LLMs) for whatever reason?
I've found this fairly accurate: https://huggingface.co/spaces/multimodalart/jev-decision-ind...
This blog post acted as a rabbit hole into some interesting stuff:
https://www.perplexity.ai/hub/blog/pii-trace-detecting-perso...
https://www.perplexity.ai/hub/blog/a-local-first-agent-for-p...
It performs well on my local benchmark, and is priced competitively. Good timing too, considering that inception Labs is discontinuing their Mercury models API, which I was previously using. It seems plausible that the bidirectional attention performs better for decision models.
Complete lack of creativity in the field of AI...
Here's new product, which is just our shitty version of this easily reproducible technology
What is this?
Same for Google and their decisions api
You seem to be confusing market capture for novelty. This almost certainly is not intended to be taken as some greenfield thing from perplexity, same goes for google
Perhaps, but if you're already inside GCP it's nice to have something easy to turn on.
Not just that, if your company approved and gets billed by X cloud, it's quite difficult to request another product from Y cloud
> version of this easily reproducible technology
so you'd prefer indefinite first-mover monopolies?
Does anyone really use such things for security?
I highly doubt it. Even if a company does internally, its highly unlikely they will reveal it publically.
One use case is live content moderation or streaming where reasoning models could be a lot slower and its hard to hardcode logic.