This kind of thing just makes me think Apple will get to a point where they have good enough local models and good enough harnesses for doing things, and most normal people will just use them… Does the LLM become another interface to computing?
i believe that for most people on the street, for most tasks, a Chat GPT 3.5 era LLM is sufficient enough. sprinkle in tool calling and other things, and that becomes enough. if you can prioritize that level of a model on-device (baking it in etc), then you can bifurcate AI users between those unwilling to pay and those who are willing to pay A LOT for frontier model performance.
Built to work with lmstudio, one of the leading easy to use local model servers. LMStudio is the closest to plug-and-play without sacrificing play that I've seen; a harness that works well with it is nothing to sniff at. Its not earth shattering either.
Wouldn't most opensource harnesses work with lmstudio? I assume it has an "openai" style chat API like every other model provider? What's special about it vs langchain deep agents or pi or pydantic-ai?
I guess it lives or dies by the harness quality then - on open models run locally by plug and players and models that fit onto peoples laptops that is going to be quite the handicap to overcome.
I run lmstudio personally with a range of harnesses (open and closed) and can't say there is that much of a leap to getting everything talking https://lmstudio.ai/docs/integrations
If you want to use local models, it's more ergonomic than fussing with GGUFs or using LM Studio as a server host and setting up the link to an agent yourself. Although, the model selector is the same as with LM Studio itself which can be overwhelming if you don't know what to look for.
I'm interested in your thought process. How did you get from his initial statement opening of "A friendly reminder ..." to thinking that this was a scandal?
This kind of thing just makes me think Apple will get to a point where they have good enough local models and good enough harnesses for doing things, and most normal people will just use them… Does the LLM become another interface to computing?
i agree.
i believe that for most people on the street, for most tasks, a Chat GPT 3.5 era LLM is sufficient enough. sprinkle in tool calling and other things, and that becomes enough. if you can prioritize that level of a model on-device (baking it in etc), then you can bifurcate AI users between those unwilling to pay and those who are willing to pay A LOT for frontier model performance.
I am not sure I get this. It seems on first glance like just another harness ...
Built to work with lmstudio, one of the leading easy to use local model servers. LMStudio is the closest to plug-and-play without sacrificing play that I've seen; a harness that works well with it is nothing to sniff at. Its not earth shattering either.
Wouldn't most opensource harnesses work with lmstudio? I assume it has an "openai" style chat API like every other model provider? What's special about it vs langchain deep agents or pi or pydantic-ai?
I guess it lives or dies by the harness quality then - on open models run locally by plug and players and models that fit onto peoples laptops that is going to be quite the handicap to overcome.
I run lmstudio personally with a range of harnesses (open and closed) and can't say there is that much of a leap to getting everything talking https://lmstudio.ai/docs/integrations
If you want to use local models, it's more ergonomic than fussing with GGUFs or using LM Studio as a server host and setting up the link to an agent yourself. Although, the model selector is the same as with LM Studio itself which can be overwhelming if you don't know what to look for.
A friendly reminder that both LM Studio app and now this new LM Studio Bionic app are closed source.
Since most people are unaware of this fact.
And? Is that a scandal?
Not a scandal ofc, but people use local models mostly for privacy
and using a closed-source, VC-backed app that might change anything in the next update might not be best for privacy
No, but you are a lot more vulnerable to enshittification and you don’t have the advantages of open source applications.
It’s an important criteria to have in mind when you select an application.
I'm interested in your thought process. How did you get from his initial statement opening of "A friendly reminder ..." to thinking that this was a scandal?
some people want to make money, others want to improve social progress.
Happy to clarify which is who and who is which.
You can do both
No one involved in LLMs want to improve social progress. That is simply not part of the equasion.
Yes.
Is that a problem for you comrad?
It is, cowboy