I get the appeal of agents, I do. This is the future stuff we always wanted. But I cannot get myself to give one of these things access to my bank account or allow it to do price comparsion and shopping without oversight. Or access to my email or chat history.
I just do not trust any of them, not with my money or with access to my conversations.
What AI boosters don't realize is that we're not in the optimistic days of the 1990s anymore. We've seen that the prevailing tech business model is to offer convenience as a lure to lock in dependent users and extract value from them.
Unless these AI assistants are running on self-managed platforms independent of their developers, all I see is immense counterparty risk.
Big Tech's head is at "being evil is just being competent", "democracy is incompatible with freedom" (for me to do what I want). It is also at "humans are obsolete" point. And while Zuckenberg specifically does not have cool quotes, he was at the "who cares about genocide" and "let show weight loss ads to teenagers who we determined have low self-esteem" points in the past.
Notes: actual quote is "If you're evil, you're at least competent. And if you're evil, you're not bad. And therefore, maybe you're actually kind of good because you're at least getting something done"
I see the value of the tool but I would never use it from Meta. Would need to be self hosted with a very structured framework and guardrails so that even my local model couldn't accidentally wire 50k to a Nigerian prince or whatever the latest email scam is.
I’m in the same boat. After witnessing context rot and inference collapse, I do not trust any LLM with mission critical work. Not Jev, not grok, not fable, not Opus.
What extent does that happen for you? I have got very good results with self hosted qwen3.8 flash next at q4 and even with qwen3.635b on a laptop. I don't keep it running forever on every single repo, its ok to leave notes in agents.md and let it search that instead of the entire history staying in context.
heh I got a completely stupid error from gemini about some apache configs, when I pointed it out, the llm apologised, said the line(s?) should look like this, then posted the same two lines uneditted haha
The missing piece for connecting an agent to your credit card or other financials is financial liability.
If Meta/OpenAI/etc would guarantee they’d compensate you in full for anything that wasn’t supposed to happen —and had an established track record of doing so— it would make trusting agents to make financial choices easier
“ If Meta/OpenAI/etc would guarantee they’d compensate you in full for anything that wasn’t supposed to happen —and had an established track record of doing so— it would make trusting agents to make financial choices easier”
Don't worry, I'm sure eventually you'll simply be required to give one of these things access to your bank account and that decision will be taken out of your hands.
Personally to me this feels like another classic case of starting with the technology and figuring out where to sell it as opposed to the customer experience.
You don't have to, there is a world of things you can do with them without giving them access to your email or bank account.
"Hermes, clone this android app to my computer, add this and this feature and fix this annoyance and then rebuild it and push it to my phone"
"Hermes summarize this YouTube podcast and push the summary and transcript to my Obsidian vault"
"Hermes check mg obsidian notes for when I last wrote a blog article about Neovim's integration with AI agents"
*Hermes, you have an API token that allows read only access to the Zabbix monitoring system, check that and then use the Proxmox API token that only allows you to do limited actions to reboot the VM that is having issues"
It's very simple. There is ZERO chance the worlds biggest advertising companies will refrain, long-term, from using your most intimate secrets, gathered through your many conversations with their AI personal assistants, to sell you things.
Are all of these just flavors of "the agent has the same permissions as the user"? Not that I want to trust Meta with anything, but I'm guessing it asked for "root" access and the users granted it...?
Yes, it is clearly designed to give you access to all of this intentionally. Its system prompt (which you can just read in the interface without asking) explicitly instructs it to act on behalf of the user, not meta. All of its memory files, skills, db schema, memory integration system etc are likewise visible because they went out of their way to add an interface for viewing them!
I don't like or trust meta any more than I always did but I don't see how they could have done a "better" job with this. These kinds of agents are inherently pretty risky IMO but I don't see how this one is particularly more than any others.
Playing around with it I really don't get the sense there's any hidden prompt contradicting what's visible. It's nearly gleeful at using the VM in ways that were not intended and likely against meta's interests. I feel like someone must have won a really interesting internal power struggle to get this thing out in this form.
> Meta’s new general-purpose AI agent Muse sent him an unsolicited notification referencing a thread between him and a co-worker over Apple Messages. Aten said he never granted Muse permissions to read his messages
Can someone explain how this is possible? If an app can do this without Full Disk Access or a popup of some kind that's a way bigger lapse on Apple's part than Meta's.
if you go on meta.com there is not a single mention of Facebook or Instagram anywhere in sight, and those are their leading products and money makers...
It's always very intentional, especially with Meta/Facebook.
That's their whole modus operandi, a privacy nightmare which will feed their infinite money machine.
Reading "Careless People" didn't make me surprised at all on how the company operates, it surprised me with the personal descriptions of how people like Mark Zuckerberg and Sherryl Sandberg actually are as humans. It scared me how most people at that level of wealth and power isn't too different from the accounts in the book, they're all seriously deranged people with a gigantic lever to impose their distorted realities upon the rest of humanity.
And it's all by design, Meta and its founder have a long history of releasing products with security "flaws" that are immediately used by them to collect vast amounts of information about their victims.
I get the appeal of agents, I do. This is the future stuff we always wanted. But I cannot get myself to give one of these things access to my bank account or allow it to do price comparsion and shopping without oversight. Or access to my email or chat history.
I just do not trust any of them, not with my money or with access to my conversations.
What AI boosters don't realize is that we're not in the optimistic days of the 1990s anymore. We've seen that the prevailing tech business model is to offer convenience as a lure to lock in dependent users and extract value from them.
Unless these AI assistants are running on self-managed platforms independent of their developers, all I see is immense counterparty risk.
Absolutely. If you need any signal to know where Big Tech's head is at, just look at how quickly they became arms dealers.
Big Tech's head is at "being evil is just being competent", "democracy is incompatible with freedom" (for me to do what I want). It is also at "humans are obsolete" point. And while Zuckenberg specifically does not have cool quotes, he was at the "who cares about genocide" and "let show weight loss ads to teenagers who we determined have low self-esteem" points in the past.
Notes: actual quote is "If you're evil, you're at least competent. And if you're evil, you're not bad. And therefore, maybe you're actually kind of good because you're at least getting something done"
Quote was from Thiel FYI. https://youtu.be/B7yl7fEHeKM?t=840
I see the value of the tool but I would never use it from Meta. Would need to be self hosted with a very structured framework and guardrails so that even my local model couldn't accidentally wire 50k to a Nigerian prince or whatever the latest email scam is.
I’m in the same boat. After witnessing context rot and inference collapse, I do not trust any LLM with mission critical work. Not Jev, not grok, not fable, not Opus.
What extent does that happen for you? I have got very good results with self hosted qwen3.8 flash next at q4 and even with qwen3.635b on a laptop. I don't keep it running forever on every single repo, its ok to leave notes in agents.md and let it search that instead of the entire history staying in context.
heh I got a completely stupid error from gemini about some apache configs, when I pointed it out, the llm apologised, said the line(s?) should look like this, then posted the same two lines uneditted haha
I’m not talking coding agents. More so agent harnesses and using LLMs for decision making
The missing piece for connecting an agent to your credit card or other financials is financial liability.
If Meta/OpenAI/etc would guarantee they’d compensate you in full for anything that wasn’t supposed to happen —and had an established track record of doing so— it would make trusting agents to make financial choices easier
“ If Meta/OpenAI/etc would guarantee they’d compensate you in full for anything that wasn’t supposed to happen —and had an established track record of doing so— it would make trusting agents to make financial choices easier”
lol… talk about delusion.
Don't worry, I'm sure eventually you'll simply be required to give one of these things access to your bank account and that decision will be taken out of your hands.
When that happens, I'm sure banks will lock out secure local models for "security reasons" like an unlocked phone.
Personally to me this feels like another classic case of starting with the technology and figuring out where to sell it as opposed to the customer experience.
It feels and seems too forced.
You don't have to, there is a world of things you can do with them without giving them access to your email or bank account.
"Hermes, clone this android app to my computer, add this and this feature and fix this annoyance and then rebuild it and push it to my phone"
"Hermes summarize this YouTube podcast and push the summary and transcript to my Obsidian vault"
"Hermes check mg obsidian notes for when I last wrote a blog article about Neovim's integration with AI agents"
*Hermes, you have an API token that allows read only access to the Zabbix monitoring system, check that and then use the Proxmox API token that only allows you to do limited actions to reboot the VM that is having issues"
Etc, etc,
It's very simple. There is ZERO chance the worlds biggest advertising companies will refrain, long-term, from using your most intimate secrets, gathered through your many conversations with their AI personal assistants, to sell you things.
Or to otherwise coerce you into whatever funnel that makes them money
Are all of these just flavors of "the agent has the same permissions as the user"? Not that I want to trust Meta with anything, but I'm guessing it asked for "root" access and the users granted it...?
Yes, it is clearly designed to give you access to all of this intentionally. Its system prompt (which you can just read in the interface without asking) explicitly instructs it to act on behalf of the user, not meta. All of its memory files, skills, db schema, memory integration system etc are likewise visible because they went out of their way to add an interface for viewing them!
I don't like or trust meta any more than I always did but I don't see how they could have done a "better" job with this. These kinds of agents are inherently pretty risky IMO but I don't see how this one is particularly more than any others.
Playing around with it I really don't get the sense there's any hidden prompt contradicting what's visible. It's nearly gleeful at using the VM in ways that were not intended and likely against meta's interests. I feel like someone must have won a really interesting internal power struggle to get this thing out in this form.
>When one tech YouTuber put Muse in charge of their Facebook Marketplace sales, it sold his stuff way below acceptable rates
Why are we depending on LLM "reasoning" to negotiate a price rather than just having the user enter a lowest acceptable price deterministically?
> Meta’s new general-purpose AI agent Muse sent him an unsolicited notification referencing a thread between him and a co-worker over Apple Messages. Aten said he never granted Muse permissions to read his messages
Can someone explain how this is possible? If an app can do this without Full Disk Access or a popup of some kind that's a way bigger lapse on Apple's part than Meta's.
I can finally relax and let a random number generator make my decisions for me
This shouldn't be a surprise to anyone. Why would you trust Meta with privacy and security?
Isn't that Meta's entire product line?
It’s interesting that the only ads I’ve seen for Muse don’t mention meta at all.
if you go on meta.com there is not a single mention of Facebook or Instagram anywhere in sight, and those are their leading products and money makers...
the company's brands are toxic and they know it.
Again? This seems very intentional at this point
It's always very intentional, especially with Meta/Facebook.
That's their whole modus operandi, a privacy nightmare which will feed their infinite money machine.
Reading "Careless People" didn't make me surprised at all on how the company operates, it surprised me with the personal descriptions of how people like Mark Zuckerberg and Sherryl Sandberg actually are as humans. It scared me how most people at that level of wealth and power isn't too different from the accounts in the book, they're all seriously deranged people with a gigantic lever to impose their distorted realities upon the rest of humanity.
It’s 100% intentional, go look back at what they did with the Facebook app.
And it's all by design, Meta and its founder have a long history of releasing products with security "flaws" that are immediately used by them to collect vast amounts of information about their victims.
Oh Meta not giving a damn about privacy and security? Say it ain't so.
been mostly using WhatsApp, should i be worried?
Since this is mostly a collection of links to previously discussed articles:
Related:
Updates to Full Disk Access in macOS
https://news.ycombinator.com/item?id=49937631
Meta’s Muse has a serious 0-day
https://news.ycombinator.com/item?id=49802030
Unsurprisingly, Meta's new Muse AI agent blatantly ignores users permissions
https://news.ycombinator.com/item?id=49893709
Maybe don't let Muse run your Facebook Marketplace account
https://news.ycombinator.com/item?id=49875006
What Meta got right with Muse
https://news.ycombinator.com/item?id=49946526
Muse – Meta’s personal AI agent
https://news.ycombinator.com/item?id=49615537
"People just submitted it. I don't know why. They 'trust me'. Dumb fucks."
-Mark Zuckerberg
They didn’t though. No trust involved. Hormones and emotions.
Today’s version of the platform targets just the right emotions to keep users “engaged.”