Kinda agreeing with most other highly critical comments. It's a weird insular subculture indeed, Trashfuture this week had a great summarization:
"Bizarre hypotheticals, the power fantasies, the sex life that's one step shy of the angry pirate. [It's like] a bunch of 12 year old school shooters got Tom Hanks Big'd into adults, and they've infiltrated governing the thing that's supposed to solve all the problems in the world and replace the economy.
...and then never doing anything or learning anything, its posting posting posting, while they don't understand computers, or the security problems they fantasize about. And its foisted upon and pitched to us that the computer is only understandable as a magical god, because for Eliezer Yudkowsky it actually is."
> The market should be choosing not the most capable models, but the partners with the strongest incentives towards safety, and only a community with this knowledge can spread that.
Read strictly, author is claiming that only ML4Good or equivalent has the knowledge to dictate what models we should be using. I categorically disagree. Or rather, "they're not even wrong", since "AI safety" is not even remotely encapsulated in this piece.
At the risk of ad hominem, this is an Accenture consultant expanding the definitions for their own ends. Maybe they'll define what "good" is. Someday.
We've seen this wave before with big A Agile consultants, Microsoft certification hucksters, and different hucksters from before my time.
It's a verbose (probably AI written?) piece. I gave it the charitable good-faith read. But in the end, I couldn't shake the feeling that even amongst the welcome humanizing of the concepts, this is the same kind of amorphous know-better-than-you self-promotion, opportunistically hitching their own trailer of grift to the buzzword of the day.
They're just another cult formed around a sci-fi writer, like Scientology. Except at least L. Ron Hubbard had some slightly more original ideas than "robots will be super smart and trick and kill us all".
People talked about disseminating the training within their own teams. They considered applying for AI safety fellowships. They discussed forming local communities and groups to stay on top of new safety-relevant research and think about how it could inform legal practice.
We did talk about things that are closer to the practical interface between AI safety and law, though. For example, we discussed how small decisions in deployment can accumulate into gradual disempowerment. We actually did a role-playing exercise based on the AI Minister in Albania case, which @Raymond Douglas included in Gradual Disempowerment Monthly Roundup
There's a saying "writing about music is like dancing about architecture" These are just computing systems deployed in the real world, on workstations and phones and rack servers in datacenters. Solve the problems with code and architecture, as our ancestors have done since time immemorial.
Not sure what talking about it in these broad leaky abstractions accomplishes, or e.g. how the European Commission thinks this prevents them from being disempowered when they control none of the stack.
It seems like to me it would be more effective to focus on the object level aspects of their arguments rather than their weird lives? Their argument isn’t “look at my credentials, I’m an expert you must believe me” it’s that “ML research is largely math and science engineering, as AI gets better at math and science it can use that to speed up the cycle at which point something bad could happen”.
These Bay Area people dedicated their lives to AI risk often before AlexNet even showed deep learning could scale, by definition they are eccentric people.
At the same time, people like the AI 2027 guys (who that article speaks poorly of)… have been basically fully on the money so far.
Why that was a weird rabbit hole to go into. Thank you. I don't fall for this stuff though. It looks like a conspiracy theory like the world is flat, etc. That's enough internet for today.
This reminds me of during covid, when one of the biggest public proponents of vaccines in the UK was having an affair. The anti-vaxers then, instead of focusing on whether vaccines were helpful, focused on the affair. That allowed them to dismiss the pro-vaccine point of view without actually engaging on the merits of their argument.
I think the thing is engaging with “ML is mostly math and science engineering, AI is getting better at autonomous math and science, if it gets fully super human the rate of change could be mind boggling” is hard to fully dismiss in the same way you can fully dismiss weird Bay Area people. So that’s why we are seeing the pivot towards the personal attacks on these people.
In conducting his affair he was breaking the lockdown rules that he had written, which harmed their credibility. The antivax crowd (I am not a member) quite reasonably extrapolated that if the architect of lockdown was not locking himself down then he wasn't too worried about the virus, ergo vaccines were being oversold.
When you are a public figure like that then the personal is political.
This stuff? A spanish speaking critic of AI safety says, "Hey guys, it's really not so dangerous." What did you mean by "this stuff" precisely? Is it like "you people" perhaps?
I don't know about anyone else, but I'm completely over the self-appointed "AI safety practitioners" leading workshops and writing overwrought blogs while (by their own accounts!) acting as handmaidens to the AI apocalypse.[0]
Enough. Let's take them at their word: it's time to sue them, jail them, stop them from killing us all like they insist they are trying to do.
"Defendants admit that they provide a service without fully knowing how it works. This is not just any service. It is one that Defendants themselves concede poses an existential risk to the continued survival of humankind. Defendants claim they cannot stop barreling forward with their potentially civilization-ending endeavors unless they are forced to do so by the government. They have asked the government to tie them to the mast. Plaintiff brings good news to the Defendants: The Florida Attorney General is answering your cry for help with a motion to enjoin you from harming Floridians with your reckless, unacceptably risky product."[1]
turtle island is not a literal place. its an indiginous reference to the world. a dei expert claimed there was an island the indiginous people were from. the point being they were not an expert. similar challenges happening today for ai safety experts
Kinda agreeing with most other highly critical comments. It's a weird insular subculture indeed, Trashfuture this week had a great summarization:
"Bizarre hypotheticals, the power fantasies, the sex life that's one step shy of the angry pirate. [It's like] a bunch of 12 year old school shooters got Tom Hanks Big'd into adults, and they've infiltrated governing the thing that's supposed to solve all the problems in the world and replace the economy.
...and then never doing anything or learning anything, its posting posting posting, while they don't understand computers, or the security problems they fantasize about. And its foisted upon and pitched to us that the computer is only understandable as a magical god, because for Eliezer Yudkowsky it actually is."
- https://www.youtube.com/watch?v=zcPR6rde1GQ or - https://www.patreon.com/trashfuture/posts/last-days-of-17089...
> The market should be choosing not the most capable models, but the partners with the strongest incentives towards safety, and only a community with this knowledge can spread that.
Read strictly, author is claiming that only ML4Good or equivalent has the knowledge to dictate what models we should be using. I categorically disagree. Or rather, "they're not even wrong", since "AI safety" is not even remotely encapsulated in this piece.
At the risk of ad hominem, this is an Accenture consultant expanding the definitions for their own ends. Maybe they'll define what "good" is. Someday.
We've seen this wave before with big A Agile consultants, Microsoft certification hucksters, and different hucksters from before my time.
It's a verbose (probably AI written?) piece. I gave it the charitable good-faith read. But in the end, I couldn't shake the feeling that even amongst the welcome humanizing of the concepts, this is the same kind of amorphous know-better-than-you self-promotion, opportunistically hitching their own trailer of grift to the buzzword of the day.
They're just another cult formed around a sci-fi writer, like Scientology. Except at least L. Ron Hubbard had some slightly more original ideas than "robots will be super smart and trick and kill us all".
link in the comments of the article is also interesting https://www.lesswrong.com/posts/bfwH88cFEm3r7PWfv/what-lawye...
People talked about disseminating the training within their own teams. They considered applying for AI safety fellowships. They discussed forming local communities and groups to stay on top of new safety-relevant research and think about how it could inform legal practice.
We did talk about things that are closer to the practical interface between AI safety and law, though. For example, we discussed how small decisions in deployment can accumulate into gradual disempowerment. We actually did a role-playing exercise based on the AI Minister in Albania case, which @Raymond Douglas included in Gradual Disempowerment Monthly Roundup
There's a saying "writing about music is like dancing about architecture" These are just computing systems deployed in the real world, on workstations and phones and rack servers in datacenters. Solve the problems with code and architecture, as our ancestors have done since time immemorial.
Not sure what talking about it in these broad leaky abstractions accomplishes, or e.g. how the European Commission thinks this prevents them from being disempowered when they control none of the stack.
My entire opinion of AI safety has been colored by the fact that I recently learned AI safety is a sex cult.
https://archive.today/bkOXn
It seems like to me it would be more effective to focus on the object level aspects of their arguments rather than their weird lives? Their argument isn’t “look at my credentials, I’m an expert you must believe me” it’s that “ML research is largely math and science engineering, as AI gets better at math and science it can use that to speed up the cycle at which point something bad could happen”.
These Bay Area people dedicated their lives to AI risk often before AlexNet even showed deep learning could scale, by definition they are eccentric people.
At the same time, people like the AI 2027 guys (who that article speaks poorly of)… have been basically fully on the money so far.
Why that was a weird rabbit hole to go into. Thank you. I don't fall for this stuff though. It looks like a conspiracy theory like the world is flat, etc. That's enough internet for today.
This reminds me of during covid, when one of the biggest public proponents of vaccines in the UK was having an affair. The anti-vaxers then, instead of focusing on whether vaccines were helpful, focused on the affair. That allowed them to dismiss the pro-vaccine point of view without actually engaging on the merits of their argument.
I think the thing is engaging with “ML is mostly math and science engineering, AI is getting better at autonomous math and science, if it gets fully super human the rate of change could be mind boggling” is hard to fully dismiss in the same way you can fully dismiss weird Bay Area people. So that’s why we are seeing the pivot towards the personal attacks on these people.
In conducting his affair he was breaking the lockdown rules that he had written, which harmed their credibility. The antivax crowd (I am not a member) quite reasonably extrapolated that if the architect of lockdown was not locking himself down then he wasn't too worried about the virus, ergo vaccines were being oversold. When you are a public figure like that then the personal is political.
You fall for this stuff? It's obviously not a sex cult
>You fall for this stuff?
This stuff? A spanish speaking critic of AI safety says, "Hey guys, it's really not so dangerous." What did you mean by "this stuff" precisely? Is it like "you people" perhaps?
painting groups you don't agree with with obviously false descriptors like 'sex cult'
Hi MattYg
https://forum.effectivealtruism.org/users/matthew-yglesias-1
I don't know about anyone else, but I'm completely over the self-appointed "AI safety practitioners" leading workshops and writing overwrought blogs while (by their own accounts!) acting as handmaidens to the AI apocalypse.[0]
Enough. Let's take them at their word: it's time to sue them, jail them, stop them from killing us all like they insist they are trying to do.
"Defendants admit that they provide a service without fully knowing how it works. This is not just any service. It is one that Defendants themselves concede poses an existential risk to the continued survival of humankind. Defendants claim they cannot stop barreling forward with their potentially civilization-ending endeavors unless they are forced to do so by the government. They have asked the government to tie them to the mast. Plaintiff brings good news to the Defendants: The Florida Attorney General is answering your cry for help with a motion to enjoin you from harming Floridians with your reckless, unacceptably risky product."[1]
[0] https://www.theguardian.com/technology/2026/oct/03/openai-sa...
[1] https://www.myfloridalegal.com/sites/default/files/plaintiff...
reminds me of a time when my company had a dei consultant come in and he couldnt find turtle island on google maps...
I don't understand why this is memorable.
turtle island is not a literal place. its an indiginous reference to the world. a dei expert claimed there was an island the indiginous people were from. the point being they were not an expert. similar challenges happening today for ai safety experts