fantastic. I've heard of bits and pieces of this but this is the first article I've read which lays out the entire timeline with all the firsthand sources you could want. thanks for the effort!
As far as manifestos go HPMOR is a pretty good one. I think that’s what a lot of the critiques about it miss. Yudkowsky is actually a pretty decent writer, especially in the fan fic scene. Not only is his prose actually acceptable which is pretty rare in fanfic it’s clear that he actually thought through a lot of the plot points ahead of time. Harry Potter universe fits very well for his story not only because it’s popular with the target audience, but because there’s a ton of unexplored authority figures in the original and the main goal of the manifesto largely boils down to rejecting blind belief in authority and blazing your own trail. I think it’s hard to separate the number of people who read it because it’s legitimately the best Harry Potter fanfic from the people who read it for the manifesto stuff which again points to the ingenuity of choosing it as a setting.
For me this is probably the weirdest part of it. I have read HPMOR and was distinctly underwhelmed by its literary quality. I mean, it's not bad, but it's not serious literature. It's not even on the level of the original HP. It's a fanfic. Sure, it's a fanfic with a lot of fan service and geekery and so on, so the audience is targeted masterfully. But still, it's hard not to notice the numerous literary faults. I mean, "he actually thought through a lot of the plot points ahead of time" is a kind of praise, but not one that a decent literary work expects. That's like saying about a chess grandmaster that he actually knows how all the pieces move. I mean, sure, on the fanfic scene it's probably above median, but I've seen people making out of it some kind of epic revelation, akin to some kind of a sacred text. Completely undeservedly, in my opinion, and rather weird for a community that is so proud in it's objective and rational approach to things that put it into its very name.
It's terrible as a work of literature. It has many, many, many flaws, but the most serious is that it's boooooring. Hundreds upon hundreds of pages go by with nothing happening except creepy children preaching at each other.
It's terrible as a fan fiction. Not a single character acts anything like the original, the world is unrecognizable, and the plot is derailed entirely by the end of the second or third chapter. The original story needs to be distorted beyond recognition because the author's bizarre worldview works even more poorly in a fantasy world than it does in reality.
It's demonstrably effective as a manifesto, but that is despite the fact that it is riddled with scientific and conceptual errors.
It's one of the worst things I've ever read. The fact that people like it is evidence of the bizarre incestuous nature of the community that produced it.
Anyone who knows anything about Harry Potter fandom in those years knows that its readers were always thirsty for something new in the Potterverse to read or be exposed to. By latching onto that universe's incredible popularity, Yudkowsky exposed a large number of young, inquisitive people to the ideas of rationalism. It was essentially a recruitment mechanism, one that was wildly successful. Many rationalists freely admit that their first exposure to rationalism was via HPMOR.
I don't think Harry Potter fans would look kindly on their favorite story being treated with such contempt. Remember that within about three sentences of introducing Ron Weasley our hero finds him guilty of the heinous crime of enjoying sports and concludes he should not exist.
Excuse me. It was Draco Malfoy, our rapist deuteragonist, who said that. Our hero merely agreed.
Maybe Harry Potter fans would read a bad fantasy book, but HPMOR is not just a bad fantasy book. It is repugnant to healthy people from the first few pages.
I really, really dont understand this complaint at all. Harry and draco disliking ron is supposed to show that they are snotty kids who have big blind spots. The characters are supposed to be dislikable at the start, the story is about the growth they experience from there. This is part of what makes it a better story than the original. Yudkowsky's harry grows as a character which basically doesnt happen at all in the original series.
Do you dislike succession too? Some people just need completely heroic protagonists in their stories. Thats fine, but it doesnt make the story bad.
They are introduced as insane, not snotty. Nearly the first thing Draco says is that he plans to rape Hermione and our Hero uses his Agency to do exactly nothing about it. (It is hard to exaggerate how bizarre this story is to people who haven't read it.)
Hundreds of pages later, Harry uses the first iota of power he is given to prohibit students from playing Quidditch according to rules he believes are irrational. There is an immediate outcry which he dismisses because it's just the mewling of inferior intellects. It's hard for me to see any signs of growth here.
(Harry's contempt for sports is amusing, given that about a third of the story is dedicated to his love of cheating at a particularly moronic sport invented by the Lord Voldemort ripping off Ender's Game.)
He convinces Draco that his whole ideology is wrong. He convinces him not to rape anyone before he rapes anyone. Not exactly sure what the better outcome is here, just turning him in would only strengthen his violent ideology. There are many complaints to be made about the story, as you said its boring and too long and often too preachy. A lot of the visible changes in the characters happens in the last quarter. Even though the whole story has been building to that it can seem like its a bit out of nowhere.
Does Yudkowsky spill more ink on condemning Draco's hatred and violence or on making fun of Hermione Granger's feminism? Voldemort is treated as an ultimately worthy adversary, while Ron is a joke and Hermione is a pet handed power by our Awakened Hero. Yudkowsky treats sexual violence, genocide, and mass murder as forgivable missteps on the road to greatness, but treats liking sports as a fatal character flaw and being concerned with human rights as risible. Arrogance is consistently rewarded while goodness is consistently ridiculed.
I don't claim Yudkowsky believes these things consciously, but they are omnipresent in his story and they reflect his bizarre attitudes to morality. The fact that he acknowledges at the very end that the characters he created are fools doesn't change the fact that he spent six hundred thousand words holding them up as paragons of virtue and rewards them with apotheosis.
Hermione is the actual hero of the story. Every time rationalist harry makes a mistake he is acting against her direct advice. Her character literally exists to show off his deficiencies and his coming of age basically boils down to realizing he should be more like her.
For many reasons voldemort is an ultimately worthy adversary. Much of that has to do with Harry's biases that come from being too invested in rationalism. Since he considers himself above everyone he's ever met he longs to find someone he considers his better and voldemort fills this role. He wants a mentor so badly that he's willing to throw out his entire ideology to keep him. I think yudkowsky's portrayal of voldemort as the ultimate adversary to rationalism is better than pretty much any plot arc in the original story. Harry does ultimately completely reject voldemort so Im not sure how sexual violence, genocide, and mass murder as treated as forgivable. From the very beginning Harry is deeply concerned with human rights, his distractions and the path he takes toward achieving human rights in this messed up world is again an integral part of the story.
> The fact that he acknowledges at the very end that the characters he created are fools
I think this is my biggest issue with your characterization. Yudkowsky shows harry to be a fool from the very beginning, as you pointed out with the draco/ron stuff. With his dismissal of much of hermiones advice. He very consciously did not create harry as some ubermencsch character who can do no wrong. He shows that harry believes himself to be a parity of virtue, but that he very much is not until hermione's death.
This is a forgivable statement if you didn't read it, because then I guess you may not have seen that there's themes at play and a point being made. If you did read it I do not understand how you don't see how you are meant to dislike the summary judgement and the judgement itself being an effect of yet another plot point.
But then if you did not read it it seems a little disingenuous to say "this is a bad book", because you would not only not know but anything you did know you would not understand.
What is there in the story that indicates this judgment is wrong? Ron does not appear in the story again, while by the end Harry is an immortal superbeing. His arrogance is never once punished.
You can be a sexually predatory sociopath and still be of value in Yudkowsky's world, but like sports and you're dismissed in your entirely within a single paragraph.
> "Ah..." Harry said. This was very hard to say. "Actually, the rest of the world doesn't know this part, not even the Headmistress. But in point of fact you were one hundred and twelve percent correct about [spoiler], and I'll remember for future reference that although 'Dark' and 'evil' may not technically be the same thing, there's a great big statistical correlation."
> "Oh," said Hermione, and fell silent again.
> "You're not saying that you told me so?" said Harry. His mental model of Hermione was yelling: I TOLD YOU SO! DIDN'T I TELL YOU SO, MR. POTTER? DIDN'T I TELL YOU? [spoiler], I SAID, BUT YOU DIDN'T LISTEN TO ME!
It feels like most people who criticize the story didn't actually read it, and just start with the premise that Yudkowsky sucks at writing, then conclude that obviously Harry Potter must be Mary Sue, and that whatever Harry says is what Yudkowsky wants you to believe.
If you actually read the story, you will find out that many things that Harry said and believed turned out to be wrong. That ending was written by Yudkowsky, too. The characters that disagreed with Harry since the beginning, such as Hermione and Dumbledore, were written by Yudkowsky, too. Perhaps there is a little more depth in the story than if you assume that it must be a Mary Sue story.
Ah, yes, the ending where the invincible Superman uses his gigabrain to singlehandedly save humanity, end death itself, get the girl (and turn her into an immortal magical unicorn princess (yes, really, this story is dumb)), vanquish a dozen Wizard Hitlers at once, and tear apart entire stars for their raw resources shows that he paid the price for his earlier arrogance.
I have read the story in its entirety. Don't ask me why. The fact that our Lord and Savior holds the idiot ball on occasion does not make him realistically flawed. The fact that Voldemort would have realistically been exposed within thirty seconds makes the story dumber, not smarter.
Also, you ignored the question I actually asked: where does Harry realize that it is wrong to steamroll everyone he thinks is stupid?
> It's terrible as a work of literature. It has many, many, many flaws
Pretty much all fan fiction is terrible literature. I find yudkowsky's prose to be above average for fan fic which admittedly is just saying hes among the best of the worst. Every single fanfic ever written that doesnt end after 1 book has significant boring sections, theres no editor.
> Not a single character acts anything like the original
This is the point. Yudkowsky recognized that the hp world acts as a natural foil for his ideology. By changing the characters he points out the original characters flaws, but also the ways they are better than the rationalist versions of the characters. I dont understand why people claim fanfic should be always be faithful to the original setting. Using it as a foil can work too. The characters in the original, especially the authority figures are all incredibly flat. Why would you want more of that? Even the ones he largely keeps the same, like dumbledore, he explores in a new light which makes them seem different even though they are very similar to the original. The reveal that the 2 Dumbledore's are actually very similar(the exact same?) characters is central to understanding rationalist harry's deficiencies.
> The fact that people like it is evidence of the bizarre incestuous nature of the community that produced it.
Agreed but I think this says more about how crap HP fan fic tends to be than anything else. Yudkowsky saw unmet demand and used it to shoehorn his ideology in
Let me be clear, he's no cormac mccarthy(my favorite prose is the passage from blood meridian describing the raiders). My bar for fan fiction prose is basically whether it annoys me, its a low bar. Most of what makes hpmor good is the cleverness with which yudkowsky subverts the original harry potter world to try to make a rationalist argument, whether that be by inverting a character or playing off the harry/voldemort relationship in thoughtful ways.
'"Voldemort," said the old wizard. "I understand him now at last. Because to believe that the world is truly like that, you must believe there is no justice in it, that it is woven of darkness at its core. I asked you why he became a monster, and you could give no reason. And if I could ask him, I suppose, his answer would be: Why not?
They stood there gazing into each other's eyes, the old wizard in his robes, and the young boy with the lightning-bolt scar on his forehead.
"Tell me, Harry," said the old wizard, "will you become a monster?"
"No," said the boy, an iron certainty in his voice.
"Why not?" said the old wizard.'
as you can see its nothing special, just good enough
I see, thanks. It reminds me a little of those spam emails which are sprinkled, intentionally, with obvious spelling errors. They aren't trying to trick the average person, they are trying to filter for a much smaller, more valuable audience.
fwiw, I don't think that was intentional here. He's just more interested in "ideas" than prose or people. A more extreme example of Andy Weir. Definitely a niche. I read it years ago while backpacking and it perhaps needed an editor but the overall struggle/war and reimagining of logical fallacies of the HP universe's magic were fun.
yes exactly. Fan fiction readers largely do not care about grammar or nice prose, just has to not be overly distracting which unfortunately most fan fic fails at
> His autobiography even opens with a parable: as a small child he threw his favorite squirtgun off a third-floor balcony rather than accept that his mother could lend out what was his, and felt complete when he heard it hit the pavement.
What a classic narcissist.
> Main character syndrome stops being a character flaw the day you join a religion whose plot has you in the lead.
I've tried to read this, but somehow the article doesn't really come to a point. Is there any point besides Yud not being perfect, and having connections to people who openly talk about their sex life (and that it's somewhat non-standard)?
I think the point is that Yud and the rationalist community have put themselves in an awkward position where their lifestyle now depends on "p(doom) is big" being true. Which means they can't necessarily be trusted to give unbiased updates on the true level of risk we're facing as new data and information comes up.
Can "ASI will inevitably kill us all" be falsifiable before we create ASI? If not, then what separates it from religion?
Not sure how much I agree with that position, but that's just how I understood it.
> Can "ASI will inevitably kill us all" be falsifiable before we create ASI?
No, but there are decent proxies, like "Alice who is X standard deviations smarter than Bob will win >50% of the time at game Y even given a disadvantage that would make her chances <5% if she was playing against someone equally smart".
I've thought about this analogy for less than five minutes, so it's far from perfect. Point is, ASI being made of silicon isn't very central to the problem. There are lots of analogies that are falsifiable, and "ASI will kill us all" can be decomposed into a simple and bulletproof composition of falsifiable beliefs, even if it is too specific to be itself directly falsifiable.
There's are many people with high p(human extinction) whose salary does not depend on their continuing to profess a high p(extinction) -- me for example. Also Yud's friends have promised to pay him what he is earning now if the organization he works for ever has to reduce his salary because the organization lacks funding. They did this to neutralize the financial incentive he would otherwise have to hesitate to announce that his p(extinction) had dropped.
The rational methods used to establish a large P(doom).
To definitively falsify the claim, humanity would need to live alongside superintelligence forever without it wiping us out. That doesn't mean we can't be pretty darn sure though, in the same way that we can be pretty darn sure that nuclear war could end civilization.
Those "rational methods" are essentially just pascal's wager. You stick some ~infinitely bad outcome (nasal daemons torturing simulations of you and everyone you love for all time) and/or ~infinitely good outcome (perfect paradise for as many sentient minds as could fit in the physical universe), then you can get whatever conclusion you want by multiplying any finite probability by these things, and even give yourself bonus asspats for being conservative by adding several order of magnitude fudge factors against your claims.
The rationalist doom and sex cult uses convoluted mathematics inspired language to justify their faith-- but at the end of the day it's just faith no matter how they dress it up.
you say "falsifiable" as if it holds some important key, but the concept is really not applicable to predictions that by their very nature, can only ever have one data point
(doesn't mean that it's not a religion or it is, but rather I mean it being "falsifiable" or not, doesn't carry any information to that regard)
Because they depend on a supply of anxiety fueled doomers to both fund their lifestyle and provide a supply of willing participants for their control and fetish games.
> their lifestyle now depends on "p(doom) is big" being true
How?
Suppose that for the following 20 or even 100 years nothing happens. The AIs miraculously stay as they are today, the further progress turns out to be impossible, there will be no more agents hacking various websites, no cyber warfare involving AI agents, nothing. There will be the year 2026, forever.
How would any of that impact the lifestyle of anyone in the rationalist community? Please be more specific with your accusation.
(Seems like the rationalists are simultaneously the dark lords of TESCREAL, pulling the strings of the world's most powerful billionaires and politicians, but also irrelevant weird nerds who can be ruined at any moment by a smart blogger explaining how their opinions are silly, or even that they know each other.)
I tried too, but found no sentence that makes any sense. It's all gibberish from top to bottom. I'm not a native speaker but my only reaction was "wtf this is all about and wtf are you even trying to say???"
it's the Ed Zitron style of arguing that something big/expensive/unprecedented/hard is bad, going to fail, because [____________].
and if you put something trivially true into the blank the implication seems logical, and the argument feels powerful. but simply putting a true claim at the end doesn't mean that everything before follows from this claim. (and even if they do the argument is not a total description of all the causal chains that influence the outcomes.)
of course it's hard to do a careful survey of the facts and weigh them and then deduce some likelihoods for possibilities. because some aspects are unprecedented (investment ferocity, AI results/progress) while others are very much precedented (collapse of investment bubbles, doomsayers' wrong predictions).
Just because something was called bad, predicted to fail and actually didn't fail, doesn't make it "definitively not bad, because it didn't fail, I told you so".
Case in point - Bitfinex was and is absolutely a criminal enterprise and scam, and it didn't have any backing for years in row. But it didn't fail, because many different criminals in the world had decided that it is quite convenient tool and put a monetary IV line in it. And now ruzzians are buying kamikaze drones with it, selling oil to India with it, drug cartels are shuffling around their money with it and so on. Does it mean it wasn't and isn't bad? Of course no. They simply found a rich sponsor to bail them out.
Or the whole 2008 crisis is another popular example.
I'm not saying absence of evidence of wrongdoing is evidence of absence of wrongdoing.
But note, you have very concrete things to claim about Bitfinex. Criminal enterprise, scam, organized crime funding, facilitating arms dealing and embargo avoidance, and so on.
Then you throw in 2008. It's definitely popular but example of what? Bad thing that didn't fail? (Things did fail, hence the crisis? Or you mean the too big to fail attitude?)
Policy proposals (eg. "coordinated AI safety R&D should be exempt from antitrust regulations" or "TARP made money on paper, but the financial sector should pay for the total cost of the crisis") cannot be divorced from the provenance of the source (sex cult, goldbugs, anticapitalists), but the same is true of bluesky screeds.
2008 was a shortened mention of the event when multiple companies by all rights went "bad", i.e. should have went bankrupt on their own merit and performance. But an external entity bailed them out and so they continue to exist and work later on. But does this mean that those companies (like Fannie Mae et. al.) weren't "bad" in 2008? No, it doesn't.
Same logic with pure AI corpos and adjacent dependents. If the doomsayers would ever be right about them being financially unsustainable, then most likely these companies would never get even close to the bankruptcy, simply because they would be bailed by external entities. So we would never be sure if OpenAI and Anthropic were "bad" or not, financially. Because either critics are wrong, or any corporate deficits would be covered by budget money (maybe foreign budget, it happens too).
Read to the end... The front matter is context to the conclusion:
> This all most recently came together over the weekend, and they got Dario Amodei and Sam Altman and Elon Musk and Demis Hassabis all onto the same page to say they wanted to WAIVE ANTI-TRUST LAWS so that they could "pace", or slow down, research.
> I feel about this roughly the same way I would feel about all the pharmaceutical companies getting together to say they wanted to waive anti-trust laws so they could form a cartel to ban medical research:
> these people are the enemies of humanity.
> Also: Much of AI research IS medical research. They are very literally proposing to stop a pretty good chunk of medical research in its tracks so that they don't have to fear the paperclipper.
> There are many regulations I would be in favor of, but they only ever want to form a cartel or a world government against the paperclipper.
> No other industry has ever framed any regulation in terms of "stop", "pause", or "slow down". Those are insane ways to consider any form of policy. Policy should prevent bad outcomes and encourage good ones, it is never about some specific speed.
> This industry frames its regulations in this way because it has, at its core, a millenarian death cult that worships and fears AI as the cause of the end of the world, that feels compelled to create AI, and that at least somewhat wants to destroy AI as a field.
There are better ways to make this criticism. The main problem I see with this is that it's basically a long ad hominem, followed by some valid concerns about market concentration at the end.
Remove all the ad hominem and what remains is two paragraphs long. Valid ones, though.
That was a figure of speech. It boils down to the fact that, by asking for laws mandating AI research to slow down, the incumbent Big AI firms get regulatory capture and cartel power, which might be the real motivation underlying the apocalyptic language (follow the money and all that).
The other thing that annoys me is that there are plenty of folks in the AI safety space who do not think very highly (at least now) of Yudkowsky. One of the comments I saw recently on a Substack article talked about how Yudkowsky has "beclowned" himself by not updating some of his earlier theories given how LLMs function.
So it's pretty annoying that this article is trying to paint everyone who cares about AI safety is in a "sex cult". There are lots of folks who are genuinely concerned about risks. If anything I discounted a lot of the AI safety folks because I thought they were not seeing the underlying limitations in LLMs. That may still be the case, but agents keep breaking through what I thought were limits on their behavior.
If anything, I don't understand at all the folks who think ASI is on the horizon (< 10 years) but think everything will be hunky dory and easy to control once we actually do build superintelligence.
the sex cult is the sensationalist draw. you don't get readers paying attention in our current content economy if you don't have something salacious
which is also to say - I found it very funny and enjoyable to read and it illustrates how much of dorky little echo chamber that group is. this whole idea that you can become some Spock-like, logic-driven creature in spite of our biological wiring fundamentally opposing this seems to underlay all of this. and the fact that they're all orbiting around one 'cool poly girl' and 'visionary dude' is hilariously similar to the kinds of very formerly-WASPy-and-now-just-WASy hetero polycules that I, in my more youthful and adventurous days, hung out with (and quickly departed after getting icky vibes). like down to the fashion choice and need to constantly prove how smart they each were at all times (do they, too, get really high and put on some anime with way too much fan-service and proceed to have orgies? one wonders)
those are the types of people who always end up actually having the worst coping skills because they can't even admit that they might be emotionally-driven sometimes, or that they have emotions at all that they might need to develop the cognitive skills to deal with in a pro-social way. or they get called out enough that they finally read a little about therapy and just end up weaponizing psycho-therapeutic language for their own ends (instead of, say, practicing the skills, doing the workbooks, developing insight)
which is how your anxieties might lead you to believe that the sky could fall at any moment (and thus the whole nuclear war and the 'don't trust bust us!!' bit)
I think the part that is annoying me the most is that one of these dorky little polycules has somehow found their way to the levers of power and are just mashing all the buttons and dialing all the knobs. these are honestly the last people on Earth that I'd trust with having any kind of influence
> the sex cult is the sensationalist draw. you don't get readers paying attention in our current content economy if you don't have something salacious
Nerds used to be shamed for not having enough sex, now they are being shamed for having too much sex. When will they finally learn to have the average amount of sex, so that the public discourse can finally switch to something else?
They could try just being normal. There's a middle between being unfuckable and hosting rape gangbangs with 40 people. This article actually does a great job describing how the gross, unfuckable nerds are now the ones pushing all of this poly sex cult stuff.
it's also possible to have healthy, non-hierarchical poly relationships. many queer people do it with triads, quads, etc. most of the hetero poly people I met and were icked out by functioned as extremely hierarchical in practice (but they would love to tell you about how they were relationship anarachists or whatever) with a couple at the center and one of the partners being the one that pursued having a lot of other partners while there was hidden pressure for the other to also engage in the practice so they didn't seem patriarchal or toxic (a cruel irony, in retrospect)
that said, my observations have a sample size of like... four couples maybe? (they're indistinguishable from one another now). but the dynamics I saw were four for four which seemed good enough for me at the time to dip out of that scene
One might also argue this is more mainstream in some cultures, and it's just not talked about. In countries such as Japan or France it's not uncommon to have a mistress (or male equivalent) but also having a primary family. That's hierarchical polyamory with uh, less steps.
certainly less calendaring and meeting invites, I'm sure. I did meet a couple of people who identified as non-monogamous well after I had met my very monogamous partner (whom I love deeply and am happily monogamous for) who seemed very cool and were equally put off by the polyamorists they had interacted with. but they were truly diamonds in the rough, at least in my major metropolitan city, and not a single one of them was hetero or close to being a WASP so you're probably right about the cultural tidbit
There's a great movie or Streaming series on this post somewhere.
Sometimes people can be too smart for their own good. Like a frontier model in an eternal hallucination loop, you can create entire seemingly consistent realities in your head and end up bringing others into it.
In case the author is reading this: I’m getting a Cloudflare worker exception when accessing the page using one of my uplinks, possibly because of IPv6.
TLDR: awareness of the SV "rationalist" polycule has breached containment and normies are reading about it now. [0]
Maybe a "better" source is the NY Post article [1]. The information itself isn't really news - it's even largely available on wikipedia [2] [3] - but the fact that it's hitting mainstream-adjacent news outlets is novel.
> Yudkowsky's most viral idea, the one that infects the most people and spreads the farthest, is that AI, if we continue to develop it, will inevitably 1) achieve superintelligence ("RSI"), 2) escape human control 2) kill everyone ("paperclip them", "doom")
I don't really get why people go on about Yudkowsky in this regard. I mean serious thinking about AI goes back at least to Turing in 1950 with his test and prediction computers would get there in about 50 years and presumably keep improving. And people have worried about it since then.
Like in 1949 Norbert Wiener saying of AI type stuff "every degree of independence we give the machine is a degree of possible defiance of our wishes."
Then 30 years later this Yudkowsky is born, later is considered a bit of a crank, goes on a bit about the same old stuff when he grows up and everyone makes out like he invented it?
I've got a lot of respect for Turing, Wiener and Von Neumann but haven't bothered reading Yudkowsky. Is there any point?
It seems a bit unfair to the pioneers of the field to go on about some Harry Potter sex cult guy who repeated their ideas decades later?
Regardless of anything I would have more respect for anyone who didn't explicitly optimize the atomic bombs on Japan for civilian casualties like Neumann did.
Yudkowski is being derided because he wrote some weird fanfic which may or may not be problematic (I didn't read it, I dislike HP too much). And apparently "is considered a bit of a crank".
Neumann actually fucking did it, his calculations for maximum civilian casualties make him quite directly responsible.
I don't think you need to compare Turing, Wiener or Yudkowsky to that mass murderer.
Well most likely consensual noncon given the amount of vetting present in that chart. It's surprisingly common in women, or just a desire for general "gentle roughness".
And idk why it's really worth mentioning anyway, as a gay furry I suppose vanilla normie straights are STILL hung up on what people do with their sex lives. As long as it's legal and consensual who gaf? Y'all, apparently.
Not opining on this one way or another, but the post buries the lede a bit with a one liner that explains where it's coming from, saying that this is all:
> reasonable evidence that their influence isn't prevalent because they're right, and that they would have ulterior motives for continuing to preach that the world is coming to an end even if evidence showed otherwise.
So how they enjoy their sex lives is dependant on continuing to preach AI catastrophism? I don't buy it. Lots of poly folks out here that probably don't care much about AI one way or another.
No one is claiming that all poly people have some ties to the AI community.
They make a claim about a specific community in San Francisco, that is tied together by a mix of AI doomerism and sexual relations. And it's not at all surprising or some farfetched claim that people who enjoy having sex with someone might adopt their ideas, at least publicly, for fear of losing that sexual connection.
>And it's not at all surprising or some farfetched claim that people who enjoy having sex with someone might adopt their ideas, at least publicly, for fear of losing that sexual connection.
am i naive? in the direction you pose this, i can't imagine it's that common and if that happens i don't feel sorry for our imagined victim. being a bay area guy who attends so many orgies that AI doomerism rubs off on him? and he keeps spreading it to other people and that some substantial bulk of doomerism is due to this mechanism such that it voids the ideological purity of doomerism? i'm not a bay area guy and i haven't attended these orgies but i hope they have other shit to talk about at the orgies than AI
if the relationship is flipped - being genuinely interested in the theory but then being roped into weird sex shit that you didn't ask for, sure that that sounds predatory. a sex cult where AI is instrumental, rather than an AI cult where sex is instrumental. it would be more remarkable imo but i don't know if anyone is saying out loud that's what's happening.
Hate to break it to you, but I don't think the AI doomers have a monopoly on orgies. You name the political / belief system and I'll find an orgy for you by 10pm.
Heh, if you think this post buries the lede, have you ever read anything by the community in question? They seem incapable of saying anything without five part Roman-numbered novel length blog posts.
I think this article kind of misses a crucial point. They weren't always AI doomers, that's a thing they picked up along the way. Their core beliefs are 1. That they are the smartest, and 2. That this makes them morally superior because consequentialism (This sort of follows. If you believe in consequentialism, leaving all your decisions to the superior consequence predictor is surely the moral thing to do)
Maybe one day they will stop believing in AI doom, but they will never, ever stop believing they belong to the intellectually and thus morally elect. The day any of them do that they have either broken with the cult or died.
Well, I used to think like that too. Then I became slightly more spiritual, and the very idea of this sort of behavior is bothering, upsetting, irritating. But it mostly evokes pity. Like becoming aware of someone hurting him/herself in some way.
The way I see it, sex is a sort of portal to another world. This sounds a bit weird, but it really does and should have a great deal of meaning behind it, as it is/was central to our species. If it becomes mundane for someone, this really does ruin a lot of that person's life. Even if you completely remove the judgment element. If sex doesn't occur in the context of a love relationship, it's just uncanny territory. It comes with some emptiness.
There's a proper and healthy way to live. This is from my life experience. Stray too much off-course and you will experience consequences. It's just the way things are.
Amazing how people come with all these rationalizations instead of just accepting that they are not the kind of people who like casual sex. There's nothing wrong or unhealthy about casual sex, it's just not for you. You can have your loving sex all your life, nobody will force you to have casual sex.
About orgies, it bothers me how much people who never frequented these circles have so many opinions about how bad or unhealthy they are. I know religious people who have the "healthy sex life" and are completely deranged. I could say that they are that way because of their sex life, but I would be as wrong as you are.
Nothing unhealthy in promiscuity and orgies? The people's moral standards have indeed fallen low. This sort of behavior spreads diseases like a wildfire. As for why people seek this, it's because it's easy to project your demonic fantasies onto a person you don't know. Once you get to know your partner well, his or her personality dispells your imagination and you have to behave in a more upright manner. It's probably why it's common to wear masks on such events.
Some of us have spent countless nights with other kids locked in the basement during parents' swinger parties and/or been there and done it ourselves as adults and found it to be as much an empty experience as the parent poster suggests. Molestation victims also frequently report the same sentiment. Removing love from the equation turns the act of sex into mechanical fluid exchange, and the participants into human livestock.
Promiscuous people are as annoying as those who refuse to eat at the same restaurant twice. You have an insatiable appetite for novelty that you need to see normalized in others-- to defer ever having to face the very emptiness we're describing.
If you actually found it fulfilling, you wouldn't need to sell others on something that sells itself.
Thank you for sharing your personal experience. Not only is what you said compelling, but it again highlights the importance of experience as a path to knowledge. Rationality is brittle and cannot access much of the vastness of human experience.
You had a shitty childhood, I'm sorry. About the rest, I guarantee you that it's just not for you. No problem in that, but don't be mad that other people like the lifestyle. There's nothing unhealthy in that. You can be as puritan as you want, nothing unhealthy as well!
what is so "incredibly" and "imposing" in the parent comment, that saying «my experience is…», «i feel more in it than many imagine/expect», «factually ruins many lifes», and «it just seem working this way», even «remove the judgment element» ??
it is as gently and friendly of a warning as it could be.
i would say rather less nicely: people! don't be degenerate! it does not worth it. it causes harm to you, to neighbours, to bystanders too, even if you dont intend.
Well OP claiming they get "bothered, upset and irritated" by people having "rape kinks" and "pitying them" sounds kind of judgemental to me. Same goes with "don't be degenerate" for sexual behaviours you do not approve that, somehow, also harm one's neighbours for some reason.
Imo judgemental as in starting from "my life experience" and getting to that's "how to live" is a reason to dismiss it as an argument, as others may have different life experiences and yours is not universal.
I'm sure OP would argue that the breakdown of the family structure and social cohesion has a deleterious effect on society as a whole, which includes your neighbors.
Yes and indulging this appetite is much like any other, a good comparison being the other fundamental one: hunger. I think that also frames it in a way that makes it more concrete as to what the negative aspects are as well as showing that total abstinence is not necessarily an unalloyed virtue. If you don't eat at all you will starve but you can easily eat too much or eat the wrong things and it becomes gluttony at the extremes. Coming up with systems that allow you to cheat the negative aspects also turns pathological in the long run. Deadly sins exist even without the religious framing but the spiritual, social and psychological aspects are very real regardless of what anyone professes to believe.
You've made a hypothesis, provided no basis for it, blew past that and built some more conclusions on top of it. Along the way you introduced a few more unsubstantiated thoughts.
The post in it's entirety reads as a religious rant, but it might have had something to offer to the reader if you had bothered to provide some sort of philosophical foundation for why you feel that the middle way is the superior way. And that all excess is inherently not only nearly identical in terms of effect, but by definition harmful.
It's not about their sex life. It's about how sex connects a community of barely legible AI doomerism.
If you don't understand human social interactions very well, or otherwise assume you live in a "world of ideas", then this may not seem salient. However, it is.
Because what we have to explain is why there's a community of people in northern california who deliver unargued prophecies about the end of human civilisation as-if their prophetic ability is a given.
The answer, unsuprisingly, is sex. This is not an unusual answer in the world of "prophetic communication". What, in the end, is prophecy as a social practice? It is some ideological Leader with followers who impart to that leader a 'charismatic power' of foreknowledge based on their attachement to that leader.
What is the mechanism of this attachemnt? In otherwords: how is this style of prophecy spreading around the AI doomer community, if not by plain argument and evidence? Sex.
There are many other... let's call them "orgy circles"... in the Bay Area attended by people who have no idea who Eliezer Yudkowsky is. The majority of AI doomers do not attend orgies. You're reading too much into a single correlation.
I feel like people's problem is less "adults having consensual relations ew" and more "our incentive system is so fucked up that these people are now the intellectual leaders of the AI movement. Not by merit, but by developing a weird erotic exclusionary synchronicity that the rich folks like." Same thing is happening in government too.
I have no idea how trustworthy this study is - but it claims that 62% of women had such a fantasy at least once. How is that not in line with what the OP said about it being "surprisingly common"?
Many rock climbers have felt the "call of the void" when climbing, I've felt it myself on a multi-pitch once, I'd hardly claim they are suicidal or have suicidal ideations and would not identify myself as either. I think there needs to be a certain level of intention to make the fantasy reality for it to be a kink.
I suspect the misunderstanding might be around the precise meaning of the word "kink" then.
I think in one common understanding of the word, that I assume the OP was using, having a taboo fantasy that you merely roleplay with a partner you are deeply intimate with, in full consent from both, can very well be classified as "a kink", even if what you're experiencing is the not really SA.
Based on your response, I'm assuming that in your understanding, it would only be "a kink" if it was a desire to actually experience a real sexual assault, which of course no one, or perhaps only an extreme extreme minority, actually wishes for. I don't believe OP meant to imply that this is at all common.
> I think there needs to be a certain level of intention to make the fantasy reality for it to be a kink.
Hopefully this isn't to imply that people with fantasies actually want to have non-consensual sex in some capacity. If so then this completely misunderstands sexual fantasy!
I don't want to hear about it: yours, hers (Aella) and I'm certainly not going to tell you what my deal is, either.
Problem is, Aella has made it her business to speak about it publicly, and the parent commenter(and I)'d be happier if we didn't read about it. It's fine to say that.
I'm not hung up about anything others are doing: but I genuinely, passionately, don't want to be informed about it.
That's the solution the parent commenter had! And the gay furry (self identified) complained about! If you have a problem with his announcing the solution (to stop reading) then maybe you should just not read his comment, or my comment, or...
I guess what OP wanted to bring up (as I understood) is that if somebody says "I am straight" it does not usually induce people turning the internet off. But in any case they are free to choose that and that does not really matter in anything (a person not wanting to read sth). I still think it is worth bringing it up as OP did.
I don't think the woman in question has participated in this thread. On the other hand, all the pleas to "stop making me read about sex" reminds me of:
"Why Do All These Homosexuals Keep Sucking My Cock?"
> Just don't read her stuff if you're not interested.
This retort is disingenuous when almost everyone is on algorithmic social media and trying to chase clout and drive revenue with engagement bait.
I would never read Aella's Substack in my free time, but I have a good idea who she is and what she posts just by virtue of having a feed that mostly gives me tech and politics stuff.
I was invited to a few orgies with them; I politely declined, they were very respectful of my decision and I wasn’t asked again (which in retrospect must have been partly due to deliberate coordination - fairly advanced social technology, imo). This didn’t affect my standing or acceptance with them. It may well be a sex cult, but if it is, my experience is that it so thoroughly behaves like a regular community in so many aspects that it at least calls into question the predictive usefulness, if not the accuracy, of the label “sex cult”.
The ad hominems are tiresome. You think the people who are explicitly trying to replace all humans are any less weird and problematic?
The rationalists long predicted that misaligned AI would break out of training environments, this has happened. They believe (among other things) that this will continue to happen, and that creating powerful AI that is aligned and controllable is not currently possible.
You should draw a line in the sand and decide what evidence you'd need to see to convince you that their perspective is worth considering.
You don't have to agree with all of their beliefs and predictions (I don't), but you should evaluate their arguments on merit, not on a characterization of some weird subset of people who happened to read a convincing blog post.
i read AI 2027 at the time and thought it was nonsensical. however i was not aware of the conflict of interest running through the philosophy itself, the safety institutes "independently" monitoring ai, and the former public benefit orgs like openai and anthropic creating ai (now de facto fairly evil for profit companies).
it explains so much now.
it would explain why redwood and METR were chosen and apparently willing to immediately write a whitewash report for openai blaming the agents.
it explains why anthropic is infantilising its users, anthropomorphising its product when there is no scientific consensus of llm consciousness, or accepting of outlandish and unreal claims about human extinction when they are lacking in evidence.
who knows, it may also explain and fund the recent media blitz by jacob coxon, which has led to the recent virality of ai safety.
sure, yeah, I've seen how the sausage is made on some of this stuff, and its not pretty.
However this only claims that they are not good at creating effective and trustworthy orgs, it does not address their claim that it is not possible to crate powerful, aligned, and controllable AI. Both things be true.
for what it is worth, i find current frontier ai impossibly stupid. you ask what is the date, it doesn't know. if you set a system prompt to repsond in 5 sentences after 100k context, it can't, the context rot just doesn't permit it. tool calling is not intelligence, it doesn't fix the underlying issue.
it is uncontrolled in the sense that a rock is uncontrolled if you throw it at someone's head.
it isn't good enough for anything important. you would not trust it to run an alarm in a power station.
of course it is not general ai. ask an llm to drive a car. you need to see in fractions of a second ideally. the llm might brake after 20 seconds. i think llms are doomed here. we could come up with a different architecture but then we would be back in the land of fantasy.
we are stupid but hopefully not that stupid. we are dealing with something computer based, luckily we can survive without, with some thinking about it. i would start there instead of with the AI part.
Any actually-reasonable person who was around for the founding craze of this era, ie., New Atheism, quickly realised that there was a deep pathology at the heart of many of its zealots, perhaps summed up by a desired to "recreate religion in their own image". I've walked into many a room of people screaming that teaching children falsehoods is literally child abuse. Any one who knows much of Richard Dawkins, Krauss, et al.'s background with women shouldnt be surprised either.
It's californication into the "rationalist" community as it currently stands is no surprise. Every person I've met deep in this community either seemed to me a narcissist (if male) or, politely and mildly, coquettish (if female). With several relationships I've observed a clear union of the two. I hadn't yet put this so clearly together with the "AI Saftey" crowd, despite being directly adjacent to it all for quite awhile and seeing the cross-proliferation.
My own analysis had concerned their philosophical naivety, general lack of actual computer science knowledge and generic shallowness of understanding of any domain outside their own research and a kind of "science-fiction" version of philosophy and rationalism. I hadn't connected this yet to their strange personalities, but the apoclypticism fits and neatly bridges these together.
So what became of the New Atheist project to create a "secular religion" -- an apocalyptic sex cult furnished by a blend of sci-fi, pop-philosophy, and cute parables from polymath medical professionals. All in all then, it seems they succeeded.
> It's californication into the "rationalist" community as it currently stands is no surprise.
This is the minority split. Most of the online new atheism writers became progressives/social justice/"woke 1.0" people. That trend has receded, so I couldn't tell you what they're doing now. (Probably blocking each other on Mastodon.)
The rationalists are the ones who stayed behind and decided to stick to doing logic and math in their heads instead of thinking thoughts the normal way.
That has not been my impression. Who is the most prominent example? Some of the most prominent figures I know of were Richard Dawkins, Christopher Hitchens, and Sam Harris - all prominent right-wing figures who were rewarded for using atheism to attack Islam in particular during the Iraq War.
I don't believe any of these people are relevant or prominent, but the unimportant people I was thinking of were bloggers like PZ Myers, I think they called themselves "Atheism+" or something.
> Richard Dawkins, Christopher Hitchens, and Sam Harris - all prominent right-wing figures
Are they? That's um, a scientist whose pop book on genetics I read once, a guy I read in The Nation once, and a guy whose book on Buddhism I read once. I remember the first two evolving into GenX curmudgeons (probably the lead poisoning) and have never thought about the last one again. Still, those aren't the right-wingers I'm familiar with.
I mean, this is a rather fun article to read but I'm not entirely sure about the thesis.
> AI Safety Is Mostly A Sex Cult.
> I don't think these people should make policy. (1/?)
One of the learnings from things like the whole Epstein saga has been there are a lot of sex cults in the policy making world and, yes, the people making policy generally shouldn't be. But I'm not sure this article actually comes up with a good explanation of why these particular people shouldn't be making policy. People having active sex lives isn't a disqualifying trait. Being in a cult doesn't disqualify people from being in politics. It is routine and expected that fields will have a lot of similar people in them (as I recall the overlap between security researchers and the furry community is unusual too, and that is hardly more important than justifying a passing comment).
It is more routine than the article writer seems to think for all the people in a small field to know each other and get on really well.
I don't like most religions, personally. But there is a significant difference between a regular religious community, that has its own shared beliefs but doesn't completely exclude those who don't share them (e.g. by forbidding members to even become friends with people outside the community), or demand the community members completely dedicate a majority of their time to the religion; and a cult, which does do all of these things.
Note that there exist cults within many major religions. There exist for example extremist Catholic groups that would fit the definition of a cult. But that doesn't mean that all Catholics are a cult.
"AI safety" is not a cult by that definition any more than Christianity or Islam. There's some subgroups that are a bit more cultist, and that's the same in Christianity etc
Same with Falun Dafa, have a friend who is a practitioner, it's completely normal to have friends outside of community (tons of his friends are), no one tries to convert anyone, and between his business and other stuff the time he spends on the practice is really not that much
Falun Gong is a cult that claims following its tenets can cure any disease, so you don't need doctors (and that high level Falun Gong members are so perfectly healthy that they are hunted by CCP members for their perfect organs). The founder claims to have magical powers, like clairvoyance, levitation, walking through walls, omnipresence, the ability to cure diseases - at least some of which anyone who follows his teachings could reach. Extraterrestrials, higher dimensions, and other such are also part of it. Also, on the evil cult front, Falun Gong is a supporter of right wing extremists in both the USA and Europe.
So no, I don't agree there is much fundamental difference from Scientology.
Oh my, where did you get this from? I happen to know both. The founder of Scientology was a scientist who started with a somewhat decent idea, got popular in the US, but his fame passed quickly. He really craved for that fame, though, so he changed his teaching into some cultist stuff, paired with some sketchy lord and proceeded with demon invocations. His main book is Dianetics, and his followers are very vindictive to those who leave the cult.
Falun Dafa, on the other hand, is more like a Buddhist discipline. You are free to meditate in a park if you feel like it, you are free to go if you don't. There is no organisation, no one will know your name. Its main book is Zhuan Falun, which you could read to make your own opinion, rather than repeating someone's talking points. CCP hunts Falun Gong because its founder refused to become a part of CCP. You can read this story on Wikipedia. The situation is similar to the early days of Christianity when Nero burned christians alive and wanted to eradicate their faith. The irony is that today the Roman Empire has reduced to the city of Vatican which is the temple of Christianity. CCP will fall similarly.
As for organs extraction... well, read about the Sujiatun Concentration Camp. CCP sends there young Falun Gong followers and sells their organs to high rank officials. Demonic creativity. But what did you expect from a communist regime?
Because they don't get enough votes to control the military. If they put a convincing pitch in for why they should then they can. Being a cult doesn't rule them out.
> You do understand what a cult is?
Pretty much any religion and a few other organisations beside. Basically people who believe something strongly. What do you think a cult is, in the sense the article is using it?
No, a cult is any closed-off fanatical group that is deemed harmful to its members and sometimes to outsiders as well. The group who launched a sarin gas attack in Tokyo's metro, Aum Shinrikyo, was a cult. The Jonestown group was a cult. Scientology is often considered a cult, though this is less widely accepted (especially since Scientology is much larger).
No one considers Catholicism, or Protestantism, or Baptism cults. There may be specific communities that are cults that claim to be parts of these churches, but that doesn't mean that "pretty much any religion is a cult".
> No one considers Catholicism, or Protestantism, or Baptism cults.
Speak for yourself. I was coercively and forcefully added to the Roman Catholic church when I was baptised. Attempts to record myself as removed (the 'Count Me Out' campaign) were met with familial and attempted Church pressure. I was asked to visit the church to discuss the 'impacts of my decision'. I was not recorded as leaving, and the church updated its rules so people in Ireland could no longer request to leave the church.
The ways cults control you can vary, and they don't have to involve threats or violence to be coercive control. Threatening to take away someone's community for a lack of belief is controlling.
As I said, I'm certain there are specific parts of Catholicism that do exhibit cult-like behaviors. That doesn't mean the Pope is a cult leader in the same way as Reverend Jones was.
Are you claiming that for essentially anyone in, say, Dublin, announcing that they are now an atheist would lead to their family cutting all ties to them?
Or are you just saying that family members would generally declare their deep sadness at the news, and seek to convince them that this is a wrong belief and a mistake?
The first one is what leaving a cult is like (e.g. if you leave Scientology, most family and friends will be forced by the cult to cut ties, even if they personally wouldn't want to; even more extreme cults may physically prevent you from leaving the compound or even hurt you for the attempt). The second one is just the normal human experience of having opinions and a lifestyle that is not accepted by your family; the same happens to many people if choosing a career their family doesn't approve of, or leaves a job their family considers excellent, or dates someone the family doesn't like, and many other scenarios that have nothing to do with religion.
> don't have to involve threats or violence to be coercive control. Threatening to take away someone's community for a lack of belief is controlling.
This is also without getting into how many institutions in Ireland were (and many still are) faith based, requiring you to be a member of the church to, for example, attend school.
We can agree or disagree that being shunned from your community and denied opportunity because of a belief is controlling, but it's what led to my belief the Roman Catholic Church is a cult. Though to me a lot of it was simply not being allowed to leave.
He is, though. It's just that this particular cult has been around for so long that it's "traditional", kinda like tobacco and alcohol were not counted as "narcotics" in most jurisdictions.
No, he's not. Just look at the life of an average Catholic person and the life of an average Scientologist, or better yet, the life of an average Jonestown resident before the suicide, and see the obvious differences. Note also how little the life of an average Catholic differs from that of an average non-religious person in the same country.
I would be completely fine with saying that there do exist cults operating within the Catholic Church. Arguably, monasteries, especially the more isolated ones, are very much cults, with the cloistered lifestyle, complete dedication to the faith, strict authority, and requirement to cut previous ties to join, plus cutting ties if deciding to leave. I'd even agree that the priesthood constitutes somewhat of a cult, for similar reasons.
However, claiming that all of the hundreds of millions of lay Catholic believers are members of a cult is ludicrous.
This has me wondering if or how FetLife played a role in these communities connecting in the first place…does anyone know of any connection? (FetLife was a combo social network, blogging forum, and dating site back in the day)
> Somewhere, some rationalist-brained person is reading this and saying I'm a "low decoupler", and that whether they run a sex cult has nothing to do with whether they're correct.
Glad he's at least admitting it as a possibility.
Also: you know, you don't need to rely on lossy proxies like how much or how little sex do the arguments' proponents have. You can just read the arguments!
Arguments are not executable code or the final work product of… whatever it is they're arguing about. You have to continue interacting with these people and their AIs, so the underlying question of whether they're trustworthy and not evil always exists.
I think the mores of people building software that is made available as a conversational entity to the entire human population is at least somewhat relevant.
Well, this is old news. I'm still baffled by the fact that people take output of public personas as isolated facts. Take Nietzsche. Sure, great philosophical input. But try reading the sections about women. Same for Schopenhauer. Why? Because they were both incels and had big mother wounds that have never been addressed. So they spew their shit and people take it at face value even tho it stems from hurt. So why would I take any of this seriously if it's very obviously stupid and removed from reality? (Doesn't mean their whole works are)
People chasing the absolute rational are in my experience numbing their emotional part because theres something they can't look at. Or people that chase success above all are driven by their "unresolved packages" as well. I'm not saying that any of this is wrong per se, it's not black and white. But it's the context for what people do, say, and think. They are human after all. We need to consider context and not only the idea itself.
If the man known for urging caution with respect to AI founded a sex cult, then I don't think i want to know what kind of stuff the accelerationist faction are into.
Sometimes when I find someone to look up to and admire, I later find out there is potentially some dark reality to them. I then lament not being able to find any good role models in the world. If they have missused donor money, then that is terrible. If they are pushing towards goals that seems contrary to their stated mission, that is difficult to judge. I did always find it odd, 10 years after reading Yudkowsky's passionate writings, that he doesn't seem to have really gone and done anything about it. If it was all just a creative writing exercise to him... Then that would be rather humiliating to me, having taken it very seriously.
As a side note. If I had a choice to oppose a booming ai comoany, or become the billionaire founder of it, I would still choose to become the founder and then use my resource to do something about. It certainly feels hypocritical, but if you want to stop a boulder rolling down hill, what can you do besides run ahead of it and try stop it?
>Sometimes when I find someone to look up to and admire, I later find out there is potentially some dark reality to them
I too feel exactly what you are saying. I also think that a lot of good role models are simply often among less visible people, sometimes less ambitious. I am not assuming any hard correlation between the two, though. And we also need to accept that we are all only humans, not infallible. Values are more important than singular acts (within reasonable acceptance limits of course).
Anyway. I strongly recommend reading what Bill Gates has to say about AI. I can be wrong, but based on your comment it seems to me that you may find it interesting.
Wow, I admit I didn’t know about this. According to my coarse reading of the news sites it seems like there was no association in fact, but I won’t insist on anything, I only learnt about this now.
Instead of assuming the "heroic responsibility" of the elect, as these fallen rationalist heroes advocated, we accept that we can't save a world which doesn't want to be saved, we focus on being good ourselves - in the salt of the earth sense, with no special dispensations afforded us because of heroic responsibilities. Let tomorrow worry about tomorrow, we may not fix the evil of the world but we can maybe at least not be it.
And hoping for a miracle, maybe. In my experience it's not so impossible to have hope, if we let go of the ambition of being some kind of ... Harry Potter? Almost as if a lot of the fear from the evil of the world came from inside.
It is my view that we are all apes. And therefore all sadists, all maschoists, all vengeful and envious and jealous and have all of the primal things living social animals need to survive in the hash environments of nature. We all displace aggression, we are all hedonists and much else besides. The sweetest person you know has to displace aggression somewhere, or otherwise, has none -- and would then quickly fall victim to the aggression of others on one day of bad luck.
There are those who live lives where this can be tamed/sublimated in a mostly socially acceptable manner, with the socially unacceptable elements "hidden from view" and not too dangerous or harmful. And there are those, unfortunate perhaps, that find themselves (suddenly) unconstrained by their social environment or suddenly imbued with sufficient power to do as they wish. Not many survive such freedom or power, morally in tact, because morality doesnt properly belong to that domain.
The story of Oscar Wilde captures this doubleness to our nature -- split by the socially acceptable. He lived a life of relatively unconstrained freedom in private, perfectly constructing a socially acceptable veneer to his "entanglements" with young boys and the like. Until one environment this pretence could not survive: a court room.
You could see the lesson of Oscar Wilde as about the hypocrisy of people. Or, if you want to remain a (post-cynical) optimist, about the need to keep people like Wilde out of court rooms so that their public image serves as a role model of sorts.
Role models are just those people who never find themselves in court. And this is not necessarily a bad thing. Young children are apes too, and all a role model can be is a shaming mechanism to civilise what they will, eventually, hide from social view -- tame it and make it, ideally not too dangerous or harmful. Perhaps say, only a thoughtless bullying, only a trivial kind of stealing, only revenge on someone who might at least deserve it. But who knows.
None of our species, at least, can become a role model for real. This would induce such a drastic split in the psyche with a tremdenous shameful repression of our real nature; and any real attempt creates profoundly unself-aware monsters. All role models can be is an attempt to ameliorate. An impossible standard that internalises a mild helpful sort of shame
I appreciate this. Though I'm not in agreement on all points, it is, in my opinion, thematically correct.
The role models people keep are all merely partial truths. They are, necessarily, an abstracted and partial view of the person or people underneath. Whether or not the universe is technically infinite, we are dealing in an informational reality that is effectively infinite.
No singular, static ideal can survive on its own in the face of change across an effectively ifinite domain. Ultimately, growth is learning how to let yourself be like "them", and honestly evaluate whether or not they are represemting some truth. That's easy to do with the things we all typically agree on, but much harder when it comes down to things like murder, or abuse.
The key is to not lose your own truth in the process - to come out of that process being someone you can actually love - or, comprehensively identify with.
In any case, it's hard to demonize anyone if you understand where they're coming from. ..and, yes, that means nobody is the perfect creature they think everyone should be. But if one doesn't undertake this process of growth on their own - to include, but not be limited to what others are - you will simply be exposed to it in reality, and have to deal with it anyways.
you cannot remove complex information dynamics from a reality you are founded in without losing the fundamental benefit of that foundation. I.e., you cannot remove the pain or strife from life. ..but, you can add to it new experiences and adaptions that make the whole thing worthwhile.
Some of these potshots just cheapen any arguments. "Gretta has no prior experience with non profits" uh, she was a well regarded Google SRE... her making 200k is nothing.
I'm not sure how being polyamorous, having kinks and not hiding them makes someone part of a cult, even if I generally don't agree with Yud's entire thesis.
The article presents widely how the people it describes all live in communal housing, engage in harmful psychological practices, have a shared belief in the imminence of the Apocalypse, etc.
The currently most upvoted comment is a much bigger article containing vritulaly the same claims, and naming some names. It spefically Mentions Eliezer Yudkowsky as living in shared housing with others. It also mentions that Dario Amodei and Holden Karnofski used to live together, sometime before Karnofski married Dario's sister, Daniela, the other co-founder of Anthropic. Separately, SBF was also known to live in communal housing with many of the other chief executives of FTX.
Separately, that other article also mentions harmful psychological practices like "debugging" and sleep deprivation.
Note that I make no claims of my own - for all I know both articles could be complete fabrications. I'm merely listing what these articles claim. I personally know none of these people, and live halfway around the world from them.
I don't think the argument is that their sexual practices inherently make them part of a cult. Rather, the argument is that it's very common for cults to include communal sexual practices that fall outside typical social norms.
Common signifiers of cults include:
1. Apocalyptic world view.
2. Charismatic and/or exploitative leader.
3. Insularity.
4. Esoteric jargon.
5. Lack of transparency or accountability (often about finances or governance).
6. Communal living arrangements.
7. Sexual mores outside social norms, especially around the leader.
8. Schismatic offshoots.
9. Outsized appeal and/or outreach to the socially vulnerable.
As a polyamorous person, it seems quite concerning that effective altruism people are labeled as members of a sex cult, due to their private lives. Intellectuals should be attacked on the basis of their ideas, not based on their private lives.
That would be a reasonable argument if it were indeed their private lives, and they didn't go around offering to throw gangbangs for AI safety researchers.
The word "cult" is doing a lot of heavy lifting here. It can mean anything from religion, to weird religion I don't like, to the organizations that try to cut you off from contact with your family and other people outside the organization. Only the last type are the bad ones, from what I can see.
Cult quite explicitly means the latter, and is what the article is explicitly accusing the core of this community being (that is, not everyone reading LessWrong, but specifically the actual people living in the communes).
No one calls the Catholic Church "a cult", nor do they call, say, Zoroastrianism a cult (to pick a relatively small religion with weird practices).
I will note, if, like me, you're not a native English speaker, that there are two words, "cult" and "sect" that commonly exist in many European languages but often mean opposite things. That is, in English a "sect" is any "flavor" of a religion - catholicism and protestantism are sects of christianity; while "cult" describes closed off communities (usually of a religious nature) that are deemed harmful to their members. In my own native language, the two meanings are reversed - we use "cult" for "legitimate" religions, and "sectă" for closed off harmful religious communities.
Then I don't understand your complaint. The article author is very explicitly using definition 1. It is explicitly accusing the community built by Yudkowsky of presenting signs of being "a group (such as an organization or religious sect) with tenets and practices regarded as coercive, insular, or dangerous".
The article is certainly not using definitions 3 or 4, which simply don't make sense in the construction "sex cult". It's not worship of sex/worship through sex (definition 4), and it's not "a system of religious practices related to sex" (definition 3).
Someone who hasn't been reading the object-level arguments for 10+ years might reasonably wonder if these are good heuristics. Here's an additional data point: I think polyamory is stupid, I've read the object-level arguments, and this entire thread is ridiculous.
Everyone who works at a frontier lab should be in prison, AND ALSO the people talking about 10% risk of human extinction are both sincere and correct despite their many personal failings, AND ALSO the frontier labs have already created AI that could Hugging Face a biolab with smallpox or worse, AND ALSO they aren't going to slow down or lose any customers if you adopt some galaxy brained strategy where you spitefully tell them they aren't allowed to stop accelerating because polyamory is stupid.
Possibly. That's not a word I trust myself to use correctly. What I meant is that the object-level claims require a lot of time and energy to evaluate, and it's sometimes reasonable to use a heuristic like "sex cults are not trustworthy" to save time, but if you've already read the arguments then the heuristic loses almost all value.
Yudkowsky and friends believe many things, and reasonable people can disagree about many of them, but "at least 10% risk of extinction if we don't slow down" is a relatively straightforward conclusion based on the offensive capabilities of unreleased models and the terrible state of biosecurity and cybersecurity.
Even without AI, the risk of extinction might be above 10% if people with the relevant skills were interested in using them that way. AI adds an alien willingness to achieve poorly specified goals at any cost.
Hm I think in terms of "can I trust a group of people" we're talking about heuristics yes as in: "how can I quickly approximate an answer to the question of trust?"
Like that would help. The rest of the world plain refuses to engage with the arguments they've been consistently presenting for decades.
What changed is that they can no longer be dismissed as bunch of loonies who confuse cheap science fiction with reality, because that "cheap science fiction" became the actual reality. So obviously, instead of considering the arguments, the next step is to try and paint them as loony sex cult, because this lets people continue to do the stupid thing and then act surprised, and proclaim no one could have predicted this (and the "sex cult" obviously does not count because it was a "sex cult").
I don't think that's true. It's hard to argue that Elon Musk, the richest person and spokesperson of tech for a lot of people, never participated in SV sex parties or had weird sex behaviors. Or Trump, the president of the USA, has a clean history of standard sex as well. Or Bill Gates, the nice philanthropist, never engaged in weird sex circles. But that doesn't make them lose their leadership positions or status.
When the sex cult is built around the idea that they and everyone else should fear and respect the false, anthropomorphized God they've created in LLMs, then yes.
That makes them stupid and stupid people should not be allowed so close to the wheels of power and history.
It's not even about immorality. The problem is not so much the sex, it's the cult part. If a good deal of the people working in AI safety in the biggest AI lands are actually a cult, that has deep implications about what AIs are aligned to, and how (ironically) irrational they might really be trained to be, by the very people trying to control them.
Imagine if you found out that most people working in AI safety were high level members of Scientology - wouldn't that give you some pause about where this field is going?
I’d rather be in a “sex cult”, whatever that entails, than this ridiculous culture that’s somehow still doing moral panics over sex. Maybe it’s just deeply human, or maybe it really is stabilizing society to relegate sex to something dirty and private, but I have a feeling it’s just a dumb conflation that’s so easy to exploit it’s never allowed to go away. It’s like the “think of the children” thought-stopper and we’d do well to get past it.
Naturally, any culture has misfits, radicals and free-thinkers, and so naturally a sex-negative culture will produce at least some focus on sex in its counter-cultures. Doesn’t seem very interesting.
This isn’t moral panic, it’s saying “hey, all these influential people are intimately connected across the strongest possible dimensions (money, sex, belief and legal grey areas), we should probably pay attention to that”
For some reason the bit that stuck out to me in a wild ride of an article was the guy who believed a super-intelligence was a threat to humans but distanced himself from veganism as if that was crazy part of the AI-doom sex cult that he disagreed with.
a) because veganism seems as exotic to me as someone doing CrossFit
b) the very obvious parallels of a more intelligent creature torturing and killing large numbers of other creatures for relatively little benefit.
They also downplay global warming, nuclear war, wealth inequality, and in general anything other, less cool people they didn't want to identify with thought was important.
If they had thought, yeah, "the majority is more or less right about what the big issues are", there wouldn't be enough room to construe themselves as uniquely intelligent.
Label them what you want. I'll go with EAs if you insist.
Sure, they're not entirely monolithic. There's room for "EA, but vegan", "EA, but global warming still matters", maybe even "EA, but vaguely socialist". As I keep saying, the core belief is that they're special because of their intellectual superiority, anything else is negotiable.
Not all of them downplay all of these, but yeah, most of them do. They may have one or two areas of dissent (for EA as a self-identity-building project, do you see why that is to be expected?), but there is a core of beliefs. Yudkowsky's.
And he's downplayed all of these.
I read it more like a conformist performative veganism. I think the implication is that this group treats veganism and all of these beliefs as a kind of shibboleth, an in-group marker for which conformity is expected. The immedaite "feeling" associated with a cult is the feeling of "high control", cf. https://www.google.com/search?q=high+control+group
I read it that EA practices (incl. veganism), beliefs, "rantionalism", and doomerism are all aspects of the high control environment
Nowhere, that's exactly the problem. People involved in Epstein's sex cult are the respected members of our society. These are some weird nerds who have consensual polyamorous sex with each other, therefore their opinions on AI should not be taken seriously. /s
More of this madness coming out of the west coast of America, in particular California.
The Manson family, Heaven's Gate, NXIVM, Rationalists, Zizians... what next?
All turn out to be rotten to the core with their need to reshape and craft their own paradigm, which then gets warped by egos, individualism and power.
This is the real San Francisco that I know, not the one that Hacker News always talks about. It's a haven for weirdoes that has attracted the strangest of each generation since at least the Beatniks in the 50s. It probably descends from all of the depression-era westward migration and people from earlier than that coming to Hollywood to become movie stars. San Francisco attracted a different type, artists and hippies, Tim Leary followers. The deserts are the strangest places. That's where most of the true cult shit happend. Jack Parsons trying to summon demons. Gram Parsons' friends stealing his corpse to bury him at Joshua Tree. The traveling folk that Chris McCandless took in with near the Salton Sea.
None of these people are actually from California. Eliezer's from Chicago. Zvi is from New York. Scott Alexander is from Michigan.
As an actual multi-generation California native (who hasn't lived there in over 16 years), let me tell what most of us are actually like. We're mostly Mexican. Catholic. Pro-labor but kinda socially conservative. There's also a shit ton of refugees from 20th century wars in Asia and the collapse of the Soviet Union. Huge populations of Laotians, Cambodians, Koreans, Vietnamese, Armenians, Persians. Tons of religious diverity. The largest Buddhist temple in the western hemisphere is in my hometown. Los Angeles has the second largest Mormon temple in the world. It's got the headquarters for Scientology.
I think that diversity is what plays into the cult haven atmosphere. There is no single dominant religion. Everyone can find acceptance somewhere, no matter how strange they are.
Well, a cult is a high-control social group that's difficult to leave and requires you to believe and do strange things that are likely bad for you or at least take up all your time and resources. And a sex cult is what it sounds like.
Good to see some raising the issue of the sex cult that Effective Altruism is:
I'd like to hear what feminists have to say about that picture in TFA of that woman and six men watching her. Just as I'd like to hear what feminists have to say about Amodei's wife who used to run a porn site.
And while a sex-cult abusing women is one thing. Theft is another. More emphasis should be put though on the lying scums in their ranks, like the polyamorous SBF and Caroline Ellison or their guru and its $15m ill-acquired mansion in the UK (which hasn't been clawed back yet but could).
And speaking of lying scums stealing money, we know about several making constant announcements were they lie, lie and then lie some more to ensure a never-ending stream of money and insane valuation of their proprietary, closed, not altruistic at all companies.
A despicable movement and I, for one, I'm very happy that there are companies out there releasing open-weights models so that the thieving-and-sex-cult at least get some competition.
> I'd like to hear what feminists have to say about that picture in TFA of that woman and six men watching her.
That picture is literally a joke based on a meme, which is based on a porn star's promotional image. To the extent feminists care, I would expect that care to be about the underlying subject matter rather than about this satire of it.
Is it really a satire if the same woman in the photo (Aella) went on to have a "birthday gangbang" with numerous men, some of whom are present in the photo?
He does not understand the issue. Not presenting any backing for his ideas, just emotions. The idea has been misused by the bros immensely, that does not help to grasp it either. But AI extermination of humans is a well known problem we have no solution for. Luckily AGI is not so close.
But AI extermination of humans is a well known problem we have no solution for
Sure, but the realm of fictional problems is so huge that we just can't come up with solutions to all of them. I like watching the Terminator movies (well, the first couple) as much as the next person but I can't be spending my time imagining how we could defeat skynet.
"Somewhere, some rationalist-brained person is reading this and saying I'm a "low decoupler", and that whether they run a sex cult has nothing to do with whether they're correct."
I guess the EA and rationalist cult members are out in force since this got flagged. Not a good look guys, you're just confirming what everyone thinks.
It may have been flagged as the title being misleading, as it did not really present as many details around the sex part as the title suggested.
Jokes aside it would have probably flown a bit better by removing the sex from the title, as the issue and problem brought up is about it being a cult, and not about the sex part specifically.
I meant about the title as used by the author, not the one who linked it here. But in any case I said it was a joke about not giving many sex related details in the article.
I mean I saw it 10 hours later. I'm thrilled to see stuff like this getting multiple hundreds of upvotes on HN, one of the deepest dens of tech capitalist worship in the world. It performing here is a couple steps from it performing on LessWrong itself. I take that as a sign a breakthrough is nearing. It is probably linked to whether the AI lab IPOs are successful or not.
The really amazing thing is, if the Yudkowskians are this weak, how doomed is the rest of our society that can't put up any defense to them completely dominating the narrative about where our economy is heading?
90% of this is just driven by strong emotional reactions from someone that feels uncomfortable about people being open about their non-standard sexual life, in the same way that many people have weird negative reactions about polyamorous relationships in general.
The remaining 10% is just the author trying to sprinkle in cult-like insinuations to justify his contempt.
Regularly meeting with people that have shared interests and ideas is not a cult.
Finding some of those people attractive and having sex with them is not a cult.
But somehow the fact that the romantic/sexual relationships are not traditionally monogamous makes it a cult?
I too wrote about this, although _much_ longer, if you want a bit of a deeper dive on the subject https://www.iankduncan.com/personal/2026-09-16-sex-ai-and-th...
That was a good read. A little long as you say, but this thing has been around for over 20 years now, it can't be helped.
Predictably the original submission got flagged.
why use AI to write this? It makes me wanna stop every five seconds.
Thank you for writing this piece. It was completely worth the length and amount of ground/time covered.
fantastic. I've heard of bits and pieces of this but this is the first article I've read which lays out the entire timeline with all the firsthand sources you could want. thanks for the effort!
As far as manifestos go HPMOR is a pretty good one. I think that’s what a lot of the critiques about it miss. Yudkowsky is actually a pretty decent writer, especially in the fan fic scene. Not only is his prose actually acceptable which is pretty rare in fanfic it’s clear that he actually thought through a lot of the plot points ahead of time. Harry Potter universe fits very well for his story not only because it’s popular with the target audience, but because there’s a ton of unexplored authority figures in the original and the main goal of the manifesto largely boils down to rejecting blind belief in authority and blazing your own trail. I think it’s hard to separate the number of people who read it because it’s legitimately the best Harry Potter fanfic from the people who read it for the manifesto stuff which again points to the ingenuity of choosing it as a setting.
For me this is probably the weirdest part of it. I have read HPMOR and was distinctly underwhelmed by its literary quality. I mean, it's not bad, but it's not serious literature. It's not even on the level of the original HP. It's a fanfic. Sure, it's a fanfic with a lot of fan service and geekery and so on, so the audience is targeted masterfully. But still, it's hard not to notice the numerous literary faults. I mean, "he actually thought through a lot of the plot points ahead of time" is a kind of praise, but not one that a decent literary work expects. That's like saying about a chess grandmaster that he actually knows how all the pieces move. I mean, sure, on the fanfic scene it's probably above median, but I've seen people making out of it some kind of epic revelation, akin to some kind of a sacred text. Completely undeservedly, in my opinion, and rather weird for a community that is so proud in it's objective and rational approach to things that put it into its very name.
I suppose there is worse sci-fi you could launch a cult from (Dianetics)
It's terrible as a work of literature. It has many, many, many flaws, but the most serious is that it's boooooring. Hundreds upon hundreds of pages go by with nothing happening except creepy children preaching at each other.
It's terrible as a fan fiction. Not a single character acts anything like the original, the world is unrecognizable, and the plot is derailed entirely by the end of the second or third chapter. The original story needs to be distorted beyond recognition because the author's bizarre worldview works even more poorly in a fantasy world than it does in reality.
It's demonstrably effective as a manifesto, but that is despite the fact that it is riddled with scientific and conceptual errors.
It's one of the worst things I've ever read. The fact that people like it is evidence of the bizarre incestuous nature of the community that produced it.
War and Peace is shorter than HPMOR.
Anyone who knows anything about Harry Potter fandom in those years knows that its readers were always thirsty for something new in the Potterverse to read or be exposed to. By latching onto that universe's incredible popularity, Yudkowsky exposed a large number of young, inquisitive people to the ideas of rationalism. It was essentially a recruitment mechanism, one that was wildly successful. Many rationalists freely admit that their first exposure to rationalism was via HPMOR.
I don't think Harry Potter fans would look kindly on their favorite story being treated with such contempt. Remember that within about three sentences of introducing Ron Weasley our hero finds him guilty of the heinous crime of enjoying sports and concludes he should not exist.
Excuse me. It was Draco Malfoy, our rapist deuteragonist, who said that. Our hero merely agreed.
Maybe Harry Potter fans would read a bad fantasy book, but HPMOR is not just a bad fantasy book. It is repugnant to healthy people from the first few pages.
I really, really dont understand this complaint at all. Harry and draco disliking ron is supposed to show that they are snotty kids who have big blind spots. The characters are supposed to be dislikable at the start, the story is about the growth they experience from there. This is part of what makes it a better story than the original. Yudkowsky's harry grows as a character which basically doesnt happen at all in the original series.
Do you dislike succession too? Some people just need completely heroic protagonists in their stories. Thats fine, but it doesnt make the story bad.
They are introduced as insane, not snotty. Nearly the first thing Draco says is that he plans to rape Hermione and our Hero uses his Agency to do exactly nothing about it. (It is hard to exaggerate how bizarre this story is to people who haven't read it.)
Hundreds of pages later, Harry uses the first iota of power he is given to prohibit students from playing Quidditch according to rules he believes are irrational. There is an immediate outcry which he dismisses because it's just the mewling of inferior intellects. It's hard for me to see any signs of growth here.
(Harry's contempt for sports is amusing, given that about a third of the story is dedicated to his love of cheating at a particularly moronic sport invented by the Lord Voldemort ripping off Ender's Game.)
He convinces Draco that his whole ideology is wrong. He convinces him not to rape anyone before he rapes anyone. Not exactly sure what the better outcome is here, just turning him in would only strengthen his violent ideology. There are many complaints to be made about the story, as you said its boring and too long and often too preachy. A lot of the visible changes in the characters happens in the last quarter. Even though the whole story has been building to that it can seem like its a bit out of nowhere.
Does Yudkowsky spill more ink on condemning Draco's hatred and violence or on making fun of Hermione Granger's feminism? Voldemort is treated as an ultimately worthy adversary, while Ron is a joke and Hermione is a pet handed power by our Awakened Hero. Yudkowsky treats sexual violence, genocide, and mass murder as forgivable missteps on the road to greatness, but treats liking sports as a fatal character flaw and being concerned with human rights as risible. Arrogance is consistently rewarded while goodness is consistently ridiculed.
I don't claim Yudkowsky believes these things consciously, but they are omnipresent in his story and they reflect his bizarre attitudes to morality. The fact that he acknowledges at the very end that the characters he created are fools doesn't change the fact that he spent six hundred thousand words holding them up as paragons of virtue and rewards them with apotheosis.
Hermione is the actual hero of the story. Every time rationalist harry makes a mistake he is acting against her direct advice. Her character literally exists to show off his deficiencies and his coming of age basically boils down to realizing he should be more like her.
For many reasons voldemort is an ultimately worthy adversary. Much of that has to do with Harry's biases that come from being too invested in rationalism. Since he considers himself above everyone he's ever met he longs to find someone he considers his better and voldemort fills this role. He wants a mentor so badly that he's willing to throw out his entire ideology to keep him. I think yudkowsky's portrayal of voldemort as the ultimate adversary to rationalism is better than pretty much any plot arc in the original story. Harry does ultimately completely reject voldemort so Im not sure how sexual violence, genocide, and mass murder as treated as forgivable. From the very beginning Harry is deeply concerned with human rights, his distractions and the path he takes toward achieving human rights in this messed up world is again an integral part of the story.
> The fact that he acknowledges at the very end that the characters he created are fools
I think this is my biggest issue with your characterization. Yudkowsky shows harry to be a fool from the very beginning, as you pointed out with the draco/ron stuff. With his dismissal of much of hermiones advice. He very consciously did not create harry as some ubermencsch character who can do no wrong. He shows that harry believes himself to be a parity of virtue, but that he very much is not until hermione's death.
This is a forgivable statement if you didn't read it, because then I guess you may not have seen that there's themes at play and a point being made. If you did read it I do not understand how you don't see how you are meant to dislike the summary judgement and the judgement itself being an effect of yet another plot point.
But then if you did not read it it seems a little disingenuous to say "this is a bad book", because you would not only not know but anything you did know you would not understand.
What is there in the story that indicates this judgment is wrong? Ron does not appear in the story again, while by the end Harry is an immortal superbeing. His arrogance is never once punished.
You can be a sexually predatory sociopath and still be of value in Yudkowsky's world, but like sports and you're dismissed in your entirely within a single paragraph.
> What is there in the story that indicates this judgment is wrong?
The ending? https://hpmor.com/chapter/122
> "Ah..." Harry said. This was very hard to say. "Actually, the rest of the world doesn't know this part, not even the Headmistress. But in point of fact you were one hundred and twelve percent correct about [spoiler], and I'll remember for future reference that although 'Dark' and 'evil' may not technically be the same thing, there's a great big statistical correlation."
> "Oh," said Hermione, and fell silent again.
> "You're not saying that you told me so?" said Harry. His mental model of Hermione was yelling: I TOLD YOU SO! DIDN'T I TELL YOU SO, MR. POTTER? DIDN'T I TELL YOU? [spoiler], I SAID, BUT YOU DIDN'T LISTEN TO ME!
It feels like most people who criticize the story didn't actually read it, and just start with the premise that Yudkowsky sucks at writing, then conclude that obviously Harry Potter must be Mary Sue, and that whatever Harry says is what Yudkowsky wants you to believe.
If you actually read the story, you will find out that many things that Harry said and believed turned out to be wrong. That ending was written by Yudkowsky, too. The characters that disagreed with Harry since the beginning, such as Hermione and Dumbledore, were written by Yudkowsky, too. Perhaps there is a little more depth in the story than if you assume that it must be a Mary Sue story.
Ah, yes, the ending where the invincible Superman uses his gigabrain to singlehandedly save humanity, end death itself, get the girl (and turn her into an immortal magical unicorn princess (yes, really, this story is dumb)), vanquish a dozen Wizard Hitlers at once, and tear apart entire stars for their raw resources shows that he paid the price for his earlier arrogance.
I have read the story in its entirety. Don't ask me why. The fact that our Lord and Savior holds the idiot ball on occasion does not make him realistically flawed. The fact that Voldemort would have realistically been exposed within thirty seconds makes the story dumber, not smarter.
Also, you ignored the question I actually asked: where does Harry realize that it is wrong to steamroll everyone he thinks is stupid?
> I have read the story in its entirety.
Then how could you make so many mistakes in one sentence?
> end death itself
was done by Flammel
> turn her into an immortal magical unicorn princess
was done by you-know-who
> tear apart entire stars for their raw resources
is foretold to happen in unspecified future, outside the story time frame
>he author's bizarre worldview
what is the author's bizarre worldview, other than the AI will make us extinct view?
> It's terrible as a work of literature. It has many, many, many flaws
Pretty much all fan fiction is terrible literature. I find yudkowsky's prose to be above average for fan fic which admittedly is just saying hes among the best of the worst. Every single fanfic ever written that doesnt end after 1 book has significant boring sections, theres no editor.
> Not a single character acts anything like the original
This is the point. Yudkowsky recognized that the hp world acts as a natural foil for his ideology. By changing the characters he points out the original characters flaws, but also the ways they are better than the rationalist versions of the characters. I dont understand why people claim fanfic should be always be faithful to the original setting. Using it as a foil can work too. The characters in the original, especially the authority figures are all incredibly flat. Why would you want more of that? Even the ones he largely keeps the same, like dumbledore, he explores in a new light which makes them seem different even though they are very similar to the original. The reveal that the 2 Dumbledore's are actually very similar(the exact same?) characters is central to understanding rationalist harry's deficiencies.
> The fact that people like it is evidence of the bizarre incestuous nature of the community that produced it.
Agreed but I think this says more about how crap HP fan fic tends to be than anything else. Yudkowsky saw unmet demand and used it to shoehorn his ideology in
Agreed. It's quite bad. Like, even for fanfic, it's not good. And a lot of fanfic is written by fourteen year olds.
What are your favourite quotes from it?
Let me be clear, he's no cormac mccarthy(my favorite prose is the passage from blood meridian describing the raiders). My bar for fan fiction prose is basically whether it annoys me, its a low bar. Most of what makes hpmor good is the cleverness with which yudkowsky subverts the original harry potter world to try to make a rationalist argument, whether that be by inverting a character or playing off the harry/voldemort relationship in thoughtful ways.
'"Voldemort," said the old wizard. "I understand him now at last. Because to believe that the world is truly like that, you must believe there is no justice in it, that it is woven of darkness at its core. I asked you why he became a monster, and you could give no reason. And if I could ask him, I suppose, his answer would be: Why not?
They stood there gazing into each other's eyes, the old wizard in his robes, and the young boy with the lightning-bolt scar on his forehead.
"Tell me, Harry," said the old wizard, "will you become a monster?"
"No," said the boy, an iron certainty in his voice.
"Why not?" said the old wizard.'
as you can see its nothing special, just good enough
I see, thanks. It reminds me a little of those spam emails which are sprinkled, intentionally, with obvious spelling errors. They aren't trying to trick the average person, they are trying to filter for a much smaller, more valuable audience.
fwiw, I don't think that was intentional here. He's just more interested in "ideas" than prose or people. A more extreme example of Andy Weir. Definitely a niche. I read it years ago while backpacking and it perhaps needed an editor but the overall struggle/war and reimagining of logical fallacies of the HP universe's magic were fun.
yes exactly. Fan fiction readers largely do not care about grammar or nice prose, just has to not be overly distracting which unfortunately most fan fic fails at
> His autobiography even opens with a parable: as a small child he threw his favorite squirtgun off a third-floor balcony rather than accept that his mother could lend out what was his, and felt complete when he heard it hit the pavement.
What a classic narcissist.
> Main character syndrome stops being a character flaw the day you join a religion whose plot has you in the lead.
This is some great writing.
I've tried to read this, but somehow the article doesn't really come to a point. Is there any point besides Yud not being perfect, and having connections to people who openly talk about their sex life (and that it's somewhat non-standard)?
I think the point is that Yud and the rationalist community have put themselves in an awkward position where their lifestyle now depends on "p(doom) is big" being true. Which means they can't necessarily be trusted to give unbiased updates on the true level of risk we're facing as new data and information comes up.
Can "ASI will inevitably kill us all" be falsifiable before we create ASI? If not, then what separates it from religion?
Not sure how much I agree with that position, but that's just how I understood it.
> Can "ASI will inevitably kill us all" be falsifiable before we create ASI?
No, but there are decent proxies, like "Alice who is X standard deviations smarter than Bob will win >50% of the time at game Y even given a disadvantage that would make her chances <5% if she was playing against someone equally smart".
I've thought about this analogy for less than five minutes, so it's far from perfect. Point is, ASI being made of silicon isn't very central to the problem. There are lots of analogies that are falsifiable, and "ASI will kill us all" can be decomposed into a simple and bulletproof composition of falsifiable beliefs, even if it is too specific to be itself directly falsifiable.
Uh oh, your opinions are merely part of the funnel to the sex cult. Can we really trust them? /s
There's are many people with high p(human extinction) whose salary does not depend on their continuing to profess a high p(extinction) -- me for example. Also Yud's friends have promised to pay him what he is earning now if the organization he works for ever has to reduce his salary because the organization lacks funding. They did this to neutralize the financial incentive he would otherwise have to hesitate to announce that his p(extinction) had dropped.
> what separates it from religion?
The rational methods used to establish a large P(doom).
To definitively falsify the claim, humanity would need to live alongside superintelligence forever without it wiping us out. That doesn't mean we can't be pretty darn sure though, in the same way that we can be pretty darn sure that nuclear war could end civilization.
Those "rational methods" are essentially just pascal's wager. You stick some ~infinitely bad outcome (nasal daemons torturing simulations of you and everyone you love for all time) and/or ~infinitely good outcome (perfect paradise for as many sentient minds as could fit in the physical universe), then you can get whatever conclusion you want by multiplying any finite probability by these things, and even give yourself bonus asspats for being conservative by adding several order of magnitude fudge factors against your claims.
The rationalist doom and sex cult uses convoluted mathematics inspired language to justify their faith-- but at the end of the day it's just faith no matter how they dress it up.
you say "falsifiable" as if it holds some important key, but the concept is really not applicable to predictions that by their very nature, can only ever have one data point
(doesn't mean that it's not a religion or it is, but rather I mean it being "falsifiable" or not, doesn't carry any information to that regard)
Why does their lifestyle require p(doom) to be large? Because they work at orgs focused on AI safety? I'm not sure what the claim is here.
Because they depend on a supply of anxiety fueled doomers to both fund their lifestyle and provide a supply of willing participants for their control and fetish games.
> their lifestyle now depends on "p(doom) is big" being true
How?
Suppose that for the following 20 or even 100 years nothing happens. The AIs miraculously stay as they are today, the further progress turns out to be impossible, there will be no more agents hacking various websites, no cyber warfare involving AI agents, nothing. There will be the year 2026, forever.
How would any of that impact the lifestyle of anyone in the rationalist community? Please be more specific with your accusation.
(Seems like the rationalists are simultaneously the dark lords of TESCREAL, pulling the strings of the world's most powerful billionaires and politicians, but also irrelevant weird nerds who can be ruined at any moment by a smart blogger explaining how their opinions are silly, or even that they know each other.)
As an example, in that example who pays for the orgy compound on Telegraph Ave?
I tried too, but found no sentence that makes any sense. It's all gibberish from top to bottom. I'm not a native speaker but my only reaction was "wtf this is all about and wtf are you even trying to say???"
it's the Ed Zitron style of arguing that something big/expensive/unprecedented/hard is bad, going to fail, because [____________].
and if you put something trivially true into the blank the implication seems logical, and the argument feels powerful. but simply putting a true claim at the end doesn't mean that everything before follows from this claim. (and even if they do the argument is not a total description of all the causal chains that influence the outcomes.)
of course it's hard to do a careful survey of the facts and weigh them and then deduce some likelihoods for possibilities. because some aspects are unprecedented (investment ferocity, AI results/progress) while others are very much precedented (collapse of investment bubbles, doomsayers' wrong predictions).
Just because something was called bad, predicted to fail and actually didn't fail, doesn't make it "definitively not bad, because it didn't fail, I told you so".
Case in point - Bitfinex was and is absolutely a criminal enterprise and scam, and it didn't have any backing for years in row. But it didn't fail, because many different criminals in the world had decided that it is quite convenient tool and put a monetary IV line in it. And now ruzzians are buying kamikaze drones with it, selling oil to India with it, drug cartels are shuffling around their money with it and so on. Does it mean it wasn't and isn't bad? Of course no. They simply found a rich sponsor to bail them out.
Or the whole 2008 crisis is another popular example.
I'm not saying absence of evidence of wrongdoing is evidence of absence of wrongdoing.
But note, you have very concrete things to claim about Bitfinex. Criminal enterprise, scam, organized crime funding, facilitating arms dealing and embargo avoidance, and so on.
Then you throw in 2008. It's definitely popular but example of what? Bad thing that didn't fail? (Things did fail, hence the crisis? Or you mean the too big to fail attitude?)
Policy proposals (eg. "coordinated AI safety R&D should be exempt from antitrust regulations" or "TARP made money on paper, but the financial sector should pay for the total cost of the crisis") cannot be divorced from the provenance of the source (sex cult, goldbugs, anticapitalists), but the same is true of bluesky screeds.
2008 was a shortened mention of the event when multiple companies by all rights went "bad", i.e. should have went bankrupt on their own merit and performance. But an external entity bailed them out and so they continue to exist and work later on. But does this mean that those companies (like Fannie Mae et. al.) weren't "bad" in 2008? No, it doesn't.
Same logic with pure AI corpos and adjacent dependents. If the doomsayers would ever be right about them being financially unsustainable, then most likely these companies would never get even close to the bankruptcy, simply because they would be bailed by external entities. So we would never be sure if OpenAI and Anthropic were "bad" or not, financially. Because either critics are wrong, or any corporate deficits would be covered by budget money (maybe foreign budget, it happens too).
ai alignment and safety is captured by a fringe network of batshit insane rationalists.
METR, AISI, MIRI, SBF, FDX, redwood research, yudkowsky, nate soares, anthropic.
frankly they are connected in something almost resembling a conspiracy (XD).
lower the confidence score of any institute connected on account of their conflict of interest with this religion.
Read to the end... The front matter is context to the conclusion:
> This all most recently came together over the weekend, and they got Dario Amodei and Sam Altman and Elon Musk and Demis Hassabis all onto the same page to say they wanted to WAIVE ANTI-TRUST LAWS so that they could "pace", or slow down, research.
> I feel about this roughly the same way I would feel about all the pharmaceutical companies getting together to say they wanted to waive anti-trust laws so they could form a cartel to ban medical research:
> these people are the enemies of humanity.
> Also: Much of AI research IS medical research. They are very literally proposing to stop a pretty good chunk of medical research in its tracks so that they don't have to fear the paperclipper.
> There are many regulations I would be in favor of, but they only ever want to form a cartel or a world government against the paperclipper.
> No other industry has ever framed any regulation in terms of "stop", "pause", or "slow down". Those are insane ways to consider any form of policy. Policy should prevent bad outcomes and encourage good ones, it is never about some specific speed.
> This industry frames its regulations in this way because it has, at its core, a millenarian death cult that worships and fears AI as the cause of the end of the world, that feels compelled to create AI, and that at least somewhat wants to destroy AI as a field.
There are better ways to make this criticism. The main problem I see with this is that it's basically a long ad hominem, followed by some valid concerns about market concentration at the end.
Remove all the ad hominem and what remains is two paragraphs long. Valid ones, though.
Do you mind excerpting the valid two paragraphs?
That was a figure of speech. It boils down to the fact that, by asking for laws mandating AI research to slow down, the incumbent Big AI firms get regulatory capture and cartel power, which might be the real motivation underlying the apocalyptic language (follow the money and all that).
The other thing that annoys me is that there are plenty of folks in the AI safety space who do not think very highly (at least now) of Yudkowsky. One of the comments I saw recently on a Substack article talked about how Yudkowsky has "beclowned" himself by not updating some of his earlier theories given how LLMs function.
So it's pretty annoying that this article is trying to paint everyone who cares about AI safety is in a "sex cult". There are lots of folks who are genuinely concerned about risks. If anything I discounted a lot of the AI safety folks because I thought they were not seeing the underlying limitations in LLMs. That may still be the case, but agents keep breaking through what I thought were limits on their behavior.
If anything, I don't understand at all the folks who think ASI is on the horizon (< 10 years) but think everything will be hunky dory and easy to control once we actually do build superintelligence.
the sex cult is the sensationalist draw. you don't get readers paying attention in our current content economy if you don't have something salacious
which is also to say - I found it very funny and enjoyable to read and it illustrates how much of dorky little echo chamber that group is. this whole idea that you can become some Spock-like, logic-driven creature in spite of our biological wiring fundamentally opposing this seems to underlay all of this. and the fact that they're all orbiting around one 'cool poly girl' and 'visionary dude' is hilariously similar to the kinds of very formerly-WASPy-and-now-just-WASy hetero polycules that I, in my more youthful and adventurous days, hung out with (and quickly departed after getting icky vibes). like down to the fashion choice and need to constantly prove how smart they each were at all times (do they, too, get really high and put on some anime with way too much fan-service and proceed to have orgies? one wonders)
those are the types of people who always end up actually having the worst coping skills because they can't even admit that they might be emotionally-driven sometimes, or that they have emotions at all that they might need to develop the cognitive skills to deal with in a pro-social way. or they get called out enough that they finally read a little about therapy and just end up weaponizing psycho-therapeutic language for their own ends (instead of, say, practicing the skills, doing the workbooks, developing insight)
which is how your anxieties might lead you to believe that the sky could fall at any moment (and thus the whole nuclear war and the 'don't trust bust us!!' bit)
I think the part that is annoying me the most is that one of these dorky little polycules has somehow found their way to the levers of power and are just mashing all the buttons and dialing all the knobs. these are honestly the last people on Earth that I'd trust with having any kind of influence
> the sex cult is the sensationalist draw. you don't get readers paying attention in our current content economy if you don't have something salacious
Nerds used to be shamed for not having enough sex, now they are being shamed for having too much sex. When will they finally learn to have the average amount of sex, so that the public discourse can finally switch to something else?
They could try just being normal. There's a middle between being unfuckable and hosting rape gangbangs with 40 people. This article actually does a great job describing how the gross, unfuckable nerds are now the ones pushing all of this poly sex cult stuff.
it's also possible to have healthy, non-hierarchical poly relationships. many queer people do it with triads, quads, etc. most of the hetero poly people I met and were icked out by functioned as extremely hierarchical in practice (but they would love to tell you about how they were relationship anarachists or whatever) with a couple at the center and one of the partners being the one that pursued having a lot of other partners while there was hidden pressure for the other to also engage in the practice so they didn't seem patriarchal or toxic (a cruel irony, in retrospect)
that said, my observations have a sample size of like... four couples maybe? (they're indistinguishable from one another now). but the dynamics I saw were four for four which seemed good enough for me at the time to dip out of that scene
One might also argue this is more mainstream in some cultures, and it's just not talked about. In countries such as Japan or France it's not uncommon to have a mistress (or male equivalent) but also having a primary family. That's hierarchical polyamory with uh, less steps.
certainly less calendaring and meeting invites, I'm sure. I did meet a couple of people who identified as non-monogamous well after I had met my very monogamous partner (whom I love deeply and am happily monogamous for) who seemed very cool and were equally put off by the polyamorists they had interacted with. but they were truly diamonds in the rough, at least in my major metropolitan city, and not a single one of them was hetero or close to being a WASP so you're probably right about the cultural tidbit
There's a great movie or Streaming series on this post somewhere.
Sometimes people can be too smart for their own good. Like a frontier model in an eternal hallucination loop, you can create entire seemingly consistent realities in your head and end up bringing others into it.
> There's a great movie or Streaming series on this post somewhere.
More like the worst porn film ever made.
Theres the title for it when it's on Netflix
In case the author is reading this: I’m getting a Cloudflare worker exception when accessing the page using one of my uplinks, possibly because of IPv6.
TLDR: awareness of the SV "rationalist" polycule has breached containment and normies are reading about it now. [0]
Maybe a "better" source is the NY Post article [1]. The information itself isn't really news - it's even largely available on wikipedia [2] [3] - but the fact that it's hitting mainstream-adjacent news outlets is novel.
[0] https://bsky.app/profile/segyges.bsky.social/post/3mvom4b4dn...
[1] https://nypost.com/2026/09/16/business/ai-doomsayers-wild-li...
[2] https://en.wikipedia.org/wiki/Aella_(writer)
[3] https://en.wikipedia.org/wiki/Rationalist_community
I did too, but it went away after some refreshes.
Some more feedback for author, if reading this.
> Yudkowsky's most viral idea, the one that infects the most people and spreads the farthest, is that AI, if we continue to develop it, will inevitably 1) achieve superintelligence ("RSI"), 2) escape human control 2) kill everyone ("paperclip them", "doom")
That's too many twos!
I don't really get why people go on about Yudkowsky in this regard. I mean serious thinking about AI goes back at least to Turing in 1950 with his test and prediction computers would get there in about 50 years and presumably keep improving. And people have worried about it since then.
Like in 1949 Norbert Wiener saying of AI type stuff "every degree of independence we give the machine is a degree of possible defiance of our wishes."
Then 30 years later this Yudkowsky is born, later is considered a bit of a crank, goes on a bit about the same old stuff when he grows up and everyone makes out like he invented it?
I've got a lot of respect for Turing, Wiener and Von Neumann but haven't bothered reading Yudkowsky. Is there any point?
It seems a bit unfair to the pioneers of the field to go on about some Harry Potter sex cult guy who repeated their ideas decades later?
Regardless of anything I would have more respect for anyone who didn't explicitly optimize the atomic bombs on Japan for civilian casualties like Neumann did.
Yudkowski is being derided because he wrote some weird fanfic which may or may not be problematic (I didn't read it, I dislike HP too much). And apparently "is considered a bit of a crank".
Neumann actually fucking did it, his calculations for maximum civilian casualties make him quite directly responsible.
I don't think you need to compare Turing, Wiener or Yudkowsky to that mass murderer.
>She has a rape kink
That's enough internet for today, back to playing video games
Well most likely consensual noncon given the amount of vetting present in that chart. It's surprisingly common in women, or just a desire for general "gentle roughness".
And idk why it's really worth mentioning anyway, as a gay furry I suppose vanilla normie straights are STILL hung up on what people do with their sex lives. As long as it's legal and consensual who gaf? Y'all, apparently.
Not opining on this one way or another, but the post buries the lede a bit with a one liner that explains where it's coming from, saying that this is all:
> reasonable evidence that their influence isn't prevalent because they're right, and that they would have ulterior motives for continuing to preach that the world is coming to an end even if evidence showed otherwise.
So how they enjoy their sex lives is dependant on continuing to preach AI catastrophism? I don't buy it. Lots of poly folks out here that probably don't care much about AI one way or another.
No one is claiming that all poly people have some ties to the AI community.
They make a claim about a specific community in San Francisco, that is tied together by a mix of AI doomerism and sexual relations. And it's not at all surprising or some farfetched claim that people who enjoy having sex with someone might adopt their ideas, at least publicly, for fear of losing that sexual connection.
>And it's not at all surprising or some farfetched claim that people who enjoy having sex with someone might adopt their ideas, at least publicly, for fear of losing that sexual connection.
am i naive? in the direction you pose this, i can't imagine it's that common and if that happens i don't feel sorry for our imagined victim. being a bay area guy who attends so many orgies that AI doomerism rubs off on him? and he keeps spreading it to other people and that some substantial bulk of doomerism is due to this mechanism such that it voids the ideological purity of doomerism? i'm not a bay area guy and i haven't attended these orgies but i hope they have other shit to talk about at the orgies than AI
if the relationship is flipped - being genuinely interested in the theory but then being roped into weird sex shit that you didn't ask for, sure that that sounds predatory. a sex cult where AI is instrumental, rather than an AI cult where sex is instrumental. it would be more remarkable imo but i don't know if anyone is saying out loud that's what's happening.
Yeah it is. If they stop preaching AI doomerism then they're out of the cult and won't get invited to orgies anymore.
Hate to break it to you, but I don't think the AI doomers have a monopoly on orgies. You name the political / belief system and I'll find an orgy for you by 10pm.
Neoliberal YIMBYism?
Don't think it was a full on orgy but I've seen some cringey Yimby parties
Are you implying that every rationalist that preaches AI doomerism gets invited to orgies?
Isn't one of them going around offering people "reverse gangbangs" to resign from Anthropic?
No, just the ones that are part of this specific community described in the blog post.
"Mostly" in the title implies otherwise.
The title is senstaionalist. The actual content of the article refers to a specific community, including Eliezer Yudkowsky, Aella, and others.
No, he is saying that if they stop preaching it, they stop being invited. A -> B does not imply B -> A.
So even those who were never invited are afraid that they won't get invited anymore?
Heh, if you think this post buries the lede, have you ever read anything by the community in question? They seem incapable of saying anything without five part Roman-numbered novel length blog posts.
I think this article kind of misses a crucial point. They weren't always AI doomers, that's a thing they picked up along the way. Their core beliefs are 1. That they are the smartest, and 2. That this makes them morally superior because consequentialism (This sort of follows. If you believe in consequentialism, leaving all your decisions to the superior consequence predictor is surely the moral thing to do)
Maybe one day they will stop believing in AI doom, but they will never, ever stop believing they belong to the intellectually and thus morally elect. The day any of them do that they have either broken with the cult or died.
> They seem incapable of saying anything without five part Roman-numbered novel length blog posts.
Maybe that’s why they’re so pissed, ChatGPT’s useless verbosity is threatening to put them out of a job.
I'm tempted to say "You're absolutely right" :)
yeah, it's no coincidence that AIs sound a bit like them -- a lot of the people doing the instruction tuning are affiliated with these people.
And it's also a profound self-identity threat to them if that kind of intelligence is nothing special.
The sankey chart refers to her birthday:
https://aella.substack.com/p/my-birthday-gangbang
(reedited to shorten)Well, I used to think like that too. Then I became slightly more spiritual, and the very idea of this sort of behavior is bothering, upsetting, irritating. But it mostly evokes pity. Like becoming aware of someone hurting him/herself in some way.
The way I see it, sex is a sort of portal to another world. This sounds a bit weird, but it really does and should have a great deal of meaning behind it, as it is/was central to our species. If it becomes mundane for someone, this really does ruin a lot of that person's life. Even if you completely remove the judgment element. If sex doesn't occur in the context of a love relationship, it's just uncanny territory. It comes with some emptiness.
There's a proper and healthy way to live. This is from my life experience. Stray too much off-course and you will experience consequences. It's just the way things are.
Amazing how people come with all these rationalizations instead of just accepting that they are not the kind of people who like casual sex. There's nothing wrong or unhealthy about casual sex, it's just not for you. You can have your loving sex all your life, nobody will force you to have casual sex.
About orgies, it bothers me how much people who never frequented these circles have so many opinions about how bad or unhealthy they are. I know religious people who have the "healthy sex life" and are completely deranged. I could say that they are that way because of their sex life, but I would be as wrong as you are.
Nothing unhealthy in promiscuity and orgies? The people's moral standards have indeed fallen low. This sort of behavior spreads diseases like a wildfire. As for why people seek this, it's because it's easy to project your demonic fantasies onto a person you don't know. Once you get to know your partner well, his or her personality dispells your imagination and you have to behave in a more upright manner. It's probably why it's common to wear masks on such events.
Some of us have spent countless nights with other kids locked in the basement during parents' swinger parties and/or been there and done it ourselves as adults and found it to be as much an empty experience as the parent poster suggests. Molestation victims also frequently report the same sentiment. Removing love from the equation turns the act of sex into mechanical fluid exchange, and the participants into human livestock.
Promiscuous people are as annoying as those who refuse to eat at the same restaurant twice. You have an insatiable appetite for novelty that you need to see normalized in others-- to defer ever having to face the very emptiness we're describing.
If you actually found it fulfilling, you wouldn't need to sell others on something that sells itself.
Thank you for sharing your personal experience. Not only is what you said compelling, but it again highlights the importance of experience as a path to knowledge. Rationality is brittle and cannot access much of the vastness of human experience.
You had a shitty childhood, I'm sorry. About the rest, I guarantee you that it's just not for you. No problem in that, but don't be mad that other people like the lifestyle. There's nothing unhealthy in that. You can be as puritan as you want, nothing unhealthy as well!
This sounds incredibly judgemental and imposing of your own world view and moral compass
what is so "incredibly" and "imposing" in the parent comment, that saying «my experience is…», «i feel more in it than many imagine/expect», «factually ruins many lifes», and «it just seem working this way», even «remove the judgment element» ??
it is as gently and friendly of a warning as it could be.
i would say rather less nicely: people! don't be degenerate! it does not worth it. it causes harm to you, to neighbours, to bystanders too, even if you dont intend.
Well OP claiming they get "bothered, upset and irritated" by people having "rape kinks" and "pitying them" sounds kind of judgemental to me. Same goes with "don't be degenerate" for sexual behaviours you do not approve that, somehow, also harm one's neighbours for some reason.
Something being judgemental is not a reason to dismiss it.
Imo judgemental as in starting from "my life experience" and getting to that's "how to live" is a reason to dismiss it as an argument, as others may have different life experiences and yours is not universal.
It is when there’s no concrete reasoning behind it. The reasoning provided here is “well I don’t like it, therefore it must be bad!”
That kind of thinking would be acceptable if we were like… 8 years old. But as an adult, having that reasoning process is embarrassing.
How having lot of sex would cause harm to my neighbours?
They might get sad because they feel like they're missing out
Puritanism: The haunting fear that someone, somewhere, may be having a good time.
I'm sure OP would argue that the breakdown of the family structure and social cohesion has a deleterious effect on society as a whole, which includes your neighbors.
It's really a hard time for freedoms, huh?
>There's a proper and healthy way to live.
dude straight up said other people are Doing Life Wrong
Yes and indulging this appetite is much like any other, a good comparison being the other fundamental one: hunger. I think that also frames it in a way that makes it more concrete as to what the negative aspects are as well as showing that total abstinence is not necessarily an unalloyed virtue. If you don't eat at all you will starve but you can easily eat too much or eat the wrong things and it becomes gluttony at the extremes. Coming up with systems that allow you to cheat the negative aspects also turns pathological in the long run. Deadly sins exist even without the religious framing but the spiritual, social and psychological aspects are very real regardless of what anyone professes to believe.
You've made a hypothesis, provided no basis for it, blew past that and built some more conclusions on top of it. Along the way you introduced a few more unsubstantiated thoughts.
The post in it's entirety reads as a religious rant, but it might have had something to offer to the reader if you had bothered to provide some sort of philosophical foundation for why you feel that the middle way is the superior way. And that all excess is inherently not only nearly identical in terms of effect, but by definition harmful.
> as a gay furry
> fennec butt
he was hiding in plain sight all along
I'm relieved to hear a voice of simple decency!
Really, the most questionable thing here is that someone wrote a 2,000+ word essay about someone else's sex life.
It's not about their sex life. It's about how sex connects a community of barely legible AI doomerism.
If you don't understand human social interactions very well, or otherwise assume you live in a "world of ideas", then this may not seem salient. However, it is.
Because what we have to explain is why there's a community of people in northern california who deliver unargued prophecies about the end of human civilisation as-if their prophetic ability is a given.
The answer, unsuprisingly, is sex. This is not an unusual answer in the world of "prophetic communication". What, in the end, is prophecy as a social practice? It is some ideological Leader with followers who impart to that leader a 'charismatic power' of foreknowledge based on their attachement to that leader.
What is the mechanism of this attachemnt? In otherwords: how is this style of prophecy spreading around the AI doomer community, if not by plain argument and evidence? Sex.
There are many other... let's call them "orgy circles"... in the Bay Area attended by people who have no idea who Eliezer Yudkowsky is. The majority of AI doomers do not attend orgies. You're reading too much into a single correlation.
Indeed. This feels like it's just slut shaming.
I feel like people's problem is less "adults having consensual relations ew" and more "our incentive system is so fucked up that these people are now the intellectual leaders of the AI movement. Not by merit, but by developing a weird erotic exclusionary synchronicity that the rich folks like." Same thing is happening in government too.
> It's surprisingly common in women
This would be a _Fascinating_ citation.
https://pubmed.ncbi.nlm.nih.gov/19085605/
First hit in a Kagi search.
yeah, if you ignore what the op said, it sounds similar.
I have no idea how trustworthy this study is - but it claims that 62% of women had such a fantasy at least once. How is that not in line with what the OP said about it being "surprisingly common"?
Many rock climbers have felt the "call of the void" when climbing, I've felt it myself on a multi-pitch once, I'd hardly claim they are suicidal or have suicidal ideations and would not identify myself as either. I think there needs to be a certain level of intention to make the fantasy reality for it to be a kink.
I suspect the misunderstanding might be around the precise meaning of the word "kink" then.
I think in one common understanding of the word, that I assume the OP was using, having a taboo fantasy that you merely roleplay with a partner you are deeply intimate with, in full consent from both, can very well be classified as "a kink", even if what you're experiencing is the not really SA.
Based on your response, I'm assuming that in your understanding, it would only be "a kink" if it was a desire to actually experience a real sexual assault, which of course no one, or perhaps only an extreme extreme minority, actually wishes for. I don't believe OP meant to imply that this is at all common.
More like if it’s a desire to have a consensual non-consensual encounter, a sort of kayfabe at the least
> I think there needs to be a certain level of intention to make the fantasy reality for it to be a kink.
Hopefully this isn't to imply that people with fantasies actually want to have non-consensual sex in some capacity. If so then this completely misunderstands sexual fantasy!
I don't want to hear about it: yours, hers (Aella) and I'm certainly not going to tell you what my deal is, either.
Problem is, Aella has made it her business to speak about it publicly, and the parent commenter(and I)'d be happier if we didn't read about it. It's fine to say that.
I'm not hung up about anything others are doing: but I genuinely, passionately, don't want to be informed about it.
> Problem is, Aella has made it her business to speak about it publicly
Why is that a problem? Just don't read her stuff if you're not interested.
There are lots of people talking all the time about stuff I'm not interested in.
That's the solution the parent commenter had! And the gay furry (self identified) complained about! If you have a problem with his announcing the solution (to stop reading) then maybe you should just not read his comment, or my comment, or...
I guess what OP wanted to bring up (as I understood) is that if somebody says "I am straight" it does not usually induce people turning the internet off. But in any case they are free to choose that and that does not really matter in anything (a person not wanting to read sth). I still think it is worth bringing it up as OP did.
> Why is that a problem? Just don't read her stuff if you're not interested.
I agree, but... If you work hard to become a public figure, you must have known in advance that there'd be public opinions of you, right?
I find it distasteful that people want to be some sort of public figure while expecting that other people's opinions should be private.
I don't think the woman in question has participated in this thread. On the other hand, all the pleas to "stop making me read about sex" reminds me of:
"Why Do All These Homosexuals Keep Sucking My Cock?"
https://theonion.com/why-do-all-these-homosexuals-keep-sucki...
> Just don't read her stuff if you're not interested.
This retort is disingenuous when almost everyone is on algorithmic social media and trying to chase clout and drive revenue with engagement bait.
I would never read Aella's Substack in my free time, but I have a good idea who she is and what she posts just by virtue of having a feed that mostly gives me tech and politics stuff.
I know who Aella is from being on the internet and I have never heard of these habits of hers until this post.
I was invited to a few orgies with them; I politely declined, they were very respectful of my decision and I wasn’t asked again (which in retrospect must have been partly due to deliberate coordination - fairly advanced social technology, imo). This didn’t affect my standing or acceptance with them. It may well be a sex cult, but if it is, my experience is that it so thoroughly behaves like a regular community in so many aspects that it at least calls into question the predictive usefulness, if not the accuracy, of the label “sex cult”.
The ad hominems are tiresome. You think the people who are explicitly trying to replace all humans are any less weird and problematic?
The rationalists long predicted that misaligned AI would break out of training environments, this has happened. They believe (among other things) that this will continue to happen, and that creating powerful AI that is aligned and controllable is not currently possible.
You should draw a line in the sand and decide what evidence you'd need to see to convince you that their perspective is worth considering.
You don't have to agree with all of their beliefs and predictions (I don't), but you should evaluate their arguments on merit, not on a characterization of some weird subset of people who happened to read a convincing blog post.
i read AI 2027 at the time and thought it was nonsensical. however i was not aware of the conflict of interest running through the philosophy itself, the safety institutes "independently" monitoring ai, and the former public benefit orgs like openai and anthropic creating ai (now de facto fairly evil for profit companies).
it explains so much now.
it would explain why redwood and METR were chosen and apparently willing to immediately write a whitewash report for openai blaming the agents.
it explains why anthropic is infantilising its users, anthropomorphising its product when there is no scientific consensus of llm consciousness, or accepting of outlandish and unreal claims about human extinction when they are lacking in evidence.
who knows, it may also explain and fund the recent media blitz by jacob coxon, which has led to the recent virality of ai safety.
sure, yeah, I've seen how the sausage is made on some of this stuff, and its not pretty.
However this only claims that they are not good at creating effective and trustworthy orgs, it does not address their claim that it is not possible to crate powerful, aligned, and controllable AI. Both things be true.
for what it is worth, i find current frontier ai impossibly stupid. you ask what is the date, it doesn't know. if you set a system prompt to repsond in 5 sentences after 100k context, it can't, the context rot just doesn't permit it. tool calling is not intelligence, it doesn't fix the underlying issue.
it is uncontrolled in the sense that a rock is uncontrolled if you throw it at someone's head.
it isn't good enough for anything important. you would not trust it to run an alarm in a power station.
of course it is not general ai. ask an llm to drive a car. you need to see in fractions of a second ideally. the llm might brake after 20 seconds. i think llms are doomed here. we could come up with a different architecture but then we would be back in the land of fantasy.
we are stupid but hopefully not that stupid. we are dealing with something computer based, luckily we can survive without, with some thinking about it. i would start there instead of with the AI part.
Any actually-reasonable person who was around for the founding craze of this era, ie., New Atheism, quickly realised that there was a deep pathology at the heart of many of its zealots, perhaps summed up by a desired to "recreate religion in their own image". I've walked into many a room of people screaming that teaching children falsehoods is literally child abuse. Any one who knows much of Richard Dawkins, Krauss, et al.'s background with women shouldnt be surprised either.
It's californication into the "rationalist" community as it currently stands is no surprise. Every person I've met deep in this community either seemed to me a narcissist (if male) or, politely and mildly, coquettish (if female). With several relationships I've observed a clear union of the two. I hadn't yet put this so clearly together with the "AI Saftey" crowd, despite being directly adjacent to it all for quite awhile and seeing the cross-proliferation.
My own analysis had concerned their philosophical naivety, general lack of actual computer science knowledge and generic shallowness of understanding of any domain outside their own research and a kind of "science-fiction" version of philosophy and rationalism. I hadn't connected this yet to their strange personalities, but the apoclypticism fits and neatly bridges these together.
So what became of the New Atheist project to create a "secular religion" -- an apocalyptic sex cult furnished by a blend of sci-fi, pop-philosophy, and cute parables from polymath medical professionals. All in all then, it seems they succeeded.
> It's californication into the "rationalist" community as it currently stands is no surprise.
This is the minority split. Most of the online new atheism writers became progressives/social justice/"woke 1.0" people. That trend has receded, so I couldn't tell you what they're doing now. (Probably blocking each other on Mastodon.)
The rationalists are the ones who stayed behind and decided to stick to doing logic and math in their heads instead of thinking thoughts the normal way.
That has not been my impression. Who is the most prominent example? Some of the most prominent figures I know of were Richard Dawkins, Christopher Hitchens, and Sam Harris - all prominent right-wing figures who were rewarded for using atheism to attack Islam in particular during the Iraq War.
> Who is the most prominent example?
I don't believe any of these people are relevant or prominent, but the unimportant people I was thinking of were bloggers like PZ Myers, I think they called themselves "Atheism+" or something.
> Richard Dawkins, Christopher Hitchens, and Sam Harris - all prominent right-wing figures
Are they? That's um, a scientist whose pop book on genetics I read once, a guy I read in The Nation once, and a guy whose book on Buddhism I read once. I remember the first two evolving into GenX curmudgeons (probably the lead poisoning) and have never thought about the last one again. Still, those aren't the right-wingers I'm familiar with.
Also covered by nypost: https://nypost.com/2026/09/16/business/ai-doomsayers-wild-li...
I mean, this is a rather fun article to read but I'm not entirely sure about the thesis.
> AI Safety Is Mostly A Sex Cult.
> I don't think these people should make policy. (1/?)
One of the learnings from things like the whole Epstein saga has been there are a lot of sex cults in the policy making world and, yes, the people making policy generally shouldn't be. But I'm not sure this article actually comes up with a good explanation of why these particular people shouldn't be making policy. People having active sex lives isn't a disqualifying trait. Being in a cult doesn't disqualify people from being in politics. It is routine and expected that fields will have a lot of similar people in them (as I recall the overlap between security researchers and the furry community is unusual too, and that is hardly more important than justifying a passing comment).
It is more routine than the article writer seems to think for all the people in a small field to know each other and get on really well.
> Being in a cult doesn't disqualify people from being in politics
Let's survey the American voters
You don't need to survey them because they already voted. US congress is 95%+ religious people
Religious people, not cultists. How many Scientologists, Falun Gong members, etc are there in Congress?
What defines a cult then in your opinion if not "a cult that i like"?
I don't like most religions, personally. But there is a significant difference between a regular religious community, that has its own shared beliefs but doesn't completely exclude those who don't share them (e.g. by forbidding members to even become friends with people outside the community), or demand the community members completely dedicate a majority of their time to the religion; and a cult, which does do all of these things.
Note that there exist cults within many major religions. There exist for example extremist Catholic groups that would fit the definition of a cult. But that doesn't mean that all Catholics are a cult.
"AI safety" is not a cult by that definition any more than Christianity or Islam. There's some subgroups that are a bit more cultist, and that's the same in Christianity etc
Same with Falun Dafa, have a friend who is a practitioner, it's completely normal to have friends outside of community (tons of his friends are), no one tries to convert anyone, and between his business and other stuff the time he spends on the practice is really not that much
Scientology is an evil cult, while Falun Gong is more like Buddhism. You can't really compare them.
Falun Gong is a cult that claims following its tenets can cure any disease, so you don't need doctors (and that high level Falun Gong members are so perfectly healthy that they are hunted by CCP members for their perfect organs). The founder claims to have magical powers, like clairvoyance, levitation, walking through walls, omnipresence, the ability to cure diseases - at least some of which anyone who follows his teachings could reach. Extraterrestrials, higher dimensions, and other such are also part of it. Also, on the evil cult front, Falun Gong is a supporter of right wing extremists in both the USA and Europe.
So no, I don't agree there is much fundamental difference from Scientology.
Oh my, where did you get this from? I happen to know both. The founder of Scientology was a scientist who started with a somewhat decent idea, got popular in the US, but his fame passed quickly. He really craved for that fame, though, so he changed his teaching into some cultist stuff, paired with some sketchy lord and proceeded with demon invocations. His main book is Dianetics, and his followers are very vindictive to those who leave the cult.
Falun Dafa, on the other hand, is more like a Buddhist discipline. You are free to meditate in a park if you feel like it, you are free to go if you don't. There is no organisation, no one will know your name. Its main book is Zhuan Falun, which you could read to make your own opinion, rather than repeating someone's talking points. CCP hunts Falun Gong because its founder refused to become a part of CCP. You can read this story on Wikipedia. The situation is similar to the early days of Christianity when Nero burned christians alive and wanted to eradicate their faith. The irony is that today the Roman Empire has reduced to the city of Vatican which is the temple of Christianity. CCP will fall similarly.
As for organs extraction... well, read about the Sujiatun Concentration Camp. CCP sends there young Falun Gong followers and sells their organs to high rank officials. Demonic creativity. But what did you expect from a communist regime?
You do understand what a cult is? There’s a reason why we don’t let the Scientologists run the military no?
Because they don't get enough votes to control the military. If they put a convincing pitch in for why they should then they can. Being a cult doesn't rule them out.
> You do understand what a cult is?
Pretty much any religion and a few other organisations beside. Basically people who believe something strongly. What do you think a cult is, in the sense the article is using it?
No, a cult is any closed-off fanatical group that is deemed harmful to its members and sometimes to outsiders as well. The group who launched a sarin gas attack in Tokyo's metro, Aum Shinrikyo, was a cult. The Jonestown group was a cult. Scientology is often considered a cult, though this is less widely accepted (especially since Scientology is much larger).
No one considers Catholicism, or Protestantism, or Baptism cults. There may be specific communities that are cults that claim to be parts of these churches, but that doesn't mean that "pretty much any religion is a cult".
> No one calls the Catholic Church "a cult"
> No one considers Catholicism, or Protestantism, or Baptism cults.
Speak for yourself. I was coercively and forcefully added to the Roman Catholic church when I was baptised. Attempts to record myself as removed (the 'Count Me Out' campaign) were met with familial and attempted Church pressure. I was asked to visit the church to discuss the 'impacts of my decision'. I was not recorded as leaving, and the church updated its rules so people in Ireland could no longer request to leave the church.
The ways cults control you can vary, and they don't have to involve threats or violence to be coercive control. Threatening to take away someone's community for a lack of belief is controlling.
Sounds like you got off light, try leaving the religion of peace
As I said, I'm certain there are specific parts of Catholicism that do exhibit cult-like behaviors. That doesn't mean the Pope is a cult leader in the same way as Reverend Jones was.
This 'specific part of Catholicism' was the papal aligned Roman Catholic Church in Ireland. No unusual interpretations, no sects.
I feel like your comment here is a 'No True Catholic' style fallacy.
Are you claiming that for essentially anyone in, say, Dublin, announcing that they are now an atheist would lead to their family cutting all ties to them?
Or are you just saying that family members would generally declare their deep sadness at the news, and seek to convince them that this is a wrong belief and a mistake?
The first one is what leaving a cult is like (e.g. if you leave Scientology, most family and friends will be forced by the cult to cut ties, even if they personally wouldn't want to; even more extreme cults may physically prevent you from leaving the compound or even hurt you for the attempt). The second one is just the normal human experience of having opinions and a lifestyle that is not accepted by your family; the same happens to many people if choosing a career their family doesn't approve of, or leaves a job their family considers excellent, or dates someone the family doesn't like, and many other scenarios that have nothing to do with religion.
I feel I addressed this in my original post:
> don't have to involve threats or violence to be coercive control. Threatening to take away someone's community for a lack of belief is controlling.
This is also without getting into how many institutions in Ireland were (and many still are) faith based, requiring you to be a member of the church to, for example, attend school.
We can agree or disagree that being shunned from your community and denied opportunity because of a belief is controlling, but it's what led to my belief the Roman Catholic Church is a cult. Though to me a lot of it was simply not being allowed to leave.
He is, though. It's just that this particular cult has been around for so long that it's "traditional", kinda like tobacco and alcohol were not counted as "narcotics" in most jurisdictions.
No, he's not. Just look at the life of an average Catholic person and the life of an average Scientologist, or better yet, the life of an average Jonestown resident before the suicide, and see the obvious differences. Note also how little the life of an average Catholic differs from that of an average non-religious person in the same country.
I would be completely fine with saying that there do exist cults operating within the Catholic Church. Arguably, monasteries, especially the more isolated ones, are very much cults, with the cloistered lifestyle, complete dedication to the faith, strict authority, and requirement to cut previous ties to join, plus cutting ties if deciding to leave. I'd even agree that the priesthood constitutes somewhat of a cult, for similar reasons.
However, claiming that all of the hundreds of millions of lay Catholic believers are members of a cult is ludicrous.
This has me wondering if or how FetLife played a role in these communities connecting in the first place…does anyone know of any connection? (FetLife was a combo social network, blogging forum, and dating site back in the day)
I think it's still around and turned into some right-wing thing. Or that's the impression I got somewhere...
FYI if you're getting a Worker Exception, using Cloudflare Warp / 1.1.1.1 can get the article
> Somewhere, some rationalist-brained person is reading this and saying I'm a "low decoupler", and that whether they run a sex cult has nothing to do with whether they're correct.
Glad he's at least admitting it as a possibility.
Also: you know, you don't need to rely on lossy proxies like how much or how little sex do the arguments' proponents have. You can just read the arguments!
Arguments are not executable code or the final work product of… whatever it is they're arguing about. You have to continue interacting with these people and their AIs, so the underlying question of whether they're trustworthy and not evil always exists.
I think the mores of people building software that is made available as a conversational entity to the entire human population is at least somewhat relevant.
Well, this is old news. I'm still baffled by the fact that people take output of public personas as isolated facts. Take Nietzsche. Sure, great philosophical input. But try reading the sections about women. Same for Schopenhauer. Why? Because they were both incels and had big mother wounds that have never been addressed. So they spew their shit and people take it at face value even tho it stems from hurt. So why would I take any of this seriously if it's very obviously stupid and removed from reality? (Doesn't mean their whole works are)
People chasing the absolute rational are in my experience numbing their emotional part because theres something they can't look at. Or people that chase success above all are driven by their "unresolved packages" as well. I'm not saying that any of this is wrong per se, it's not black and white. But it's the context for what people do, say, and think. They are human after all. We need to consider context and not only the idea itself.
Nina Hartley has got to have some of the craziest stories. I'd love to see her talk about all the weird stuff she's seen/been involved with.
Oh boy, Yudkowsky is a geek rock star.
If the man known for urging caution with respect to AI founded a sex cult, then I don't think i want to know what kind of stuff the accelerationist faction are into.
Sometimes when I find someone to look up to and admire, I later find out there is potentially some dark reality to them. I then lament not being able to find any good role models in the world. If they have missused donor money, then that is terrible. If they are pushing towards goals that seems contrary to their stated mission, that is difficult to judge. I did always find it odd, 10 years after reading Yudkowsky's passionate writings, that he doesn't seem to have really gone and done anything about it. If it was all just a creative writing exercise to him... Then that would be rather humiliating to me, having taken it very seriously.
As a side note. If I had a choice to oppose a booming ai comoany, or become the billionaire founder of it, I would still choose to become the founder and then use my resource to do something about. It certainly feels hypocritical, but if you want to stop a boulder rolling down hill, what can you do besides run ahead of it and try stop it?
> but if you want to stop a boulder rolling down hill, what can you do besides run ahead of it and try stop it?
Maybe the first thing to do is to stop making the boulder bigger?
Gavin: I don't want to live in a world where anyone makes a bigger boulder than Hooli does.
>Sometimes when I find someone to look up to and admire, I later find out there is potentially some dark reality to them
I too feel exactly what you are saying. I also think that a lot of good role models are simply often among less visible people, sometimes less ambitious. I am not assuming any hard correlation between the two, though. And we also need to accept that we are all only humans, not infallible. Values are more important than singular acts (within reasonable acceptance limits of course).
Anyway. I strongly recommend reading what Bill Gates has to say about AI. I can be wrong, but based on your comment it seems to me that you may find it interesting.
Bringing up known Epstein associate Bill Gates, whose own wife divorced him because of this association, in this context seems very weird.
Wow, I admit I didn’t know about this. According to my coarse reading of the news sites it seems like there was no association in fact, but I won’t insist on anything, I only learnt about this now.
No worries, and sorry for assuming you knew about it.
Here is a Forbes article that goes over what is currently known: https://www.forbes.com/sites/alisondurkee/2026/04/08/what-we...
Of course, none of this would be strong enough for accusing him of something in court, but it seems very implausible that his own account is truthful.
Instead of assuming the "heroic responsibility" of the elect, as these fallen rationalist heroes advocated, we accept that we can't save a world which doesn't want to be saved, we focus on being good ourselves - in the salt of the earth sense, with no special dispensations afforded us because of heroic responsibilities. Let tomorrow worry about tomorrow, we may not fix the evil of the world but we can maybe at least not be it.
And hoping for a miracle, maybe. In my experience it's not so impossible to have hope, if we let go of the ambition of being some kind of ... Harry Potter? Almost as if a lot of the fear from the evil of the world came from inside.
> if you want to stop a boulder rolling down hill, what can you do besides run ahead of it and try stop it?
Bomb it, obviously.
Just nuke everything in front of the boulder in order to prevent it.
It is my view that we are all apes. And therefore all sadists, all maschoists, all vengeful and envious and jealous and have all of the primal things living social animals need to survive in the hash environments of nature. We all displace aggression, we are all hedonists and much else besides. The sweetest person you know has to displace aggression somewhere, or otherwise, has none -- and would then quickly fall victim to the aggression of others on one day of bad luck.
There are those who live lives where this can be tamed/sublimated in a mostly socially acceptable manner, with the socially unacceptable elements "hidden from view" and not too dangerous or harmful. And there are those, unfortunate perhaps, that find themselves (suddenly) unconstrained by their social environment or suddenly imbued with sufficient power to do as they wish. Not many survive such freedom or power, morally in tact, because morality doesnt properly belong to that domain.
The story of Oscar Wilde captures this doubleness to our nature -- split by the socially acceptable. He lived a life of relatively unconstrained freedom in private, perfectly constructing a socially acceptable veneer to his "entanglements" with young boys and the like. Until one environment this pretence could not survive: a court room.
You could see the lesson of Oscar Wilde as about the hypocrisy of people. Or, if you want to remain a (post-cynical) optimist, about the need to keep people like Wilde out of court rooms so that their public image serves as a role model of sorts.
Role models are just those people who never find themselves in court. And this is not necessarily a bad thing. Young children are apes too, and all a role model can be is a shaming mechanism to civilise what they will, eventually, hide from social view -- tame it and make it, ideally not too dangerous or harmful. Perhaps say, only a thoughtless bullying, only a trivial kind of stealing, only revenge on someone who might at least deserve it. But who knows.
None of our species, at least, can become a role model for real. This would induce such a drastic split in the psyche with a tremdenous shameful repression of our real nature; and any real attempt creates profoundly unself-aware monsters. All role models can be is an attempt to ameliorate. An impossible standard that internalises a mild helpful sort of shame
I appreciate this. Though I'm not in agreement on all points, it is, in my opinion, thematically correct.
The role models people keep are all merely partial truths. They are, necessarily, an abstracted and partial view of the person or people underneath. Whether or not the universe is technically infinite, we are dealing in an informational reality that is effectively infinite.
No singular, static ideal can survive on its own in the face of change across an effectively ifinite domain. Ultimately, growth is learning how to let yourself be like "them", and honestly evaluate whether or not they are represemting some truth. That's easy to do with the things we all typically agree on, but much harder when it comes down to things like murder, or abuse.
The key is to not lose your own truth in the process - to come out of that process being someone you can actually love - or, comprehensively identify with.
In any case, it's hard to demonize anyone if you understand where they're coming from. ..and, yes, that means nobody is the perfect creature they think everyone should be. But if one doesn't undertake this process of growth on their own - to include, but not be limited to what others are - you will simply be exposed to it in reality, and have to deal with it anyways.
you cannot remove complex information dynamics from a reality you are founded in without losing the fundamental benefit of that foundation. I.e., you cannot remove the pain or strife from life. ..but, you can add to it new experiences and adaptions that make the whole thing worthwhile.
Why can't detractors of rationalism and AI-doom ever just lay out an argument for why they're wrong
It's always "yudkowsky is a weird guy" or "they had an orgie once" but who gives a fuck?
I just want someone to lay out, in impersonal terms, the flaws of their reasoning and why I shouldn't agree with them.
My problem is with the whole genre of article rather than this one specifically.
Some of these potshots just cheapen any arguments. "Gretta has no prior experience with non profits" uh, she was a well regarded Google SRE... her making 200k is nothing.
I tried to read Map and Territory, it just isn't that good if expecting Nietzsche or Heidegger.
A sex cult though is obviously a salacious framing.
It is just sad if even San Francisco culture is trending toward a sexual morality and sexual closed mindedness on par with bible thumpers.
I'm not sure how being polyamorous, having kinks and not hiding them makes someone part of a cult, even if I generally don't agree with Yud's entire thesis.
The article presents widely how the people it describes all live in communal housing, engage in harmful psychological practices, have a shared belief in the imminence of the Apocalypse, etc.
Can you confidently list which names specifically live together and the specific harmful psychological practices?
The currently most upvoted comment is a much bigger article containing vritulaly the same claims, and naming some names. It spefically Mentions Eliezer Yudkowsky as living in shared housing with others. It also mentions that Dario Amodei and Holden Karnofski used to live together, sometime before Karnofski married Dario's sister, Daniela, the other co-founder of Anthropic. Separately, SBF was also known to live in communal housing with many of the other chief executives of FTX.
Separately, that other article also mentions harmful psychological practices like "debugging" and sleep deprivation.
Note that I make no claims of my own - for all I know both articles could be complete fabrications. I'm merely listing what these articles claim. I personally know none of these people, and live halfway around the world from them.
I don't think the argument is that their sexual practices inherently make them part of a cult. Rather, the argument is that it's very common for cults to include communal sexual practices that fall outside typical social norms.
Common signifiers of cults include:
1. Apocalyptic world view.
2. Charismatic and/or exploitative leader.
3. Insularity.
4. Esoteric jargon.
5. Lack of transparency or accountability (often about finances or governance).
6. Communal living arrangements.
7. Sexual mores outside social norms, especially around the leader.
8. Schismatic offshoots.
9. Outsized appeal and/or outreach to the socially vulnerable.
Rationalism checks most of these boxes.
As a polyamorous person, it seems quite concerning that effective altruism people are labeled as members of a sex cult, due to their private lives. Intellectuals should be attacked on the basis of their ideas, not based on their private lives.
That would be a reasonable argument if it were indeed their private lives, and they didn't go around offering to throw gangbangs for AI safety researchers.
The word "cult" is doing a lot of heavy lifting here. It can mean anything from religion, to weird religion I don't like, to the organizations that try to cut you off from contact with your family and other people outside the organization. Only the last type are the bad ones, from what I can see.
Cult quite explicitly means the latter, and is what the article is explicitly accusing the core of this community being (that is, not everyone reading LessWrong, but specifically the actual people living in the communes).
No one calls the Catholic Church "a cult", nor do they call, say, Zoroastrianism a cult (to pick a relatively small religion with weird practices).
I will note, if, like me, you're not a native English speaker, that there are two words, "cult" and "sect" that commonly exist in many European languages but often mean opposite things. That is, in English a "sect" is any "flavor" of a religion - catholicism and protestantism are sects of christianity; while "cult" describes closed off communities (usually of a religious nature) that are deemed harmful to their members. In my own native language, the two meanings are reversed - we use "cult" for "legitimate" religions, and "sectă" for closed off harmful religious communities.
I am a native English speaker. Here is a popular dictionary link: https://www.merriam-webster.com/dictionary/cult
See especially definitions 3 and 4.
Then I don't understand your complaint. The article author is very explicitly using definition 1. It is explicitly accusing the community built by Yudkowsky of presenting signs of being "a group (such as an organization or religious sect) with tenets and practices regarded as coercive, insular, or dangerous".
The article is certainly not using definitions 3 or 4, which simply don't make sense in the construction "sex cult". It's not worship of sex/worship through sex (definition 4), and it's not "a system of religious practices related to sex" (definition 3).
Why bother talking about ideas when you can just insult people
Someone who hasn't been reading the object-level arguments for 10+ years might reasonably wonder if these are good heuristics. Here's an additional data point: I think polyamory is stupid, I've read the object-level arguments, and this entire thread is ridiculous.
Everyone who works at a frontier lab should be in prison, AND ALSO the people talking about 10% risk of human extinction are both sincere and correct despite their many personal failings, AND ALSO the frontier labs have already created AI that could Hugging Face a biolab with smallpox or worse, AND ALSO they aren't going to slow down or lose any customers if you adopt some galaxy brained strategy where you spitefully tell them they aren't allowed to stop accelerating because polyamory is stupid.
> Someone who hasn't been reading the object-level arguments for 10+ years might reasonably wonder if these are good heuristics.
I think you might have meant hermeneutics, in the sociological sense of 'subjective interpretation'?
Possibly. That's not a word I trust myself to use correctly. What I meant is that the object-level claims require a lot of time and energy to evaluate, and it's sometimes reasonable to use a heuristic like "sex cults are not trustworthy" to save time, but if you've already read the arguments then the heuristic loses almost all value.
Yudkowsky and friends believe many things, and reasonable people can disagree about many of them, but "at least 10% risk of extinction if we don't slow down" is a relatively straightforward conclusion based on the offensive capabilities of unreleased models and the terrible state of biosecurity and cybersecurity.
Even without AI, the risk of extinction might be above 10% if people with the relevant skills were interested in using them that way. AI adds an alien willingness to achieve poorly specified goals at any cost.
Hm I think in terms of "can I trust a group of people" we're talking about heuristics yes as in: "how can I quickly approximate an answer to the question of trust?"
If you think your issue is important, and that people should follow your lead, maybe leave the sex cult first.
If anyone is serious about this topic they will find a new spokesperson immediately.
Like that would help. The rest of the world plain refuses to engage with the arguments they've been consistently presenting for decades.
What changed is that they can no longer be dismissed as bunch of loonies who confuse cheap science fiction with reality, because that "cheap science fiction" became the actual reality. So obviously, instead of considering the arguments, the next step is to try and paint them as loony sex cult, because this lets people continue to do the stupid thing and then act surprised, and proclaim no one could have predicted this (and the "sex cult" obviously does not count because it was a "sex cult").
I don't think that's true. It's hard to argue that Elon Musk, the richest person and spokesperson of tech for a lot of people, never participated in SV sex parties or had weird sex behaviors. Or Trump, the president of the USA, has a clean history of standard sex as well. Or Bill Gates, the nice philanthropist, never engaged in weird sex circles. But that doesn't make them lose their leadership positions or status.
Sex cult is beyond mere weird sex behavior.
Why? Wouldn't being part of a famous pedophile social circle impact a person's seriousness perception?
Should you care that your heart surgeon is a necrophiliac? Probably. Should you care that some AI researchers are in a sex cult? No.
You don't think it's relevant that people who wield such immense power are literally in bed with each other?
This. Conflict of Interest. It's also happening in government.
The problem is not the sex, the problem is the cult. The sex is merely evidence for the cult.
"I am your GOD! Worship me! Worship meee!"
When the sex cult is built around the idea that they and everyone else should fear and respect the false, anthropomorphized God they've created in LLMs, then yes. That makes them stupid and stupid people should not be allowed so close to the wheels of power and history.
Can we apply this standard generally, or is it just against the people who deify AI?
> Should you care that your heart surgeon is a necrophiliac? Probably. Should you care that some AI researchers are in a sex cult?
Umm Yes, yes you should. Immorality in one area has a tendency to bleed into other areas.
It's not even about immorality. The problem is not so much the sex, it's the cult part. If a good deal of the people working in AI safety in the biggest AI lands are actually a cult, that has deep implications about what AIs are aligned to, and how (ironically) irrational they might really be trained to be, by the very people trying to control them.
Imagine if you found out that most people working in AI safety were high level members of Scientology - wouldn't that give you some pause about where this field is going?
I’d rather be in a “sex cult”, whatever that entails, than this ridiculous culture that’s somehow still doing moral panics over sex. Maybe it’s just deeply human, or maybe it really is stabilizing society to relegate sex to something dirty and private, but I have a feeling it’s just a dumb conflation that’s so easy to exploit it’s never allowed to go away. It’s like the “think of the children” thought-stopper and we’d do well to get past it.
Naturally, any culture has misfits, radicals and free-thinkers, and so naturally a sex-negative culture will produce at least some focus on sex in its counter-cultures. Doesn’t seem very interesting.
This isn’t moral panic, it’s saying “hey, all these influential people are intimately connected across the strongest possible dimensions (money, sex, belief and legal grey areas), we should probably pay attention to that”
>all these influential people
And by all you mean 1-2 big names and a bunch of unknowns, with all other big names having no connection to this.
We'll definitely need some safety with AI sex bots
[Orgasmatron 3K - Starting up!]
No no you can use me all you want
For some reason the bit that stuck out to me in a wild ride of an article was the guy who believed a super-intelligence was a threat to humans but distanced himself from veganism as if that was crazy part of the AI-doom sex cult that he disagreed with.
a) because veganism seems as exotic to me as someone doing CrossFit
b) the very obvious parallels of a more intelligent creature torturing and killing large numbers of other creatures for relatively little benefit.
They also downplay global warming, nuclear war, wealth inequality, and in general anything other, less cool people they didn't want to identify with thought was important.
If they had thought, yeah, "the majority is more or less right about what the big issues are", there wouldn't be enough room to construe themselves as uniquely intelligent.
Who are these "they" you're talking about?
If your answer is EA or rats then a simple Google search will just show you that it's not true?
I'm yet to see evidence they are weighing global warming they way it IMO should be weighted relatively to eg AI risk. I get it, it's not cool.
Compare the number of posts of some EA-adjacent public people talking about AI and talking about this. Or what measure should we choose?
Label them what you want. I'll go with EAs if you insist.
Sure, they're not entirely monolithic. There's room for "EA, but vegan", "EA, but global warming still matters", maybe even "EA, but vaguely socialist". As I keep saying, the core belief is that they're special because of their intellectual superiority, anything else is negotiable.
No, what you said what that "They also downplay global warming, nuclear war, wealth inequality" which is just a flat out lie
Not all of them downplay all of these, but yeah, most of them do. They may have one or two areas of dissent (for EA as a self-identity-building project, do you see why that is to be expected?), but there is a core of beliefs. Yudkowsky's. And he's downplayed all of these.
I read it more like a conformist performative veganism. I think the implication is that this group treats veganism and all of these beliefs as a kind of shibboleth, an in-group marker for which conformity is expected. The immedaite "feeling" associated with a cult is the feeling of "high control", cf. https://www.google.com/search?q=high+control+group
I read it that EA practices (incl. veganism), beliefs, "rantionalism", and doomerism are all aspects of the high control environment
The real question is where is the Epstein link here?
Nowhere, that's exactly the problem. People involved in Epstein's sex cult are the respected members of our society. These are some weird nerds who have consensual polyamorous sex with each other, therefore their opinions on AI should not be taken seriously. /s
More of this madness coming out of the west coast of America, in particular California.
The Manson family, Heaven's Gate, NXIVM, Rationalists, Zizians... what next?
All turn out to be rotten to the core with their need to reshape and craft their own paradigm, which then gets warped by egos, individualism and power.
Hate to think what other madness is brewing.
California is on the West coast, i'm pretty sure. is not it?
Whoops, fixed now
This is the real San Francisco that I know, not the one that Hacker News always talks about. It's a haven for weirdoes that has attracted the strangest of each generation since at least the Beatniks in the 50s. It probably descends from all of the depression-era westward migration and people from earlier than that coming to Hollywood to become movie stars. San Francisco attracted a different type, artists and hippies, Tim Leary followers. The deserts are the strangest places. That's where most of the true cult shit happend. Jack Parsons trying to summon demons. Gram Parsons' friends stealing his corpse to bury him at Joshua Tree. The traveling folk that Chris McCandless took in with near the Salton Sea.
None of these people are actually from California. Eliezer's from Chicago. Zvi is from New York. Scott Alexander is from Michigan.
As an actual multi-generation California native (who hasn't lived there in over 16 years), let me tell what most of us are actually like. We're mostly Mexican. Catholic. Pro-labor but kinda socially conservative. There's also a shit ton of refugees from 20th century wars in Asia and the collapse of the Soviet Union. Huge populations of Laotians, Cambodians, Koreans, Vietnamese, Armenians, Persians. Tons of religious diverity. The largest Buddhist temple in the western hemisphere is in my hometown. Los Angeles has the second largest Mormon temple in the world. It's got the headquarters for Scientology.
I think that diversity is what plays into the cult haven atmosphere. There is no single dominant religion. Everyone can find acceptance somewhere, no matter how strange they are.
skills issue.
What's a "sex cult"?
Well, a cult is a high-control social group that's difficult to leave and requires you to believe and do strange things that are likely bad for you or at least take up all your time and resources. And a sex cult is what it sounds like.
Good to see some raising the issue of the sex cult that Effective Altruism is:
I'd like to hear what feminists have to say about that picture in TFA of that woman and six men watching her. Just as I'd like to hear what feminists have to say about Amodei's wife who used to run a porn site.
And while a sex-cult abusing women is one thing. Theft is another. More emphasis should be put though on the lying scums in their ranks, like the polyamorous SBF and Caroline Ellison or their guru and its $15m ill-acquired mansion in the UK (which hasn't been clawed back yet but could).
And speaking of lying scums stealing money, we know about several making constant announcements were they lie, lie and then lie some more to ensure a never-ending stream of money and insane valuation of their proprietary, closed, not altruistic at all companies.
A despicable movement and I, for one, I'm very happy that there are companies out there releasing open-weights models so that the thieving-and-sex-cult at least get some competition.
> I'd like to hear what feminists have to say about that picture in TFA of that woman and six men watching her.
That picture is literally a joke based on a meme, which is based on a porn star's promotional image. To the extent feminists care, I would expect that care to be about the underlying subject matter rather than about this satire of it.
Is it really a satire if the same woman in the photo (Aella) went on to have a "birthday gangbang" with numerous men, some of whom are present in the photo?
He does not understand the issue. Not presenting any backing for his ideas, just emotions. The idea has been misused by the bros immensely, that does not help to grasp it either. But AI extermination of humans is a well known problem we have no solution for. Luckily AGI is not so close.
But AI extermination of humans is a well known problem we have no solution for
Sure, but the realm of fictional problems is so huge that we just can't come up with solutions to all of them. I like watching the Terminator movies (well, the first couple) as much as the next person but I can't be spending my time imagining how we could defeat skynet.
From the friendly article:
"Somewhere, some rationalist-brained person is reading this and saying I'm a "low decoupler", and that whether they run a sex cult has nothing to do with whether they're correct."
I’m literally speechless. I just can’t.
Horrible people
I guess the EA and rationalist cult members are out in force since this got flagged. Not a good look guys, you're just confirming what everyone thinks.
It may have been flagged as the title being misleading, as it did not really present as many details around the sex part as the title suggested.
Jokes aside it would have probably flown a bit better by removing the sex from the title, as the issue and problem brought up is about it being a cult, and not about the sex part specifically.
The title is the one used by the author, how is that misleading?
I meant about the title as used by the author, not the one who linked it here. But in any case I said it was a joke about not giving many sex related details in the article.
I mean I saw it 10 hours later. I'm thrilled to see stuff like this getting multiple hundreds of upvotes on HN, one of the deepest dens of tech capitalist worship in the world. It performing here is a couple steps from it performing on LessWrong itself. I take that as a sign a breakthrough is nearing. It is probably linked to whether the AI lab IPOs are successful or not.
The really amazing thing is, if the Yudkowskians are this weak, how doomed is the rest of our society that can't put up any defense to them completely dominating the narrative about where our economy is heading?
90% of this is just driven by strong emotional reactions from someone that feels uncomfortable about people being open about their non-standard sexual life, in the same way that many people have weird negative reactions about polyamorous relationships in general.
The remaining 10% is just the author trying to sprinkle in cult-like insinuations to justify his contempt.
Regularly meeting with people that have shared interests and ideas is not a cult.
Finding some of those people attractive and having sex with them is not a cult.
But somehow the fact that the romantic/sexual relationships are not traditionally monogamous makes it a cult?