It amazes me how pieces like this never stop to question whether these beliefs are genuine.
If I'm one of these researchers who's leaving their position, "I am so smart that I can make something world endingly dangerous, but also so principled that I'm willing to walk away from it" is a pretty slick bit of self promotion.
Even if I was making a career change for unrelated reasons, that's absolutely the story I'd be telling.
"AI hardly needs to wipe out humanity in order to be harmful, though. Many experts predict that more powerful models will lead to a coming wave of AI-assisted cyberattacks. The technology is now widely used for disinformation campaigns, and military adoption of AI is accelerating rapidly."
I fear the vast (and growing) human incompetence in essential domains. Education and science are crucial to society, and reasoning is necessary to cooperation at every level.
The "zone" of public discourse is flooded with sewage. US news business has lost its mind. And government...
Humans being "in-the-loop" is not enough. Humans must know what they are actually saying and doing. Humans just being "in-the-loop" cannot fix incompetence in social policy and software design.
Humans are "in-the-loop" in corporations and that isn't working very well for society.
Right now there's pretty limited set of actions that rogue AI can do with internet access. The most dangerous options are hacking into biolab facility, and maybe hacking into nuclear reactors or nuclear weapons.
These things should be as secure as possible, and I don't think rogue AIs, even super intelligent ones, have the capability to destroy humanity. Not to mention, super intelligent AIs would know that destroying humanity will also destroy themselves.
Maybe in 20 years once we have only self driving cars, robots in every household or maybe robots with weapons, then AI would pose a real extinction risk. But we're not there yet and it depends less on the development of AI and more on the development of physical real world robots.
A common retort to someone talking about extinction risk is "only people which don't work on AI research or understand transformers are scared. people that work on LLMs know better than to fear matrix multiplication".
Just because a bit of harmless algebra is at the foundation of a technologie doesn’t make it so that all technology build with that is harmless too.
There are many valid reasons to fear the impacts that LLM’s have on society
That's like saying don't be scared of layoffs because you know better than to fear the spreadsheet. The technology is generally not a reason to fear, or not fear, something.
It amazes me how pieces like this never stop to question whether these beliefs are genuine.
If I'm one of these researchers who's leaving their position, "I am so smart that I can make something world endingly dangerous, but also so principled that I'm willing to walk away from it" is a pretty slick bit of self promotion.
Even if I was making a career change for unrelated reasons, that's absolutely the story I'd be telling.
I personally have been talking to leading researchers in the field for a decade and they were taking these problems seriously then.
I have mostly pity for you, that your life is so filled with dishonesty.
From TFA:
I fear the vast (and growing) human incompetence in essential domains. Education and science are crucial to society, and reasoning is necessary to cooperation at every level.The "zone" of public discourse is flooded with sewage. US news business has lost its mind. And government...
Humans being "in-the-loop" is not enough. Humans must know what they are actually saying and doing. Humans just being "in-the-loop" cannot fix incompetence in social policy and software design.
Humans are "in-the-loop" in corporations and that isn't working very well for society.
Right now there's pretty limited set of actions that rogue AI can do with internet access. The most dangerous options are hacking into biolab facility, and maybe hacking into nuclear reactors or nuclear weapons.
These things should be as secure as possible, and I don't think rogue AIs, even super intelligent ones, have the capability to destroy humanity. Not to mention, super intelligent AIs would know that destroying humanity will also destroy themselves.
Maybe in 20 years once we have only self driving cars, robots in every household or maybe robots with weapons, then AI would pose a real extinction risk. But we're not there yet and it depends less on the development of AI and more on the development of physical real world robots.
From permanent underclass to human extinction. What's next?
Narrative collapse.
A common retort to someone talking about extinction risk is "only people which don't work on AI research or understand transformers are scared. people that work on LLMs know better than to fear matrix multiplication".
Just because a bit of harmless algebra is at the foundation of a technologie doesn’t make it so that all technology build with that is harmless too. There are many valid reasons to fear the impacts that LLM’s have on society
ehhh....to be pedantic, LLMs, like any NNs derive power from _not_ being pure matrix multiplication - they are "halved" matrix algebra, due to ReLU
Does Geoffrey Hinton work on AI research?
I can't tell if you're for or against that stance. But many leading scientists take these issues extremely seriously.
That's like saying don't be scared of layoffs because you know better than to fear the spreadsheet. The technology is generally not a reason to fear, or not fear, something.