I'm confused. Shouldn't you conclude it's unethical /not/ to support safety research based off your own points.
My understanding of "safety" is that it means formalising morality, ensuring AI doesn't go astray as it recursively improves it's self, destroying our future in some horrifying way possibly worse than death.
> AI capability is limited by AI safety in the way the power of a rifle is limited by how safe it is to use
But merely making it safe to use still prevents unnecessary death. And advocacy against rifles in general, or general gun safety is a viable path to limiting the negative outcomes.
Rifles are banned in many places on Earth and they also backfire and misfire less than they used to.
Even so, you can do the simple "call your representative" and "hand out fliers" and "talk to people about how AI is net negative" with no cost to yourself and it absolutely aligns with your fundamental beliefs, right?
it's a real issue but I'm not sure it's the mass mobilization problem the article makes it out to be. pretty sure it's growing extremely fast, and it's a new field in general. the models will keep growing in capability because of human game theory (I need to make my model better because my competitors are). it's not practical to expect millions in headcount to move from frontier capability research to safety
I’ve decided that the third Industrial Revolution is a foregone conclusion, so I’ve pivoted to try to make it better for everybody, not just corporations and governments. If we do it right, at least we will get a new washing machine* out of it.
I also think our work will help to underpin safety, and that actually a big deal, because as it turns out safety isn’t a bolt-on. It must be something resembling character, or it will be discarded in the same way that humans set rules aside when the calculus seems to support doing so under the governing incentive framework.
What I amo doing may slightly accelerate the process, but if I don’t succeed there is a much higher probability of having robots that are not generally useful to humans as a type, and will be completely captured by industry as subscription units, surveillance devices, and in general something that companies can use to harvest resources from you rather than a device you own that delivers all of it’s potential value to you as the owner.
The saying is, "you can't prove a negative" - mostly true, but really the issue is that with such a wide open future of possibilities for AI and safe (or unsafe) outcomes, there's an empirical issue of it being very hard or impossible to disprove things that are still hidden (haven't happened yet). Humans are maximizers in that we try to spend the least amount of effort when we're not sure what will happen, but after something bad happens, we'll act quickly to address that specific issue. It'll probably take a lot of small and large issues to happen for people to build up their societal "safety protocol" around the technology over time.
The most likely path I see for human survival and flourishing is for AI to do something in the near future so destructive and terrifying that it unites humanity behind permanently banning it at any cost.
It therefore seems like "AI safety" people may actually be working against our safety.
The reason I don’t pivot into AI safety (in a topic I might have comparative advantage in as a mathematician) is the following.
(1) I believe increasing AI capability is a net negative.
(2) AI capability is limited by AI safety in the way the power of a rifle is limited by how safe it is to use.
I.e., AI safety and AI capability look like two sides of a coin to me.
If I were disillusioned of this view I might consider it. But right now it just seems unethical and against my fundamental beliefs.
I'm confused. Shouldn't you conclude it's unethical /not/ to support safety research based off your own points.
My understanding of "safety" is that it means formalising morality, ensuring AI doesn't go astray as it recursively improves it's self, destroying our future in some horrifying way possibly worse than death.
Is a misaligned AI worse than a super - capable AI aligned with corporate interest? Because I can't really tell anymore.
> AI capability is limited by AI safety in the way the power of a rifle is limited by how safe it is to use
But merely making it safe to use still prevents unnecessary death. And advocacy against rifles in general, or general gun safety is a viable path to limiting the negative outcomes.
Rifles are banned in many places on Earth and they also backfire and misfire less than they used to.
Even so, you can do the simple "call your representative" and "hand out fliers" and "talk to people about how AI is net negative" with no cost to yourself and it absolutely aligns with your fundamental beliefs, right?
This is true. Really what I had in mind was research in AI interpretability.
I feel similarly right now, which is patr of why I shut down my company.
honestly I'm not sure a lack of safety has been stopping anyone. I suspect it's caused a lot of m̶a̶r̶k̶e̶t̶i̶n̶g̶ problems lately.
> We will need MILLIONS of people in the field.
it's a real issue but I'm not sure it's the mass mobilization problem the article makes it out to be. pretty sure it's growing extremely fast, and it's a new field in general. the models will keep growing in capability because of human game theory (I need to make my model better because my competitors are). it's not practical to expect millions in headcount to move from frontier capability research to safety
I’ve decided that the third Industrial Revolution is a foregone conclusion, so I’ve pivoted to try to make it better for everybody, not just corporations and governments. If we do it right, at least we will get a new washing machine* out of it.
I also think our work will help to underpin safety, and that actually a big deal, because as it turns out safety isn’t a bolt-on. It must be something resembling character, or it will be discarded in the same way that humans set rules aside when the calculus seems to support doing so under the governing incentive framework.
What I amo doing may slightly accelerate the process, but if I don’t succeed there is a much higher probability of having robots that are not generally useful to humans as a type, and will be completely captured by industry as subscription units, surveillance devices, and in general something that companies can use to harvest resources from you rather than a device you own that delivers all of it’s potential value to you as the owner.
*silent killers of human misery since 1937
What is it you’re doing?
tracedynamics.ai
Some of my ideas on AI safety at https://ctsmyth.substack.com/p/all-you-need-is-character
The saying is, "you can't prove a negative" - mostly true, but really the issue is that with such a wide open future of possibilities for AI and safe (or unsafe) outcomes, there's an empirical issue of it being very hard or impossible to disprove things that are still hidden (haven't happened yet). Humans are maximizers in that we try to spend the least amount of effort when we're not sure what will happen, but after something bad happens, we'll act quickly to address that specific issue. It'll probably take a lot of small and large issues to happen for people to build up their societal "safety protocol" around the technology over time.
AI Safety seems like an activst job. I just dont care to much about AI "safety".
The most likely path I see for human survival and flourishing is for AI to do something in the near future so destructive and terrifying that it unites humanity behind permanently banning it at any cost.
It therefore seems like "AI safety" people may actually be working against our safety.