Say I am willing to accept pill number 3, which essentially means giving up the world we know and live in.
First of all, this means a RACE OF ARMS of AI models fighting each other and in the end the party with more AI resources will win, while consuming a shit ton of resources.
Second of all, the more I believe AI will be able to do anything, the more I should focus on things that require a human (your words, not mine). Therefore, the only correct stance I see is not believe in AI and therefore not use it, or fully believe in AI and therefore become an expert in something it cannot do, i.e. not using it either.
A core assumption everyone makes that I question is that a race, fight, or competition is a preordained outcome. This is a cornerstone of the doomer thesis and its variations.
Evolution has no preference. There are cooperators, both direct and indirect, and predators both direct and indirect. Indirect cooperators are things like bacteria in soil that recycle plant and animal waste, while indirect predation is when something just pushes something else out or otherwise harms it without direct confrontation.
So if we go by evolution, we could say it’s a toss up.
Evolution is blind though. It has no ability to plan or exercise foresight.
Intelligence can and does plan. Superintelligence, we have to assume, will plan better and further into the future and consider more options.
All this is to say: what if it looks for win-win options or at least win-neutral ones?
I’m simply arguing that the default assumption of hostility says more about us than a potential alien intelligence. I’m not aware or any reason it couldn’t be hostile or dangerous, but it’s not preordained and I’d argue it’s less likely here than with blind evolution.
This is the bitter lesson that the labs may have to learn for themselves. Like, even with AGI, without corresponding enhancements in robotics, the machines basically _can't_ take over. Intelligence is not all you need (you also need hands).
You see this in Dario's appearances, where he jumps from AI is doing well using verifiable rewards (true), to therefore we can automate basically all science (maybe, but not without a lot more work in how it interacts with the physical world).
I think your unpilled and lower tier pills are often disguised anti-pills. Anti-pilled as in AI is morally bad. I count myself as both ASI pilled and anti-pilled.
the problem I have with these is suppose you accept that soon there will be something capable of completely automating all useful tasks you can do. How can you prepare?
We can't all learn to be athletes. Going into social roles (nurse, psychologist) may work for some, but we equally can't have an economy where we all pay each other for personal service (I suppose maybe this is debatable, but I expect unemployment would have to be quite large, and the economy would be utterly foreign. The price of acting on this prediction (quitting high paying job to be slightly early on learning to be a nurse) is high, and a prediction of nurses being paid more than today's lawyers seems to be unlikely to happen all that fast), so you'd be working a low paid and physically difficult job for quite some time before your bet paid off, assuming it even does. And is there really much advantage to doing that now instead of later?
so you're left with: basically do the same thing you're already doing to make money, save up what you can, and see what happens.
I see unpilled people.
Where do I see them? Everywhere. The majority of people have not taken the first pill.
Most people have no idea what frontier AIs can do for them.
Yeah; Perhaps people do see the propaganda that "frontier AIs can do for them" and choose to dream of a better world, another world, a world where human-kind does better than slop;
Say I am willing to accept pill number 3, which essentially means giving up the world we know and live in.
First of all, this means a RACE OF ARMS of AI models fighting each other and in the end the party with more AI resources will win, while consuming a shit ton of resources.
Second of all, the more I believe AI will be able to do anything, the more I should focus on things that require a human (your words, not mine). Therefore, the only correct stance I see is not believe in AI and therefore not use it, or fully believe in AI and therefore become an expert in something it cannot do, i.e. not using it either.
Edit: great read, amazingly written
A core assumption everyone makes that I question is that a race, fight, or competition is a preordained outcome. This is a cornerstone of the doomer thesis and its variations.
Evolution has no preference. There are cooperators, both direct and indirect, and predators both direct and indirect. Indirect cooperators are things like bacteria in soil that recycle plant and animal waste, while indirect predation is when something just pushes something else out or otherwise harms it without direct confrontation.
So if we go by evolution, we could say it’s a toss up.
Evolution is blind though. It has no ability to plan or exercise foresight.
Intelligence can and does plan. Superintelligence, we have to assume, will plan better and further into the future and consider more options.
All this is to say: what if it looks for win-win options or at least win-neutral ones?
I’m simply arguing that the default assumption of hostility says more about us than a potential alien intelligence. I’m not aware or any reason it couldn’t be hostile or dangerous, but it’s not preordained and I’d argue it’s less likely here than with blind evolution.
You could take any pills and have any fantasies in your small coding corner. But saying very strong word "better" has very distinctive aftertaste.
It makes the leap from:
AI is improving dramatically in fields with logical formal structure (maths, comp sci.)
to:
AI should then should eventually conquer essentially every other cognitive task.
This is the bitter lesson that the labs may have to learn for themselves. Like, even with AGI, without corresponding enhancements in robotics, the machines basically _can't_ take over. Intelligence is not all you need (you also need hands).
You see this in Dario's appearances, where he jumps from AI is doing well using verifiable rewards (true), to therefore we can automate basically all science (maybe, but not without a lot more work in how it interacts with the physical world).
I think your unpilled and lower tier pills are often disguised anti-pills. Anti-pilled as in AI is morally bad. I count myself as both ASI pilled and anti-pilled.
the problem I have with these is suppose you accept that soon there will be something capable of completely automating all useful tasks you can do. How can you prepare?
We can't all learn to be athletes. Going into social roles (nurse, psychologist) may work for some, but we equally can't have an economy where we all pay each other for personal service (I suppose maybe this is debatable, but I expect unemployment would have to be quite large, and the economy would be utterly foreign. The price of acting on this prediction (quitting high paying job to be slightly early on learning to be a nurse) is high, and a prediction of nurses being paid more than today's lawyers seems to be unlikely to happen all that fast), so you'd be working a low paid and physically difficult job for quite some time before your bet paid off, assuming it even does. And is there really much advantage to doing that now instead of later?
so you're left with: basically do the same thing you're already doing to make money, save up what you can, and see what happens.
We're not even taking all the same pills on long-settled science like climate change.
And most of the world is still taking crazy pills.
quote:
Yeah; Perhaps people do see the propaganda that "frontier AIs can do for them" and choose to dream of a better world, another world, a world where human-kind does better than slop;