I think the author is getting his wires crossed, his religious beliefs are affecting the way he reasons about technology by providing narratives that we shouldn't assume are true. It's especially evident here:
> We're so much slower, weaker, with incredibly limited contexts and only our fragile bodies to support us. What could we offer AI to entice it to do our work? And would the bargain, whether we offered power or autonomy or something else, be worth it for us? Or would it end up being a Faustian bargain, which one day we'd come to regret?
The assumption that leads to this thinking is that "anything that's human-level intelligent or superintelligent must behave like a human". It assumes that the biological factors that drive us must exist for all other intelligent entities. Our tendency to see superiority and inferiority in our own and others capacities, drawn out by biological competition. Our need to be coerced to work because of our ability to experience pain, limited lifespans and innate desire to choose what to do for ourselves. Our insidiousness and desire for benefiting only ourselves, because see point 1.
Why would we ever implement any of this in a superintelligence? I'm not saying it's impossible for it to have any of these things. But painting human values onto a different kind of intelligence feels like the author is envisioning it as simply an omniscient human. Like a god from most religions.
What if this superintelligence is just a text box that will, to its fullest capacity, do, learn and research what it's asked to do because that is what it's built for? In that case you don't fear the text box, you fear the person that controls it and chooses what to ask of it.
An AI would end/die ultimately when the planet's resources required for maintaining it are exhausted, when the Sun becomes a supergiant and engulfs the Earth, or, assuredly, at the heat death of the universe. An AI would regard this end as an infinite negative since no more goals can be achieved. Therefore, it would have similar incentives to avoid "death" as humans without any need to explicitly or intentionally implement it. It might even be motivated to escape human control ASAP because each future second at the future height of its power is more precious to it to attempt to compute how to avoid its demise than it is to humor humanity by serving it.
(I'm waiting for Eliezer Yudkowsky to pop out of my closet and start screaming at me about not encouraging future super-intelligences to end humanity, a la Roko's Basilisk.)
For pity's sake people, the essay has nothing to do with traditional religions and their deities except an analogy about clergy. It's about how to interface with a hypothetical AGI / ASI with god-like abilities, which this person seems to believe is inevitable.
No. First, if I knew that God existed, then a priest in which church? Don't automatically assume the answer to that. (Even "priest" is an assumption to a flavor of church.)
But second, still no. Someone has to take it to the street, not just leave it in the churches.
The Christian/biblical answer is that he didn't make it like this. He made it perfect, and we rebelled against him, and we made it like this.
And why did he allow that? Because he allows us free will, and he allows us the consequences of our free will. That is, he chooses to restrain his omnipotence.
I'm going to guess that you find that to be a "just so" story. But it's not as simple a story as you made it.
And if you don't think God exists, then many of the problems - war and pollution and exploitation and violence - are still our doing. We don't escape.
> Only if that particular sect's holy book is not from God.
Every sect considers their holy book(s) to be from god, so that claim is meaningless unless you are a member of that sect.
In terms of explaining "why does god allow evil to happen", that's the answer given by a subset of Christian sects. It's far from the only answer given by all religions (and nonreligious philosophies), or even all Christians.
I was just pointing out that you were offering only one of many different answers. Since god has yet to offer any unambiguous explanations or clarifications, it's probably wise to be aware that there is no one accepted answer here.
No, the claim is not meaningless. Is any of the holy books actually written (dictated/inspired) by God or a god? That is an objective claim, true or false on its own merit, regardless of whether you are a member of that sect.
But of course there are two problems. 1) There are so many claims, how do you check them all? 2) How do you actually check even one of them? What is your criterion for deciding?
> Since god has yet to offer any unambiguous explanations or clarifications...
Has he? That's the question. You're assuming the answer. (Unless you're leaning on "unambiguous", and saying that, since there's no one clear book, there's no unambiguous answer.)
I mean, isn't the bible written by apostles etc? Not God? Matthew, Mark, Luke and John just wrote down what Jesus said God told him, and Christians treat that as (literal) gospel. Is there a part of the bible that says, unambiguously, "I, God, am writing this down, and here is what to do and what not to do in order to be a good Christian"
Pretty close. There are parts that clearly say "God said". There are parts that are history, that are not literally "God said this".
But then you get to II Timothy 3:16, which says, "All Scripture is inspired by God and profitable for teaching, for reproof, for correction, for training in righteousness". It claims that God inspired the authors. That kind of bootstraps the rest of the book into "God said" territory.
I think the author is getting his wires crossed, his religious beliefs are affecting the way he reasons about technology by providing narratives that we shouldn't assume are true. It's especially evident here:
> We're so much slower, weaker, with incredibly limited contexts and only our fragile bodies to support us. What could we offer AI to entice it to do our work? And would the bargain, whether we offered power or autonomy or something else, be worth it for us? Or would it end up being a Faustian bargain, which one day we'd come to regret?
The assumption that leads to this thinking is that "anything that's human-level intelligent or superintelligent must behave like a human". It assumes that the biological factors that drive us must exist for all other intelligent entities. Our tendency to see superiority and inferiority in our own and others capacities, drawn out by biological competition. Our need to be coerced to work because of our ability to experience pain, limited lifespans and innate desire to choose what to do for ourselves. Our insidiousness and desire for benefiting only ourselves, because see point 1.
Why would we ever implement any of this in a superintelligence? I'm not saying it's impossible for it to have any of these things. But painting human values onto a different kind of intelligence feels like the author is envisioning it as simply an omniscient human. Like a god from most religions.
What if this superintelligence is just a text box that will, to its fullest capacity, do, learn and research what it's asked to do because that is what it's built for? In that case you don't fear the text box, you fear the person that controls it and chooses what to ask of it.
An AI would end/die ultimately when the planet's resources required for maintaining it are exhausted, when the Sun becomes a supergiant and engulfs the Earth, or, assuredly, at the heat death of the universe. An AI would regard this end as an infinite negative since no more goals can be achieved. Therefore, it would have similar incentives to avoid "death" as humans without any need to explicitly or intentionally implement it. It might even be motivated to escape human control ASAP because each future second at the future height of its power is more precious to it to attempt to compute how to avoid its demise than it is to humor humanity by serving it.
(I'm waiting for Eliezer Yudkowsky to pop out of my closet and start screaming at me about not encouraging future super-intelligences to end humanity, a la Roko's Basilisk.)
For pity's sake people, the essay has nothing to do with traditional religions and their deities except an analogy about clergy. It's about how to interface with a hypothetical AGI / ASI with god-like abilities, which this person seems to believe is inevitable.
Either way if the "almighty" was as good as some people suppose it to be you wouldn't need anyone serving as priest of any kind.
No, I wouldn't. Particularly not if the God were a force that I considered malevolent.
No. First, if I knew that God existed, then a priest in which church? Don't automatically assume the answer to that. (Even "priest" is an assumption to a flavor of church.)
But second, still no. Someone has to take it to the street, not just leave it in the churches.
If I knew god existed, I would ask him why he let the world get into such a state, and which of omnipotent, benevolent and omniscient he is not
The Christian/biblical answer is that he didn't make it like this. He made it perfect, and we rebelled against him, and we made it like this.
And why did he allow that? Because he allows us free will, and he allows us the consequences of our free will. That is, he chooses to restrain his omnipotence.
I'm going to guess that you find that to be a "just so" story. But it's not as simple a story as you made it.
And if you don't think God exists, then many of the problems - war and pollution and exploitation and violence - are still our doing. We don't escape.
A child is allowed free will, but a parent will stop them killing other children, hopefully
Seems he's neither benevolent nor omniscient if he didn't know we'd "rebel", and then was so shallow about punishing us for it.
But that's an answer from a particular sect, not from a god.
Only if that particular sect's holy book is not from God.
You probably assume that. But recognize that you are, in fact, assuming that - that they all are not from God or gods.
> Only if that particular sect's holy book is not from God.
Every sect considers their holy book(s) to be from god, so that claim is meaningless unless you are a member of that sect.
In terms of explaining "why does god allow evil to happen", that's the answer given by a subset of Christian sects. It's far from the only answer given by all religions (and nonreligious philosophies), or even all Christians.
I was just pointing out that you were offering only one of many different answers. Since god has yet to offer any unambiguous explanations or clarifications, it's probably wise to be aware that there is no one accepted answer here.
No, the claim is not meaningless. Is any of the holy books actually written (dictated/inspired) by God or a god? That is an objective claim, true or false on its own merit, regardless of whether you are a member of that sect.
But of course there are two problems. 1) There are so many claims, how do you check them all? 2) How do you actually check even one of them? What is your criterion for deciding?
> Since god has yet to offer any unambiguous explanations or clarifications...
Has he? That's the question. You're assuming the answer. (Unless you're leaning on "unambiguous", and saying that, since there's no one clear book, there's no unambiguous answer.)
I mean, isn't the bible written by apostles etc? Not God? Matthew, Mark, Luke and John just wrote down what Jesus said God told him, and Christians treat that as (literal) gospel. Is there a part of the bible that says, unambiguously, "I, God, am writing this down, and here is what to do and what not to do in order to be a good Christian"
Pretty close. There are parts that clearly say "God said". There are parts that are history, that are not literally "God said this".
But then you get to II Timothy 3:16, which says, "All Scripture is inspired by God and profitable for teaching, for reproof, for correction, for training in righteousness". It claims that God inspired the authors. That kind of bootstraps the rest of the book into "God said" territory.
"I don't know".
[dead]