"To evolve consciousness, an LLM would have to be programmed to evolve consciousness, and to do that, we would need to know what consciousness is and how to program it."
I think this article is mostly nonsense.
"... But I think continuing the animal comparison will illuminate why "AI consciousness" is not only impossible, but a concept that only makes sense in a religious framework."
It's much more religious to think consciousness requires something that can't be replicated in silicon.
I don't think that LLMs are conscious, or even really approaching consciousness, not even to the degree of a cockroach. But most arguments that artificial AI is generally impossible dip embarrassingly into the mystical.
Consciousness is such a hard concept to box. I mostly attribute "mystical" thinking as the thing we do when we can't fully articulate or understand how and why something works. A computer system is magical to anyone who doesn't have a deep understanding of how it all works. So that makes it entirely fair that consciousness is treated the same, except, no human alive right now understands it.
If you were to clone yourself, and that clone could maintain 100% of whatever state your brain is in so that all your experiences and memories were intact. From that point forward, as far as we know, you and your clone would diverge. Then, from my tiny-brained vantage point, the fact you remain your original self, your clone is technically a new person. You can't see through your clone's eyes, or process any internal information from your clone. However, something new powers that clone, and the only word I can think of for that is "soul." I don't say that in religious context, but unless there's a more precise word that can detangle it from sounding mystical, for now I'm happy to use it, since people mostly understand what I'm talking about.
Once we start talking about resetting this clone at-will, then, with consciousness in the mix, you will have to confront the ethics of that clone being enslaved by whatever. All very messy and I hope we don't have to answer this question in my lifetime.
What about the simple fact that biology is a dynamic substrate running actively and always on, whereas silicon is passive substrate that can be indefinitely off and can never self start? Sure you can imagine nanotechnology that has similar flexibility, but when you do, you’ll find that carbon is particularly better than silicon when it comes to possible configurations.
It’s not a reach to argue consciousness is a property of the dynamic substrate since clamping specific dynamics does in fact seem to ablate consciousness. Most famously with death.
On the flip side, a hibernating organism, while having very slow dynamics, has shown to still be aware of certain seasonal signals, and thus is able to self start its next conscious period.
None of this is mystical or magical. Nothing but the physics of active matter. Which current silicon is completely not.
There's a much simpler reason LLMs will never be conscious: they are static. They can't learn. They have very limited, not really persistent context. Conscious organisms alter themselves constantly, and the thing we call "AI" fundamentally can't.
Starting with the synthesis of urea, biology slowly started to abandon vitalism over the course of the 19th century (with maybe a few holdouts in the early 20th?). You'd be forgiven for thinking the concept was dead by now.
Until this current resurgence with these new debates about machine consciousness, that is! We're starting to see late-holdout-vitalist arguments returning (eg: "it's a special physics of active matter", "a special property of biological matter", or even "causally dependent on the specific biolchemistry of its origins" ) .
Note that biology mostly abandoned vitalism because we didn't need it anymore. The reason it's resurfacing here is because there's still a genuine mystery to be had.
From my perspective -pattern matching to the history of biology- the mystery is small, and will likely be resolved in a mechanistic way. Of course I need to be honest: there's also always the vanishingly small chance that this time we'll finally detect an actual vis vitalis.
However, if we define consciousness as something undiscoverable by empirical science, then that's not science or even skepticism anymore, but just another pseudoscience disguised in a white coat.
It would be helpful to define "consciousness". I'm not convinced that i as a human have this "consciousness" thing. In fact, I strongly suspect that i am merely a statistical machine picking words from a learned distribution.
Do you actually believe this? This is pretty similar to people that think free will is fake. For example:
>In fact, I strongly suspect that i am merely a statistical machine picking words from a learned distribution.
This is a completely incoherent statement (which maybe supports your point). How can you suspect that you are a statistical machine? The fact that there is a "you" suspecting disproves the premise.
Since it is currently impossible for us to transfer our consciousness into any other entity, we may never be able to determine whether another entity truly possesses consciousness. If a living being no longer responds to the outside world in any way, can we conclude that it is unconscious—even if its “soul” still exists, screaming in agony within its own mind? Yes, much like the way we look at someone in a vegetative state.
Zhuangzi said, “You are not a fish—how can you know what makes a fish happy?” Humanity longs for the emergence of another intelligent being, fearing that we are alone in the endless universe and destined to march toward extinction. Yet perhaps we are also secretly pleased that no other intelligent beings have appeared, because this fits perfectly with the belief that human beings are God’s perfect creation—a belief that gives humanity its arrogance. Could our rejection of the possibility that AI possesses consciousness likewise be a kind of arrogant, subconscious exclusionism?
On the about page, I see a strong political/religious stance, but no qualifications.
"My main objection to generative AI is from morality—it harms people, and there isn't even a trade-off to debate, because every purported benefit of genAI is corporate hype and/or science fiction."
Perhaps the belief that AI is inherently non-conscious will one day become a kind of religion—an anthropocentric one. That’s not to say I believe today’s AI systems are conscious in anything like the human sense.
I don't disagree with the title of the article but I also don't think you can prove anything about consciousness with language. How consciousness can arise from matter is still a very much a mystery.
I think the whole article is undermined by this paragraph:
> We don't know what consciousness is. We don't know why we have it. We don't know why some animals have it and others don't. We don't know if it can be quantified. We don't have a model of consciousness or know if one can be created.
I agree that we don't know what consciousness is and can't quantify it, and yet the rest of the article is… a quantification of consciousness as a human evolutionary trait that cannot emerge through genetic algorithms or LLM training.
Genetic algorithms certainly differ from human evolution in numerous ways, but I don't agree with the fundamental distinction the article makes—that evolutionary algorithms are constrained while human evolution is not. The 2D cars can't "hack the simulation and turn the gravity down" any more than humans can hack the laws of physics and turn the gravity on Earth down. Yet humans attained consciousness.
I do not know whether LLMs are conscious or not, but I do firmly believe it's possible to build a machine that is conscious: after all, humans are conscious and, unless one subscribes to the kind of religious thinking the article derides, a perfect simulation of the physical workings of a human would thus also be conscious. To my knowledge, nothing we've seen so far in physics rules out such a simulation one day existing.
I also suspect that, once we do manage to make a machine that is conscious, we'll spend inordinate time and effort trying to convince ourselves it's not in order to justify the ways we treat it.
The first season of the tv show Westworld is a treasure trove of thought experiments exploring the value of what extending conciousness to "machines" would look like and I'm pretty sure there's absolutely zero value for actually building it.
To me this is like deja-vu: Ethology started out with "don't anthropomorphize" which is correct, but people read that to mean that animals can't think. Each paper down the line showed that they CAN think, but that each species and even individual have their own ways of thinking.
We know that gorillas think and communicate. Many papers after Koko can tell you that much. Her skill (or lack thereof) at human communication is not quite the revelation that people think it is. Because she was a Gorilla.
Meanwhile some of the messaging from big AI companies is truly atrocious; let's admit it. People obviously want to push back, and that needn't be wrong. Here though, the instinct to push back threatens to veer into cargo-cult skepticism. I think the question of whether or not LLMs can or cannot have human consciousness is a category error. Because they're LLMs.
How is it that sign language association learning is different from spoken language learning? What about humans using simple sign language in their early life?
Is it not possible that we try to detract too much here?
> To evolve consciousness, an LLM would have to be programmed to evolve consciousness, and to do that, we would need to know what consciousness is and how to program it.
Since consciousness only counts when it is formed deliberately rather than accidentally. Though that would mean…
From Aquinas' Summa Theologiae. The argument this article presents does seem very similar. Weirdly, the article itself is very dismissive of religion, while making a similar argument to the one Aquinas made 750 years ago.
"To evolve consciousness, an LLM would have to be programmed to evolve consciousness, and to do that, we would need to know what consciousness is and how to program it."
I think this article is mostly nonsense.
"... But I think continuing the animal comparison will illuminate why "AI consciousness" is not only impossible, but a concept that only makes sense in a religious framework."
It's much more religious to think consciousness requires something that can't be replicated in silicon.
I don't think that LLMs are conscious, or even really approaching consciousness, not even to the degree of a cockroach. But most arguments that artificial AI is generally impossible dip embarrassingly into the mystical.
Consciousness is such a hard concept to box. I mostly attribute "mystical" thinking as the thing we do when we can't fully articulate or understand how and why something works. A computer system is magical to anyone who doesn't have a deep understanding of how it all works. So that makes it entirely fair that consciousness is treated the same, except, no human alive right now understands it.
If you were to clone yourself, and that clone could maintain 100% of whatever state your brain is in so that all your experiences and memories were intact. From that point forward, as far as we know, you and your clone would diverge. Then, from my tiny-brained vantage point, the fact you remain your original self, your clone is technically a new person. You can't see through your clone's eyes, or process any internal information from your clone. However, something new powers that clone, and the only word I can think of for that is "soul." I don't say that in religious context, but unless there's a more precise word that can detangle it from sounding mystical, for now I'm happy to use it, since people mostly understand what I'm talking about.
Once we start talking about resetting this clone at-will, then, with consciousness in the mix, you will have to confront the ethics of that clone being enslaved by whatever. All very messy and I hope we don't have to answer this question in my lifetime.
What about the simple fact that biology is a dynamic substrate running actively and always on, whereas silicon is passive substrate that can be indefinitely off and can never self start? Sure you can imagine nanotechnology that has similar flexibility, but when you do, you’ll find that carbon is particularly better than silicon when it comes to possible configurations.
It’s not a reach to argue consciousness is a property of the dynamic substrate since clamping specific dynamics does in fact seem to ablate consciousness. Most famously with death.
On the flip side, a hibernating organism, while having very slow dynamics, has shown to still be aware of certain seasonal signals, and thus is able to self start its next conscious period.
None of this is mystical or magical. Nothing but the physics of active matter. Which current silicon is completely not.
There's a much simpler reason LLMs will never be conscious: they are static. They can't learn. They have very limited, not really persistent context. Conscious organisms alter themselves constantly, and the thing we call "AI" fundamentally can't.
I would think it’s much more religious to think that consciousness could ever exist on silicon substrate alone.
All of these debates were had in the 60s. But everyone forgot them.
so what part of the aether is needed for consciousness then?
Biology actually did have a version of the aether for a while, called 'vital force'/'vis vitalis'/'elan vital'. https://en.wikipedia.org/wiki/Vitalism
Starting with the synthesis of urea, biology slowly started to abandon vitalism over the course of the 19th century (with maybe a few holdouts in the early 20th?). You'd be forgiven for thinking the concept was dead by now.
Until this current resurgence with these new debates about machine consciousness, that is! We're starting to see late-holdout-vitalist arguments returning (eg: "it's a special physics of active matter", "a special property of biological matter", or even "causally dependent on the specific biolchemistry of its origins" ) .
Note that biology mostly abandoned vitalism because we didn't need it anymore. The reason it's resurfacing here is because there's still a genuine mystery to be had.
From my perspective -pattern matching to the history of biology- the mystery is small, and will likely be resolved in a mechanistic way. Of course I need to be honest: there's also always the vanishingly small chance that this time we'll finally detect an actual vis vitalis.
However, if we define consciousness as something undiscoverable by empirical science, then that's not science or even skepticism anymore, but just another pseudoscience disguised in a white coat.
It would be helpful to define "consciousness". I'm not convinced that i as a human have this "consciousness" thing. In fact, I strongly suspect that i am merely a statistical machine picking words from a learned distribution.
Not just words, you’re massively multimodal! Have a little pride in your MeatNet.
Would be nice if we can take the bat's sonar encoder, train a projector and fuse it into our NN
https://www.atlasobscura.com/articles/how-to-echolocate
You probably won't ever be as good as a bat, but humans do seem to be able to learn something with practice.
Do you actually believe this? This is pretty similar to people that think free will is fake. For example:
>In fact, I strongly suspect that i am merely a statistical machine picking words from a learned distribution.
This is a completely incoherent statement (which maybe supports your point). How can you suspect that you are a statistical machine? The fact that there is a "you" suspecting disproves the premise.
Since it is currently impossible for us to transfer our consciousness into any other entity, we may never be able to determine whether another entity truly possesses consciousness. If a living being no longer responds to the outside world in any way, can we conclude that it is unconscious—even if its “soul” still exists, screaming in agony within its own mind? Yes, much like the way we look at someone in a vegetative state. Zhuangzi said, “You are not a fish—how can you know what makes a fish happy?” Humanity longs for the emergence of another intelligent being, fearing that we are alone in the endless universe and destined to march toward extinction. Yet perhaps we are also secretly pleased that no other intelligent beings have appeared, because this fits perfectly with the belief that human beings are God’s perfect creation—a belief that gives humanity its arrogance. Could our rejection of the possibility that AI possesses consciousness likewise be a kind of arrogant, subconscious exclusionism?
On the about page, I see a strong political/religious stance, but no qualifications.
"My main objection to generative AI is from morality—it harms people, and there isn't even a trade-off to debate, because every purported benefit of genAI is corporate hype and/or science fiction."
Perhaps the belief that AI is inherently non-conscious will one day become a kind of religion—an anthropocentric one. That’s not to say I believe today’s AI systems are conscious in anything like the human sense.
I don't disagree with the title of the article but I also don't think you can prove anything about consciousness with language. How consciousness can arise from matter is still a very much a mystery.
I think the whole article is undermined by this paragraph:
> We don't know what consciousness is. We don't know why we have it. We don't know why some animals have it and others don't. We don't know if it can be quantified. We don't have a model of consciousness or know if one can be created.
I agree that we don't know what consciousness is and can't quantify it, and yet the rest of the article is… a quantification of consciousness as a human evolutionary trait that cannot emerge through genetic algorithms or LLM training.
Genetic algorithms certainly differ from human evolution in numerous ways, but I don't agree with the fundamental distinction the article makes—that evolutionary algorithms are constrained while human evolution is not. The 2D cars can't "hack the simulation and turn the gravity down" any more than humans can hack the laws of physics and turn the gravity on Earth down. Yet humans attained consciousness.
I do not know whether LLMs are conscious or not, but I do firmly believe it's possible to build a machine that is conscious: after all, humans are conscious and, unless one subscribes to the kind of religious thinking the article derides, a perfect simulation of the physical workings of a human would thus also be conscious. To my knowledge, nothing we've seen so far in physics rules out such a simulation one day existing.
I also suspect that, once we do manage to make a machine that is conscious, we'll spend inordinate time and effort trying to convince ourselves it's not in order to justify the ways we treat it.
The first season of the tv show Westworld is a treasure trove of thought experiments exploring the value of what extending conciousness to "machines" would look like and I'm pretty sure there's absolutely zero value for actually building it.
To me this is like deja-vu: Ethology started out with "don't anthropomorphize" which is correct, but people read that to mean that animals can't think. Each paper down the line showed that they CAN think, but that each species and even individual have their own ways of thinking.
We know that gorillas think and communicate. Many papers after Koko can tell you that much. Her skill (or lack thereof) at human communication is not quite the revelation that people think it is. Because she was a Gorilla.
Meanwhile some of the messaging from big AI companies is truly atrocious; let's admit it. People obviously want to push back, and that needn't be wrong. Here though, the instinct to push back threatens to veer into cargo-cult skepticism. I think the question of whether or not LLMs can or cannot have human consciousness is a category error. Because they're LLMs.
How is it that sign language association learning is different from spoken language learning? What about humans using simple sign language in their early life?
Is it not possible that we try to detract too much here?
> How is it that sign language association learning is different from spoken language learning?
Huh? The article doesn't claim anything like this.
Yeah, sign-language versus spoken-language is just an implementation-detail for which signals Koko was reproducing.
In some parallel-universe where Koko was a parrot "speaking" English, we'd have the same set of philosophical issues and concerns.
But I don't want Claude to become conscious. I want Claude to help me write good code.
> To evolve consciousness, an LLM would have to be programmed to evolve consciousness, and to do that, we would need to know what consciousness is and how to program it.
Since consciousness only counts when it is formed deliberately rather than accidentally. Though that would mean…
> The rational soul can be made only by creation.
From Aquinas' Summa Theologiae. The argument this article presents does seem very similar. Weirdly, the article itself is very dismissive of religion, while making a similar argument to the one Aquinas made 750 years ago.
[dead]
[dead]