I imagine this is like how microwaves put restaurants out of business, how TTS obviated literacy, how the automobile age made it unnecessary to walk, how Zoom caused every office to close, how helicopters put mountaineering firmly behind us, Amazon obsoleted brick and mortar stores, and payment cards made cash disappear.
It's not only ahistorical to fail to consider preferences and politics, but it's also an indictment of the author's ability to judge the quality of a machine output versus that of a competent person. No doubt that like any of the examples above, some displacement of resources and effort will happen. But that doesn't indicate that something as central as "thinking" (lol) will fade into the past.
>My challenge ahead is to move from making a living by thinking, to making thinking part of a life worth living.
Our challenge ahead is making a living at all. Who is going to be having hobbies if the economy is 100% automated? We're not going to sleepwalk our way into a post-scarcity automated economy where everyone's needs are provided for and they can just spend all their time on hobbies. If we believe AI can replace all cognitive labor, then it follows that it'll be able to replace all physical labor with robotics fairly quickly, and we're left with no labor left for humans. We need to plan for that future or else billions will suffer from the results.
> it follows that it'll be able to replace all physical labor with robotics fairly quickly
This is not clear at all. Even the most expensive robotic/arms hands perfectly controlled fail to do thing as quickly and easily as human limbs, not to mention the chaotic environment that is the real world. It's clear that it can happen, but the timeline is not quick.
I actually don't think that follows necessarily. The future is lumpy. Some problems will be tractable years or decades ahead of others.
Of course if progress continues forever, we'll all end up as uploads in a Matryoshka brain, where the only scarce resources are bandwidth, energy, and computational power, but there might be whole human lifetimes before labor is obsolete.
I believe we're already past that point, at least in the US. Only 44% of American adults had level 3 literacy (6th-grade level) or higher in 2023 [0], down from 48% in 2017. We are probably near 40% now.
These are not necessarily unintelligent people, but they haven't developed the skills necessary to critically analyze textual information. My opinion is that without those skills it is not possible to meaningfully engage in a democratic system.
Not a rhetorical question. Outside of my family and self, and perhaps a livable planet, liberal democracy is the most important thing I can possibly imagine. I know I'm not alone. So what do we do?
Edit: Replies seem to indicate that people think I meant for this to be about the US. It's not. I'm in a well-functioning European democracy, and my worry definitely applies here too.
As an individual we need to make the case for liberal democracy. Its under heavy attack by two very authoritarian ideologies either side. Liberal Democracy needs to convince people its the right framework going forward once Liberals are back in power they need to demonstrate that they can actually solve the challenges people are facing. And it cant all be left up to politicians voters need to activate to provide energy.
> 2. it will roll all of this shit back and reform ties with Europe in an attempt to play the numbers game
Is alliance with Europe a help or a hinderance in a war with China? Lot more target area, not a lot more offensive capability.
Hasn't thinking already become a hobby for most people, and most delegate their thinking to others?
Think: those in authority figures, for church or politics. Or science or chemistry. And especially in their own health, which should always remain part art.
A few nerds on the internet, like us in hacker news, are still remaining open to question and apply our hard thinking to tough problems. But most people: they've resigned and bought the belief they are not qualified to think about hard problems.
This sounded really like elitist thinking. I don't know what most of the people do but I know my wife is a caveman of we are speaking about technology but if we start taking about law then suddenly I'm half brained chimpanzee with banana in the nose trying figure out why I can drink it
Even my uncle who work as a carperter proved to me to know more about math and physics than I would ever thought physical labourer would know.
There is no way in hell the economy will be 100 percent automated, ever. You gotta spend zero time around non-devs/managers to think that's going to be anywhere near true.
Parts of creative/intellectual labour just happen to inimical to accelerated algorithmic production. They'll produce infinitely and endlessly, uselessly. I can't see there ever not being a need to manage these things, to check, select, edit and refine their output.
And as for the vast majority of labour that doesn't involve writing code/reports/spreadsheets. I'll believe it when I have robots fixing my roof, running a kitchen, taking blood and dressing wounds.
In theory a robot could do some small components of that, maybe. But the cost, compared to a human, would be insane, and the result much worse. That even modestly rolled out in West Yorkshire before I'm long dead. No chance.
---
Edit: Down-voters, I would love to hear your vision for a fully-automated economy without human involvement, preferably this side of armageddon.
I like to imagine walking into colonial Jamestown and trying to explain to them
"it'll be able to replace all physical labor with robotics fairly quickly, and we're left with no labor left for humans. We need to plan for that future or else billions will suffer from the results."
Enclosure was well underway in England, so they probably understood the dilemma of modernization simultaneously increasing total productivity and lowering the living standards for a large population.
"And it and those robots will be owned by people who don't care if you starve or freeze to death."
And unlike those in Jamestown, who could just ignore the robots and continue working the land to survive, we will be forced to allow the robots to take over our roles.
From my understanding of how LLMs work, I'd argue thinking will become the most valuable attribute of humans (as it has been for some time). LLMs will inevitably surpass humans at functions that we know how to do, and likely through cross-domain knowledge and connecting disparate concepts will surpass us at many functions. But LLMs haven't shown they are capable of truly novel thinking, because it's next token prediction, which is inherently based on existing sources.
I think we'll see a shift towards arts, creativity, research, etc. where true novel creation and an understanding of the human experience exists at a premium.
Open to learning. How could RL yield novel discoveries? What I mean to describe are things reliant on empirical or real-world testing of human reception. How would that be synthetically derived?
We're seeing the cost of code drop dramatically and are facing the same problem in building new companies across most disciplines of, "what's a problem painful enough that someone will pay us to solve it?".
I suppose if we get to a massive enough scale we can effectively model every atom's interaction and human on earth's reaction to any stimulus but I think we're still some time off from that :)
Same! I do think a vast majority of tasks are essentially next token prediction, but that's why I'm thinking that long, creative thinking periods will be what's actually valuable/rare.
Some pertinent fragments of wisdom gleaned from other places:
Somewhere I encountered a quote like "the finest art is a life well-lived". This threw me, because, apparently, I had been thinking of art as something different in kind from life, and unrelated to the normal work or act of living. Seen in terms of this quote, art becomes once again of-a-kind with life, and may be seen as only a part of living well, instrumental to living well, or perhaps just as practice for living well.
Something similar seems to be happening with "thinking" and "intelligence". Somehow these things had become ends-in-themselves, separate from the regular act of living. We might wrap them in a similar quote to place them in relation to life, along the lines of "thinking well is for living well", or perhaps "the highest good of the intellect is to be well-fit to the problems of living well".
I am reminded also of a line which stuck with me from Augustine's Confessions, approximately: "The mind is not the supreme good"—as opposed to the body and perhaps the "spirit". Somehow the world I was born into had collectively come to think of the mind as the supreme good, and intelligence as the greatest virtue. But this is, in some way, in contradiction the basic fact of existence and mortality: no amount of cleverness can circumvent death, and all the glory attainable by feats of the intellect are only futile vanities compared to the privilege of being alive itself.
> Something similar seems to be happening with "thinking" and "intelligence". Somehow these things had become ends-in-themselves
It's been that way for a long time. At least since Descartes "Cogito, ergo sum". But even the Greeks were heading that way almost a couple thousand years earlier.
> separate from the regular act of living.
How is thinking separate from the regular act of living for human beings? Our species is known as homo sapiens - the knowing or thinking human.
> I was born into had collectively come to think of the mind as the supreme good, and intelligence as the greatest virtue.
I guess it comes down to what you think the answer to the basic question posed in the catechism is: "What is the chief end of man?" This is going to be the crucial question for us as a species in the age of abundant AI. Or as Wendell Berry posed it: "What are humans for?"
> But this is, in some way, in contradiction the basic fact of existence and mortality: no amount of cleverness can circumvent death, and all the glory attainable by feats of the intellect are only futile vanities compared to the privilege of being alive itself.
But being alive for us humans entails a lot of thinking. Is the unexamined life worth living?
what will it mean to be human, homo sapiens, in a world where the thinking is done for us?
Semi-related; I had an acquaintance who would dreamily declare 'Art is the True Life'; I always disagreed, 'Life is the True Art'. A huge amount of art is basically documenting those among us who are really out there living. I'd rather be on the side of the experiencer than the documenter.
I wonder if feeling and intuition will become the most prized, once explaining and sophistry toward any conclusion becomes something any bot can do...?
Right now, we think of feeling as second class, but I wonder if that's more because we have let it go feral, and left it unprotected from predatory actors and systems -- advertising, consumerism, even politics. We don't protect our minds from attacks on our feeling systems (e.g., we don't cover therapy as a public health measure), and so maybe we've just gotten used to giving a pretty poor assessment of its capacity.
If we institutionalized protection of intuition like we protect our ability to reason, maybe we would think more highly of that part of ourselves...
I feel one could make a similar argument about any technological advancement and subsequent abstraction. A farmer pulling a cultivator with a tractor certainly has less of a “feel” for the details of the land and climate than someone doing the same with a handheld implement, yet cultivating manually is not economically viable at scale.
Not to say there isn’t a loss; there absolutely is. But, for better or worse, it has long been the case that the individual is responsible for balancing the beauty of psychological flow and craftsmanship against the literal economic output expected of them.
I think posts like this that frame this issue as exclusively a product of the rise of AI are missing the more fundamental piece: we use the tools, and in turn the tools change us. Just as we are what we eat, we are the tools we use. For better or worse.
I was vibe coding a prototype some nights ago. In between, I ran out of my AI budget.
I felt stuck. I didn’t know what the AI had done so far or how to continue. There was nobody to message for a budget increase (it was night).
Luckily I managed to get more budget.
But that left me scared for the future. One day something is gonna fail, and nobody can do anything because the AI providers are also down. Or what if I can no longer afford paying for the frontier?
I have made some modifications to my vibe coding practices since then. I don’t want to live in this future that the author describes.
If it's any consolation, I stay on top of all changes, but when the AI API stops I feel just as stuck.
I understand what I made, the speed just drops to 0.1%. So I just start studying the codebase, planning work for later, ordering thoughts by writing DESIGN.md, adding jobs to the kanban.
It scares me that I don't own the LLM though, but someday I will, it will be local and I'll feel better when I own my means of production again.
Whilst I agree that AI costs are falling and AIs have practically infinite uptime, I fail to see how/why the current AIs will perform better overall in the work place.
The example of AIs solving mathematics is all fun and games, but it assumes that mastery of mathematics implies that everything else will also be mastered eventually. But instead we're seeing that LLMs exhibit this thing called jagged intelligence, which means that they can be extremely good in one area, but completely fail in another. In theory the LLMs could be improved if data would be available for other areas and if there are good verifiability criteria. Fact is that many fields just don't have those and would not be willing to give up their data.
Daily front page psyop telling you to accept that token predictors are ASI and humans are obsolete, despite the fact that if they were ASI nobody would ever need to write a blog post convincing you of it
I agree that’s what happened to hunting and riding (they became hobbies). But it’s hard to imagine something as broad as thinking will go the same way. Not saying it won’t, just that it’s hard to imagine and that the implications will be far more far reaching.
> Machines transformed much of physical activity from necessity into choice; similarly, AI will decouple human thinking from its economic purpose.
This could not be more decoupled from the truth.
Yes, industrialization automated a lot of physical activity but I think the author severely underestimates the value that blue collar workers produce to this day. At least this is how I perceive it.
I would not place a lot of emphasis on economic insights from software engineering academics tbh.
That depends on the country. There are a bunch of places where you get paid pretty well to do a PhD. Sure, if you consider the opportunity cost of not being a senior software engineer in San Francisco making 300k it's not a lot, but by that logic every other job is a hobby.
Alternative proposal: Since no one reasonable actually wants a future where humans are irrelevant - including the people in power who generally like being relevant - we could remember that we’re thinking beings with agency and just decide not to use AI to replace ourselves.
We made a new tool. Using the tool for some things makes life better. Using the tool too much would make life worse. So let’s not do that!
IDK, I'm thinking more than ever. I used to be in a flow, writing code, now, I'm thinking at the top of my lungs, making a new decision every 10 sec, Claude thinks through all the scenarios, makes proposals for many data models, RBAC rules etc, and I'm just focusing on understanding it all and deciding (and it can be pretty exhausting!). More thinking than ever. Am I doing it wrong? Should I just think less? Rely on the spec more?
I do the same and I was just coming home the other day after doing a massive system design session with AI and I was probably the most mentally exhausted I’ve ever been in my career. But I guess the thought is, “is that necessary?” Could I have just yolo let Claude take the wheel. I don’t know the answer. I know at the end of the day I’m accountable for this output so I feel a professional responsibility to understand what’s going on. Maybe that sense of responsibility is holding me back but I really don’t know.
I feel the same. In 2 days I meet the people that gave me this assignment, they have questions, they will look at the visual I made of the DB layout and the RBAC table, and they will say some stuff is wrong, or ask why I deviated from their proposal.
And no-way am I going to ask them to wait as I relay my questions to Claude.
So I stay on top of things. I tell them: "I simplified the RBAC because these two roles here are functionally the same and, I noticed there were two axes underneath, scope and trust which I encoded here and here..." etc. Yes, a lot of it is Opus' findings, but along the way I also stopped Claude from over-complicating things at several stages in the development.
I do also feel that I'm slowly developing this way of working, perhaps I'll learn to lean into it more, perhaps I will get better at it. Perhaps it's just exhausting for now and I can handle more and more agents as I do it more and more. That's what I hope at least, to become a one-stop software shop by employing agents, all while being uncompromising on quality. What choice do we have?
If you have responsibility for things, you want to make the best choices. At some level, this involves thinking. It is responsibility that should not be delegated to AI. The thinking required of the responsible human changes, but is not eliminated.
My initial knee-jerk reaction was to list out a whole bunch of reasons why this take is wrong, or shortsighted, or gives too much credit to LLMs as a “revolution”, or any of the other go-tos in my satchel of counterpoints to the usual ad nauseam AI discourse.
Except those don’t really fit, here, because the author is making a far more critical error in ascribing far too much to current human cognition. On the daily I see so-called “knowledge workers” failing to read basic signs, refusing to learn basic skills, and all too eager to cast off the slightest bit of critical thinking to a black box they can subscribe to - be it replacing curating a content library with streaming services and algorithms, or swapping a life of hard-won expertise for a swarm of chatbots and an API bucket. As someone with the uniquely privileged POV to be able to ingest vast sums of technical data but struggle with social data, I look at my fellow Americans and genuinely fret for what the next few decades hold for a cohort that seemingly cannot effectively communicate which direction they’re going but is all too content to hitch their accountability to an LLM they pay for.
In that sense, thinking has already become a hobby to the masses, in that it’s entirely optional to exist in current society at large. Autopay apps manage your bills, finance apps keep you on budget, fitness apps tell you to keep running, social media apps tell you who your friends (and enemies) are, video apps tell you what to watch, music apps tell you what to listen to, and cars increasingly tell you not just where to go but also to let them get there for you. I would argue that in that context, thinking is a critically endangered hobby we sorely need to sow new seeds for, lest we risk losing it entirely; AI does not seem keen to do that.
Now to be fair, so much of this is hyperspecialization knowledge overflowing into daily life and making existence a sort of cognitohazard in and of itself. We skip through hundreds of pages of legal boilerplate because we’re not lawyers, and then complain about all the rights we willingly signed away because of the volume of said agreements. Non-technical people engaging with modern computing are either addicts or miserable, unaware that this was both entirely optional and also inevitable in a populace that didn’t think for itself. Finance has become so hyperspecialized that it’s impossible for the layman to tell of something is good or bad, or even why the number went up or down on their portfolio in a given time period. LLM boosters say their tools will solve this complexity nightmare, yet the tools themselves grow increasingly complex just to eke out single-point gains on hyperspecific problems.
Which brings me to the OP again, their highly uncritical claims that AI will bring about a new revolution of thought itself, and that somehow this tool will both restore thought to an elevated pedestal while also making it nearly impossible to meter (at least on American currency; let’s not think too hard about how developing countries might be able to afford it). I don’t see any of that, but for the same reasons the OP feels it’s somehow inevitable. The only difference I can readily figure is our opinions of our fellow humans: I view us as in dire straits and able to do far better than we give ourselves credit for, while they view us as replaceable with servile digital labor for the better.
"I hate this damn machine"
"I wish they would just sell"
"It never does what I want"
"Only what I tell it."
I've yet to be convinced we will evolve these systems to a point where we can just click the "Magic Button" thoughtlessly and get results we are satisfied with.
We may have reduced the loss of data between thought and action, but nothing suggests these systems will be able to intuit our every want and desire.
In other news, research institute such as S&P, McKinsey and MIT that try to estimate when level 5 autonomous vehicles will be achieved, all agree it won't happen before 2040.
Simple example, close to home: the preponderance of proprietary products rather than freer ones, with people then mass complaining when they end up bitten by the downsides. But adopting the freer solutions requires taking it on as a hobby - it's not just a simple choice between A or B, but rather requires self-actualization to implement the freer solution as it pertains to your own life.
So we're really talking about the next step whereby it becomes even easier to just go with the flow of default options. Of course, "the flow" might not be ending up in such a successful place any more (ie no more copious white collar make work jobs in the healthcare denial industry, etc), and that's the real problem.
Such a wild take to argue that current tier LLMs do any “thinking” at all. Was chatting with a friend last week and I was saying that sure AI writes much of my code these days but my cognitive load is HIGHER, as I have to constantly make sure it’s not doing something dumb.
I have to constantly make sure it’s building things in a way that will fit neatly together, in a way doesn’t over (or under) -generalize. In a way that doesn’t waste time and energy on idiotic tests that aren’t really testing anything at all.
LLMs are powerful autocomplete with vast and broad memory, they do not think. And thinking is just a part of the human condition.
I fail to understand, in its current state, the mass hysteria around this revolution. Maybe I'm doing this wrong, but I constantly need to review and manage my agents, are we really headed to a reality where engineering mindsets are no longer needed?
I think it’s just part of the zeitgeist. Like, I’m actually pretty AI-forward, but my post is getting downvoted. Not that I care, it’s just an interesting sign of the times.
By using the models you're training them to be better at these tasks, it's where a good number of improvements have already come from. Frontier labs just feed it back in.
> LLMs are powerful autocomplete with vast and broad memory, they do not think.
This is really only true insofar as they are designed such that they output language tokens and their intermediate "chain of thought" is also done in language tokens, but now that's really only done for auditability. There are much more abstract layers before the model output layer of a single token prediciton and they could be designed to to "think" using these more abstract layers as intermediates instead like how many people think in concepts rather than words. There are even some research models on this (see Next Latent Models)[1]. Humans need to communicate in language because we don't have telepathic abilities but these models need not have such a constraint.
Calculators have overflow and underflow conditions, among others, that don't mathematically exist. Fail to understand the limitations of your abstractions (which is what the AI bros who keep bring up calculator argument are demonstrating) and you will be screwed when the abstraction breaks down.
> Such a wild take to argue that current tier LLMs do any “thinking” at all.
It does not matter whether LLMs do any thinking, or "thinking." The only thing that matters is whether they produce similar results to human thinking, even if it's through a different process (or no process at all). When a company decides to deploy an AI-powered customer service representative, it is not because the AI bot goes through the same thinking process as a human when answering a customer's request.
Suppose they're unable to innovate, but their enormity and their parasitism of human creativity makes up for it. This makes their output similar in the short term but crucially different in the long term. Things don't function through "no process at all"; things have ways in which they work. The process is relevant to the output, and gives insights into subtle but crucial differences in output. By knowing about the process we can avoid being deceived by apparent similarity.
For anyone like me who started reading science fictions as a kid, that story is a very old hat. So far, AI only seems to solve problems that you tell it to solve, and you have to carefully guide it to get results. So for now the claim seems exaggerated and still in the realm of science fiction.
By the way, there is another problem. For humans we have the saying "garbage in, garbage out." If you read a lot of trash, you will only come up with trash. If you give powerful tools to morons, they will not become geniuses. That's not just true for humans, it's also the case for AI. If you teach your AI trash, it's going to be some trashy AI. Since people do that all the time for various reasons, we're going to have a lot of artificial stupidity, too.
There is nothing that guarantees they will exist perpetually. In even a slightly moderate global crisis (man made or otherwise) LLMs and the like will probably one of the first things to go.
I have pretty heavily dumbified iPhone just for the camera and music. I wish there was a dumbphone with a decent camera with telephoto. I have standalone camera but with a decent lens it's just too big to have on me at all times and unfortunately I like taking photos.
Base install of Graphene on Pixel 10 is pretty close to dumb-phone (I speak from experience). Just get the APKs of Signal and Google Cam (or just use Graphene's cam app, it's not bad, supposedly). Pixel 10 has 3 camera, wide/normal/tele. Pretty decent imho.
I imagine this is like how microwaves put restaurants out of business, how TTS obviated literacy, how the automobile age made it unnecessary to walk, how Zoom caused every office to close, how helicopters put mountaineering firmly behind us, Amazon obsoleted brick and mortar stores, and payment cards made cash disappear.
It's not only ahistorical to fail to consider preferences and politics, but it's also an indictment of the author's ability to judge the quality of a machine output versus that of a competent person. No doubt that like any of the examples above, some displacement of resources and effort will happen. But that doesn't indicate that something as central as "thinking" (lol) will fade into the past.
>My challenge ahead is to move from making a living by thinking, to making thinking part of a life worth living.
Our challenge ahead is making a living at all. Who is going to be having hobbies if the economy is 100% automated? We're not going to sleepwalk our way into a post-scarcity automated economy where everyone's needs are provided for and they can just spend all their time on hobbies. If we believe AI can replace all cognitive labor, then it follows that it'll be able to replace all physical labor with robotics fairly quickly, and we're left with no labor left for humans. We need to plan for that future or else billions will suffer from the results.
> it follows that it'll be able to replace all physical labor with robotics fairly quickly
This is not clear at all. Even the most expensive robotic/arms hands perfectly controlled fail to do thing as quickly and easily as human limbs, not to mention the chaotic environment that is the real world. It's clear that it can happen, but the timeline is not quick.
I actually don't think that follows necessarily. The future is lumpy. Some problems will be tractable years or decades ahead of others.
Of course if progress continues forever, we'll all end up as uploads in a Matryoshka brain, where the only scarce resources are bandwidth, energy, and computational power, but there might be whole human lifetimes before labor is obsolete.
At least thinking is a cheap hobby.
Not if food is expensive.
As log as your don't desire a house to do the thinking in ;)
There's a fictional take on this which is worth looking at:
https://karmak.org/archive/2003/12/manna/manna1.htm
Even if we were to figure out the economic side: if thinking becomes a hobby, we're fucked. Democracy won't be sustainable.
I believe we're already past that point, at least in the US. Only 44% of American adults had level 3 literacy (6th-grade level) or higher in 2023 [0], down from 48% in 2017. We are probably near 40% now.
These are not necessarily unintelligent people, but they haven't developed the skills necessary to critically analyze textual information. My opinion is that without those skills it is not possible to meaningfully engage in a democratic system.
[0] https://nces.ed.gov/surveys/piaac/2023/national_results.asp
The survey in 2017 had a 56% response rate. The 2023 survey had a 27% response rate.
Comparing those 2 datasets is meaningless.
So what do we do?
Not a rhetorical question. Outside of my family and self, and perhaps a livable planet, liberal democracy is the most important thing I can possibly imagine. I know I'm not alone. So what do we do?
Edit: Replies seem to indicate that people think I meant for this to be about the US. It's not. I'm in a well-functioning European democracy, and my worry definitely applies here too.
As an individual we need to make the case for liberal democracy. Its under heavy attack by two very authoritarian ideologies either side. Liberal Democracy needs to convince people its the right framework going forward once Liberals are back in power they need to demonstrate that they can actually solve the challenges people are facing. And it cant all be left up to politicians voters need to activate to provide energy.
> once Liberals are back in power they need to demonstrate that they can actually solve the challenges people are facing
My big fear is that too many people will just go "phew, glad that terrible epoch is finally over, never to return again" :-(
Calm down.
Unless the US plans to unconditionally surrender to China, one of two things will happen:
1. it will start investing into education in an effort to compete by itself
2. it will roll all of this shit back and reform ties with Europe in an attempt to play the numbers game
> 2. it will roll all of this shit back and reform ties with Europe in an attempt to play the numbers game Is alliance with Europe a help or a hinderance in a war with China? Lot more target area, not a lot more offensive capability.
Thinking was a hobby in the ancient Athens wasn't it?
Not just a hobby, which is what is implied here.
As has been pointed out already -we're well past that point.
Hasn't thinking already become a hobby for most people, and most delegate their thinking to others?
Think: those in authority figures, for church or politics. Or science or chemistry. And especially in their own health, which should always remain part art. A few nerds on the internet, like us in hacker news, are still remaining open to question and apply our hard thinking to tough problems. But most people: they've resigned and bought the belief they are not qualified to think about hard problems.
This sounded really like elitist thinking. I don't know what most of the people do but I know my wife is a caveman of we are speaking about technology but if we start taking about law then suddenly I'm half brained chimpanzee with banana in the nose trying figure out why I can drink it
Even my uncle who work as a carperter proved to me to know more about math and physics than I would ever thought physical labourer would know.
Probably. If the remaining who think go too, we're done for.
My guy/gal, have you seen TikTok lately?
There is no way in hell the economy will be 100 percent automated, ever. You gotta spend zero time around non-devs/managers to think that's going to be anywhere near true.
Parts of creative/intellectual labour just happen to inimical to accelerated algorithmic production. They'll produce infinitely and endlessly, uselessly. I can't see there ever not being a need to manage these things, to check, select, edit and refine their output.
And as for the vast majority of labour that doesn't involve writing code/reports/spreadsheets. I'll believe it when I have robots fixing my roof, running a kitchen, taking blood and dressing wounds.
In theory a robot could do some small components of that, maybe. But the cost, compared to a human, would be insane, and the result much worse. That even modestly rolled out in West Yorkshire before I'm long dead. No chance.
---
Edit: Down-voters, I would love to hear your vision for a fully-automated economy without human involvement, preferably this side of armageddon.
I like to imagine walking into colonial Jamestown and trying to explain to them
"it'll be able to replace all physical labor with robotics fairly quickly, and we're left with no labor left for humans. We need to plan for that future or else billions will suffer from the results."
Enclosure was well underway in England, so they probably understood the dilemma of modernization simultaneously increasing total productivity and lowering the living standards for a large population.
"And it and those robots will be owned by people who don't care if you starve or freeze to death."
And unlike those in Jamestown, who could just ignore the robots and continue working the land to survive, we will be forced to allow the robots to take over our roles.
Sometimes I wonder how many people miss doing math by hand instead of using a calculator.
Historically, improvements in tech allowed us to focus on higher level reasoning. For example, spend more time on equations instead of calculations.
So if AI can replace all your reasoning, maybe its time to level up?
From my understanding of how LLMs work, I'd argue thinking will become the most valuable attribute of humans (as it has been for some time). LLMs will inevitably surpass humans at functions that we know how to do, and likely through cross-domain knowledge and connecting disparate concepts will surpass us at many functions. But LLMs haven't shown they are capable of truly novel thinking, because it's next token prediction, which is inherently based on existing sources.
I think we'll see a shift towards arts, creativity, research, etc. where true novel creation and an understanding of the human experience exists at a premium.
You think the thinking of human brains isn't based on existing sources? Unless you believe in dualism and the divine spark of inspiration, it is.
> which is inherently based on existing sources
Look at how compute share of labs is moving to RL and maybe reconsider this prerequisite to your argument.
Open to learning. How could RL yield novel discoveries? What I mean to describe are things reliant on empirical or real-world testing of human reception. How would that be synthetically derived?
We're seeing the cost of code drop dramatically and are facing the same problem in building new companies across most disciplines of, "what's a problem painful enough that someone will pay us to solve it?".
I suppose if we get to a massive enough scale we can effectively model every atom's interaction and human on earth's reaction to any stimulus but I think we're still some time off from that :)
I see your point itll be very jagged but llms are definitely very innovative. See the recent .ath proofs etc.
But i agree thinking will remain important. As my grandpa put it if something breaks who'll go and fix it ?
Quite often I feel like my mind is pretty much doing next token prediction also. Unfortunately.
Same! I do think a vast majority of tasks are essentially next token prediction, but that's why I'm thinking that long, creative thinking periods will be what's actually valuable/rare.
Trivial to prove false: ask a LLM to invent 10 new words. Then google them, most will not have any result.
> My favorite is Krezmoria - a word for missing a future you haven't experienced.
You had to prompt it to do that before it considered it, though.
depends on what you consider novel thinking.
Some pertinent fragments of wisdom gleaned from other places:
Somewhere I encountered a quote like "the finest art is a life well-lived". This threw me, because, apparently, I had been thinking of art as something different in kind from life, and unrelated to the normal work or act of living. Seen in terms of this quote, art becomes once again of-a-kind with life, and may be seen as only a part of living well, instrumental to living well, or perhaps just as practice for living well.
Something similar seems to be happening with "thinking" and "intelligence". Somehow these things had become ends-in-themselves, separate from the regular act of living. We might wrap them in a similar quote to place them in relation to life, along the lines of "thinking well is for living well", or perhaps "the highest good of the intellect is to be well-fit to the problems of living well".
I am reminded also of a line which stuck with me from Augustine's Confessions, approximately: "The mind is not the supreme good"—as opposed to the body and perhaps the "spirit". Somehow the world I was born into had collectively come to think of the mind as the supreme good, and intelligence as the greatest virtue. But this is, in some way, in contradiction the basic fact of existence and mortality: no amount of cleverness can circumvent death, and all the glory attainable by feats of the intellect are only futile vanities compared to the privilege of being alive itself.
That kind of thing.
> Something similar seems to be happening with "thinking" and "intelligence". Somehow these things had become ends-in-themselves
It's been that way for a long time. At least since Descartes "Cogito, ergo sum". But even the Greeks were heading that way almost a couple thousand years earlier.
> separate from the regular act of living.
How is thinking separate from the regular act of living for human beings? Our species is known as homo sapiens - the knowing or thinking human.
> I was born into had collectively come to think of the mind as the supreme good, and intelligence as the greatest virtue.
I guess it comes down to what you think the answer to the basic question posed in the catechism is: "What is the chief end of man?" This is going to be the crucial question for us as a species in the age of abundant AI. Or as Wendell Berry posed it: "What are humans for?"
> But this is, in some way, in contradiction the basic fact of existence and mortality: no amount of cleverness can circumvent death, and all the glory attainable by feats of the intellect are only futile vanities compared to the privilege of being alive itself.
But being alive for us humans entails a lot of thinking. Is the unexamined life worth living?
what will it mean to be human, homo sapiens, in a world where the thinking is done for us?
Semi-related; I had an acquaintance who would dreamily declare 'Art is the True Life'; I always disagreed, 'Life is the True Art'. A huge amount of art is basically documenting those among us who are really out there living. I'd rather be on the side of the experiencer than the documenter.
"I put all my genius into my life; I put only my talent into my works." Oscar Wilde.
This is a very negative take. We didn’t stop learning arithmetics after the invention of the calculator.
Reminds me of the Isaac Asimov short story "The Feeling of Power"[1].
[1]: https://lecturia.org/en/short-stories/isaac-asimov-the-feeli...
I wonder if feeling and intuition will become the most prized, once explaining and sophistry toward any conclusion becomes something any bot can do...?
Right now, we think of feeling as second class, but I wonder if that's more because we have let it go feral, and left it unprotected from predatory actors and systems -- advertising, consumerism, even politics. We don't protect our minds from attacks on our feeling systems (e.g., we don't cover therapy as a public health measure), and so maybe we've just gotten used to giving a pretty poor assessment of its capacity.
If we institutionalized protection of intuition like we protect our ability to reason, maybe we would think more highly of that part of ourselves...
Intuition is just latent data processing, can be done by machines.
Feeling is a steering of the processing, I don't really see how it will help you accumulate power, resources or status compared to just processing.
I feel one could make a similar argument about any technological advancement and subsequent abstraction. A farmer pulling a cultivator with a tractor certainly has less of a “feel” for the details of the land and climate than someone doing the same with a handheld implement, yet cultivating manually is not economically viable at scale.
Not to say there isn’t a loss; there absolutely is. But, for better or worse, it has long been the case that the individual is responsible for balancing the beauty of psychological flow and craftsmanship against the literal economic output expected of them.
I think posts like this that frame this issue as exclusively a product of the rise of AI are missing the more fundamental piece: we use the tools, and in turn the tools change us. Just as we are what we eat, we are the tools we use. For better or worse.
I was vibe coding a prototype some nights ago. In between, I ran out of my AI budget.
I felt stuck. I didn’t know what the AI had done so far or how to continue. There was nobody to message for a budget increase (it was night).
Luckily I managed to get more budget.
But that left me scared for the future. One day something is gonna fail, and nobody can do anything because the AI providers are also down. Or what if I can no longer afford paying for the frontier?
I have made some modifications to my vibe coding practices since then. I don’t want to live in this future that the author describes.
If it's any consolation, I stay on top of all changes, but when the AI API stops I feel just as stuck.
I understand what I made, the speed just drops to 0.1%. So I just start studying the codebase, planning work for later, ordering thoughts by writing DESIGN.md, adding jobs to the kanban.
It scares me that I don't own the LLM though, but someday I will, it will be local and I'll feel better when I own my means of production again.
Become your own AI provider? At least until Anthropic et al manage to get local AI declared illegal. :)
Whilst I agree that AI costs are falling and AIs have practically infinite uptime, I fail to see how/why the current AIs will perform better overall in the work place.
The example of AIs solving mathematics is all fun and games, but it assumes that mastery of mathematics implies that everything else will also be mastered eventually. But instead we're seeing that LLMs exhibit this thing called jagged intelligence, which means that they can be extremely good in one area, but completely fail in another. In theory the LLMs could be improved if data would be available for other areas and if there are good verifiability criteria. Fact is that many fields just don't have those and would not be willing to give up their data.
Everyone is always so negative. Cheer up a bit. The "bicycle of the mind" is now the "dirtbike of the mind"... have fun, pop a wheelie.
Observing some colleagues, more like "wheelchair for the mind"
... thinking is how I feed my child. Telling me to have fun is such a tasteless and tactless suggestion
Daily front page psyop telling you to accept that token predictors are ASI and humans are obsolete, despite the fact that if they were ASI nobody would ever need to write a blog post convincing you of it
People think every day, every second, where for most male, half the time it will be about sexual activities.
The author is not talking about random thoughts more like "deep thoughts" or "Reflections.
Reflection must be more like a trained "habit" than a "hobby".
I agree that’s what happened to hunting and riding (they became hobbies). But it’s hard to imagine something as broad as thinking will go the same way. Not saying it won’t, just that it’s hard to imagine and that the implications will be far more far reaching.
> Machines transformed much of physical activity from necessity into choice; similarly, AI will decouple human thinking from its economic purpose.
This could not be more decoupled from the truth.
Yes, industrialization automated a lot of physical activity but I think the author severely underestimates the value that blue collar workers produce to this day. At least this is how I perceive it.
I would not place a lot of emphasis on economic insights from software engineering academics tbh.
It’s already a hobby. Look at what it costs a person to do a PhD. The opportunity cost is even higher if you want to do research as a career.
That depends on the country. There are a bunch of places where you get paid pretty well to do a PhD. Sure, if you consider the opportunity cost of not being a senior software engineer in San Francisco making 300k it's not a lot, but by that logic every other job is a hobby.
Always has been, if you think about it.
Alternative proposal: Since no one reasonable actually wants a future where humans are irrelevant - including the people in power who generally like being relevant - we could remember that we’re thinking beings with agency and just decide not to use AI to replace ourselves.
We made a new tool. Using the tool for some things makes life better. Using the tool too much would make life worse. So let’s not do that!
I'd be quite surprised if we could "just" decide to do that given the economic incentives
IDK, I'm thinking more than ever. I used to be in a flow, writing code, now, I'm thinking at the top of my lungs, making a new decision every 10 sec, Claude thinks through all the scenarios, makes proposals for many data models, RBAC rules etc, and I'm just focusing on understanding it all and deciding (and it can be pretty exhausting!). More thinking than ever. Am I doing it wrong? Should I just think less? Rely on the spec more?
I do the same and I was just coming home the other day after doing a massive system design session with AI and I was probably the most mentally exhausted I’ve ever been in my career. But I guess the thought is, “is that necessary?” Could I have just yolo let Claude take the wheel. I don’t know the answer. I know at the end of the day I’m accountable for this output so I feel a professional responsibility to understand what’s going on. Maybe that sense of responsibility is holding me back but I really don’t know.
I feel the same. In 2 days I meet the people that gave me this assignment, they have questions, they will look at the visual I made of the DB layout and the RBAC table, and they will say some stuff is wrong, or ask why I deviated from their proposal.
And no-way am I going to ask them to wait as I relay my questions to Claude.
So I stay on top of things. I tell them: "I simplified the RBAC because these two roles here are functionally the same and, I noticed there were two axes underneath, scope and trust which I encoded here and here..." etc. Yes, a lot of it is Opus' findings, but along the way I also stopped Claude from over-complicating things at several stages in the development.
I do also feel that I'm slowly developing this way of working, perhaps I'll learn to lean into it more, perhaps I will get better at it. Perhaps it's just exhausting for now and I can handle more and more agents as I do it more and more. That's what I hope at least, to become a one-stop software shop by employing agents, all while being uncompromising on quality. What choice do we have?
If you have responsibility for things, you want to make the best choices. At some level, this involves thinking. It is responsibility that should not be delegated to AI. The thinking required of the responsible human changes, but is not eliminated.
For most people, thinking was always a hobby. AI is actually doing them good.
Well, this is horrifying.
But a the conclusion is a nice one for people that will be able to pull it off.
My initial knee-jerk reaction was to list out a whole bunch of reasons why this take is wrong, or shortsighted, or gives too much credit to LLMs as a “revolution”, or any of the other go-tos in my satchel of counterpoints to the usual ad nauseam AI discourse.
Except those don’t really fit, here, because the author is making a far more critical error in ascribing far too much to current human cognition. On the daily I see so-called “knowledge workers” failing to read basic signs, refusing to learn basic skills, and all too eager to cast off the slightest bit of critical thinking to a black box they can subscribe to - be it replacing curating a content library with streaming services and algorithms, or swapping a life of hard-won expertise for a swarm of chatbots and an API bucket. As someone with the uniquely privileged POV to be able to ingest vast sums of technical data but struggle with social data, I look at my fellow Americans and genuinely fret for what the next few decades hold for a cohort that seemingly cannot effectively communicate which direction they’re going but is all too content to hitch their accountability to an LLM they pay for.
In that sense, thinking has already become a hobby to the masses, in that it’s entirely optional to exist in current society at large. Autopay apps manage your bills, finance apps keep you on budget, fitness apps tell you to keep running, social media apps tell you who your friends (and enemies) are, video apps tell you what to watch, music apps tell you what to listen to, and cars increasingly tell you not just where to go but also to let them get there for you. I would argue that in that context, thinking is a critically endangered hobby we sorely need to sow new seeds for, lest we risk losing it entirely; AI does not seem keen to do that.
Now to be fair, so much of this is hyperspecialization knowledge overflowing into daily life and making existence a sort of cognitohazard in and of itself. We skip through hundreds of pages of legal boilerplate because we’re not lawyers, and then complain about all the rights we willingly signed away because of the volume of said agreements. Non-technical people engaging with modern computing are either addicts or miserable, unaware that this was both entirely optional and also inevitable in a populace that didn’t think for itself. Finance has become so hyperspecialized that it’s impossible for the layman to tell of something is good or bad, or even why the number went up or down on their portfolio in a given time period. LLM boosters say their tools will solve this complexity nightmare, yet the tools themselves grow increasingly complex just to eke out single-point gains on hyperspecific problems.
Which brings me to the OP again, their highly uncritical claims that AI will bring about a new revolution of thought itself, and that somehow this tool will both restore thought to an elevated pedestal while also making it nearly impossible to meter (at least on American currency; let’s not think too hard about how developing countries might be able to afford it). I don’t see any of that, but for the same reasons the OP feels it’s somehow inevitable. The only difference I can readily figure is our opinions of our fellow humans: I view us as in dire straits and able to do far better than we give ourselves credit for, while they view us as replaceable with servile digital labor for the better.
In other news, research institute such as S&P, McKinsey and MIT that try to estimate when level 5 autonomous vehicles will be achieved, all agree it won't happen before 2040.
Hasn't it been for quite some time?
Simple example, close to home: the preponderance of proprietary products rather than freer ones, with people then mass complaining when they end up bitten by the downsides. But adopting the freer solutions requires taking it on as a hobby - it's not just a simple choice between A or B, but rather requires self-actualization to implement the freer solution as it pertains to your own life.
So we're really talking about the next step whereby it becomes even easier to just go with the flow of default options. Of course, "the flow" might not be ending up in such a successful place any more (ie no more copious white collar make work jobs in the healthcare denial industry, etc), and that's the real problem.
Such a wild take to argue that current tier LLMs do any “thinking” at all. Was chatting with a friend last week and I was saying that sure AI writes much of my code these days but my cognitive load is HIGHER, as I have to constantly make sure it’s not doing something dumb.
I have to constantly make sure it’s building things in a way that will fit neatly together, in a way doesn’t over (or under) -generalize. In a way that doesn’t waste time and energy on idiotic tests that aren’t really testing anything at all.
LLMs are powerful autocomplete with vast and broad memory, they do not think. And thinking is just a part of the human condition.
I fail to understand, in its current state, the mass hysteria around this revolution. Maybe I'm doing this wrong, but I constantly need to review and manage my agents, are we really headed to a reality where engineering mindsets are no longer needed?
Even if they did exactly what you wanted, you need to want something. To decide what you want, the thinking shifts from how and when to what and why.
Can you not look back a year or even 6 months to see the progress that has been made? The pace of change is incredible.
How does that not lead you to same conclusion that everyone else is having? That this is only going to improve.
It's so shortsighted to make an assessment only on what exists today.
It's equally fallacious to assume that the curve continues. It may well be a sigmoid-like progress curve.
I think it’s just part of the zeitgeist. Like, I’m actually pretty AI-forward, but my post is getting downvoted. Not that I care, it’s just an interesting sign of the times.
By using the models you're training them to be better at these tasks, it's where a good number of improvements have already come from. Frontier labs just feed it back in.
> LLMs are powerful autocomplete with vast and broad memory, they do not think.
This is really only true insofar as they are designed such that they output language tokens and their intermediate "chain of thought" is also done in language tokens, but now that's really only done for auditability. There are much more abstract layers before the model output layer of a single token prediciton and they could be designed to to "think" using these more abstract layers as intermediates instead like how many people think in concepts rather than words. There are even some research models on this (see Next Latent Models)[1]. Humans need to communicate in language because we don't have telepathic abilities but these models need not have such a constraint.
[1] https://arxiv.org/abs/2511.05963
over time you'll need to check it less and less. do we still check if calculators compute large sums properly?
Are calculators now or have they ever been non-deterministic?
Calculators don’t do any creative work, building products is creative. Apples and oranges.
Calculators have overflow and underflow conditions, among others, that don't mathematically exist. Fail to understand the limitations of your abstractions (which is what the AI bros who keep bring up calculator argument are demonstrating) and you will be screwed when the abstraction breaks down.
> Such a wild take to argue that current tier LLMs do any “thinking” at all.
It does not matter whether LLMs do any thinking, or "thinking." The only thing that matters is whether they produce similar results to human thinking, even if it's through a different process (or no process at all). When a company decides to deploy an AI-powered customer service representative, it is not because the AI bot goes through the same thinking process as a human when answering a customer's request.
Suppose they're unable to innovate, but their enormity and their parasitism of human creativity makes up for it. This makes their output similar in the short term but crucially different in the long term. Things don't function through "no process at all"; things have ways in which they work. The process is relevant to the output, and gives insights into subtle but crucial differences in output. By knowing about the process we can avoid being deceived by apparent similarity.
The bigger shock is that programming and solving Millenium Prize math problems requires zero thinking.
For anyone like me who started reading science fictions as a kid, that story is a very old hat. So far, AI only seems to solve problems that you tell it to solve, and you have to carefully guide it to get results. So for now the claim seems exaggerated and still in the realm of science fiction.
By the way, there is another problem. For humans we have the saying "garbage in, garbage out." If you read a lot of trash, you will only come up with trash. If you give powerful tools to morons, they will not become geniuses. That's not just true for humans, it's also the case for AI. If you teach your AI trash, it's going to be some trashy AI. Since people do that all the time for various reasons, we're going to have a lot of artificial stupidity, too.
Spinellis also engages in doomerist reinforcement learning for humans now? This is exactly up Amodei's alley!
Thank, you, Diomidis! I'll remember your IOCCC contributions and shall join the AI-Jugend and report to my Untergruppenführer.
(Incidentally, the IOCCC is also suffering from AI psychosis now.)
How many AI companies are debt free and make profit?
This is the we-don’t-need-BluRays-because-all-movies-are-on-Netflix phase.
Let’s wait until the market is settled
I wish LLM never existed. At this point I wish smartphones didn't either.
There is nothing that guarantees they will exist perpetually. In even a slightly moderate global crisis (man made or otherwise) LLMs and the like will probably one of the first things to go.
Get rid of your smartphone! It did it eight months ago, no regrets.
I have pretty heavily dumbified iPhone just for the camera and music. I wish there was a dumbphone with a decent camera with telephoto. I have standalone camera but with a decent lens it's just too big to have on me at all times and unfortunately I like taking photos.
Base install of Graphene on Pixel 10 is pretty close to dumb-phone (I speak from experience). Just get the APKs of Signal and Google Cam (or just use Graphene's cam app, it's not bad, supposedly). Pixel 10 has 3 camera, wide/normal/tele. Pretty decent imho.
I wish that fish never stepped out of the water. Thanks for nothing, Tiktaalik.