I don't know if this is a good framing. "Too much" is subjective, and every heavy AI user will assert that they're just unlocking their potential, that calculators didn't make us dumber, etc.
But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you. On the flip side, if you use an LLM do most of your thinking, what's left? We have people here who use LLMs to raise their children, to manage relationships, to design products. So what's your unique contribution to this world - is it the prompt you once wrote? You're standing in front of a token-generating machine, pulling a lever, sometimes receiving gifts. Is that your edge, your unique experience, your purpose in life?
Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?
In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share. I want to support that. And I want to be a human who can write novels, the old-fashioned way. I'm not good at lifting weights or running, so my thinking is the only thing I have.
Have you read the "Whispering earring" essay? I love it for the LLM era.[1]
You can treat AI as a whispering earring - "What should we do now? How do we fix this? What do you think?" Or you can treat it like an exoskelton - "Implement kd-tree with metric space xyz for this problem, mapping this to that blah blah".
That's pre-thought execution automation that makes review much simpler - you already know the shape of the desired output. The whispering earring is atrophy.
Something I never considered much is what happens when everyone else is using the Whispering Earring.
You may be more free and independent, but you may also be unable to compete as everyone else easily gains wealth and success. Natural selection doesn't particularly care about freedom of consciousness.
Eventually, that becomes interesting. Before we get to that stage, I think we pass through (almost) everyone following almost the exact same advice as each other.
Such people are extremely predictable.
You may have already noticed, before LLMs, cliché? Talking points that make a group identifiable? Words and phrases that act as applause lights or cognitive stop signs? (That last sentence itself being a pair of clichés that you can use to identify where I hang out online).
Anyway, point is, LLMs will give us a memetic monoculture before they turn us all into a world of correctly personalised Whispering Earring wearers. That makes them predictable, that makes them exploitable. It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves.
(Of course, the fact I've said this in writing means this is in the training data; in the general case this means the LLMs will know that and account for that, but I suspect comments like this won't shift the needle all that much compared to the aggregate output of 3 billion people reacting to short-form emotional manipulation A/B slop)
My dog often gets misidentified as a restricted breed. This used to make apartment hunting difficult, because occasionally, the property manager would visually ID the dog breed as banned, I’d have to go to the vet and get paperwork, potentially gene testing, arguing she wasn't, it was a whole thing.
But recently, the apartment I moved into had an online portal where I had to upload a photo, and it would identify the breed to determine if it was approved.
I correctly assumed the portal was using an LLM for this purpose. I wrote a script which submitted different photos of my dog to the major LLM providers until it found a photo which all the LLMs identified agreed was the correct breed.
I simply submitted that photo and, as expected, passed with flying colors.
> It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves.
I don't believe this is how chess works, and I don't believe this is how Stockfish works, and I don't believe this is how AI works.
Stockfish isn't winning because it's playing a better sequence of programmed steps, and having access to "the next version of Stockfish" doesn't mean it can "guess the next move" and play against that.
You have Stockfish version n, see board state s. I have Stockfish version n, see board state s. I want to know what you're about to do, so I put Stockfish into state s, ask it what the best move is, and I know you'll make that move because I know you'll ask Stockfish version n the same question of the same state. I now know board state s+1.
The steps are not pre-programmed, but the program itself is (modulo hardware imprecision) deterministic. If there's a RNG in there then sure, this doesn't work as easily as I wrote it; and there may be randomness in the thing that this is a metaphor for, regardless of if there's one in Stockfish or not, but that's not hard to work with when you want to win against an aggregate: we invented the field of statistics to deal with random numbers because they come up so often.
"Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?"
Many of the LLM maximalists I know don't have the skills or knowledge to excel in technology and need to use LLMs to do their job. It's seen as a cheat code to get work done.
As an example, A person I went to high school with that could barely figure out how to setup a Drupal site a few years ago, is now a frontier engineer at an AI startup. His Linkedin posts are filled with AI buzz words on a daily basis.
"It's inherently worth more because it's hard-earned and anchored to experiences we share. "
At some point, it will be impossible to tell the difference. Many people already can't tell if something was generated by AI.
> At some point, it will be impossible to tell the difference. Many people already can't tell if something was generated by AI.
So what if its indistinguishable - its not a product of human intellect or effort. I feel there is a large disconnect where people look at artful output such as music or writing as a thing no different than a box of paper clips. To them, "It's just there." They don't care how it got there, they just like the feeling they get from the consumption.
That's fast food thinking; It's engineered to be "tasty" in the sense that they put the right amount of chemicals into the food to tickle the right nerve endings. It's junk food that exists to turn a profit. Whereas even the local diner puts effort into its food and has a damn fine Greek menu and the best mozzarella sticks.
If it's indistinguishable, that could be because the user doesn't care to look closely, or it could be because it's just that well made now. For simple profile pictures, I genuinely stopped being able to tell if I'm looking at a real photo or not last year.
> That's fast food thinking; It's engineered to be "tasty" in the sense that they put the right amount of chemicals into the food to tickle the right nerve endings. It's junk food that exists to turn a profit. Whereas even the local diner puts effort into its food and has a damn fine Greek menu and the best mozzarella sticks.
The digital version fast food we already have and is a little different, in that it's the tuning of "the algorithm" to addict us, while text and image models* seem to be trying to actually fool us.
* I suspect video and music models are trying to addict, but I'm not super sure either way.
But the AI would not be able to perform live on a stage, producing emotions. I can't imagine staying 3 hours in a crowded room looking at a robot or a computer screen, even if it produces great music. Or if we get to that point, what gives the right and permission for humans to exist anymore?
Real programmers write the code and throw it away after compilation. All the fixes happen in the binary. You are not a real programmer unless you debug hex dumps and add changes directly to the compiled program.
Text editor? No. You punch the program on cards and then wait 1 week to get your turn and get a compiler error.
Perhaps you underestimated the person you went to high school with? The fact he or she "could barely figure out how to set up a Drupal site a few years ago" doesn't mean much. Lots of people are capable of immense progress when they apply themselves.
>> "It's inherently worth more because it's hard-earned and anchored to experiences we share."
> At some point, it will be impossible to tell the difference. Many people already can't tell if something was generated by AI.
Nit: it won't be impossible, just so hard so most people won't bother or give up if they try, and society will settle into a mediocre, regressed state. Then wait a little while, and the next generation will justify their mediocrity as actually some kind of progress, and the people who knew better will be dead and unable to challenge that.
More technology != more progress. Just look what social media had done. At its best, it's like what came before, just more isolating.
More technology does bring more progress in the long-run.
Control of technology is a key issue. Social media wouldn’t be allowed to exist the way it does if govt’s regulated the internet much earlier. Are you happy with the trade off…?
> More technology does bring more progress in the long-run.
That's an article of faith, and I don't think it's true, or will stop being true at some point.
> Control of technology is a key issue. Social media wouldn’t be allowed to exist the way it does if govt’s regulated the internet much earlier. Are you happy with the trade off…?
Not necessarily. Human psychology needs limits and friction, and technology is almost always about removing limits and friction. We're not wise enough, collectively and individually, to say no in many, many cases when we should.
I don't think what I'm about to say is a black-and-white rule, but I think we get into trouble when technology isn't invented to solve actual problems, but invented as an end in itself.
If we're contributing anecdata I have used LLMs to tremendous effect to learn all kinds of interesting stuff because I like learning interesting stuff and LLMs can tailor the level of instruction to exactly where you're at and don't mind questions interrupting ever 6 seconds which is better than both textbooks and teachers, mostly.
"Gatekeeping" is a derogatory way of saying "maintaining standards of quality", usually by those who stand to benefit from lowered standards. I heartily approve of "gatekeeping" because I like having competent co-workers.
> But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you.
Except calculators have been a problem for decades, it's why they're not allowed in school when you're still learning. Without doing the math yourself and internalizing how it works, you won't develop the number sense to tell if the result makes sense (broken calculator, typo, wrong equation, etc).
I still remember my physics teacher using one of the student's test answers as an example of how he should have known it was wrong and gone back over it (I think it was a pendulum on an elevator, his result had negative gravity (so gravity going upwards)).
> If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share. I want to support that
I'd extend on this as well: the process of creating changes you. In a technical sense, where you approach a problem and the way you solve that problem informs you. Both your problem solving skills, creative skills, but also even understanding how a compromise works.
This is why I have minimal compunction about an experienced engineer using AI-assisted coding ("hey claude, define this data class") versus finding AI-art to be repugnant.
The act of creating an artistic work is both an expression, but also the act of ideating and then executing on that changes you. Experience, emotion, and other more intangible concepts.
I use AI to make songs and pictures for my friends. I barely even listen to music that I didn’t have a hand in making. It’s my favorite genre by far. My kids too. Personalized studio quality songs all day, every day. I suddenly have the power and time to make things I enjoy. I think there’s going to be a hollowing out that’s going to happen where we might not have another Michael Jackson or Taylor Swift or famous musicians because why bother when you can just listen to your own custom tailored tunes. All that matters to me is that I like the song. I don’t care if someone built a custom keyboard or did it on a Stradivarius violin.
I mostly agree with you but, how do you square the use of other types of automation that are not AI? e.g computer animation vs hand-drawn, CAD software for engineering and architecture… Are those lesser art forms or we’re just used to them?
> In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more.
While I appreciate you laying it out so plainly, I disagree. A novel is a bunch of words and I don't care if they were written by one person, five, an AI, or infinite monkeys on typewriters. What's valuable in a novel (or a poem) is in the words.
Wow, I could not disagree with this more. What’s valuable about a novel is the social relationships it fosters, both in relation to the author, and also all the others that have read it. I read a book to better understand what other people are thinking, how they see the world — both directly from the author, and indirectly, in discovering what other readers may have found valuable.
I can maybe understand finding value in a machine-written novel if others also read it and enjoyed it, but having an LLM spit out a novel and reading it in isolation, that would be a complete waste of time to me.
It seems to me that you don't like reading (which is fine). Some people enjoy reading words strung together in a certain way. The value comes from the simple relationship between the text and the reader, not on some kind of social connection.
It seems to be that you don't appreciate the relationship between the author and the reader and the author and the text, which is fine, but is a bit tragic.
Not that I doubt that one day people will simply gather around the AI infinite story creation bot. They just won't know what they're missing. :(
LLMs are inherently backward looking; their training corpus consists of existing works and their ability to extrapolate is ultimately constrained by that corpus. However powerful their proponents claim them to be, they aren't powerful enough to predict the evolution of human society and culture as a whole.
Changes to that corpus that reflect the real world are going to come from incorporating future works created by humans. (LLM generated training data can reinforce fidelity to the current corpus but don't alter it.) The future still belongs to human creativity.
Generally speaking - I agree with you. Source bias is a real logical fallacy, and failing to evaluate the content itself, regardless of the source, is a problematic view of the world.
In the same way that I don't need the lumber in my house "hand sawed" for it to achieve my goal of creating a habitable space.
---
But more broadly, I do think there's space to at least question the use and role of AI.
Because while content can (and should) be addressed directly, there's a valid meta-conversation about the intent of producing content, and the results producing that content might have.
What goal does producing this content achieve?
What is the role of this content in society?
Is this content, on this scale, an appropriate thing to be making?
These are MUCH harder questions - often because we've shifted from concrete (content) to abstract (value judgements).
To go back to my housing analogy: We're no longer evaluating the benefits of hand-sawed vs power sawed timber. We're discussing whether our housing is built in the right spots, if we're building enough of it, and are we allocating it in the right ways.
This is irrelevant because we’re not using the source to judge the strength of an argument. Logical fallacies have only to do with the strength of a logical argument, nothing else.
To illustrate this, if I sell you some energy and you ask whether the energy comes from burning children or from a solar farm, I can’t say that it’s a logical fallacy for you to care because all energy powers your home just the same. You don’t owe any consideration to the content (energy) at all, because energy is not an argument that is subject to the source fallacy, to treat it as such is a fundamental category error.
Even to take your tack and say “We should consider the energy generated from burning children apart from its source, because we don’t want to fall prey to the source fallacy. However, the societal effects should also be taken into consideration…” shouldn’t be countenanced.
You do care though. You may not think you care but you absolutely do. If somebody tells you that in the Hood canal west of Seattle, there is a population of endangered tree octopuses, who are unique among octopuses in that they climb up to land and lay their eggs under fallen tree trunks in old growth forests, and that they are now endangered because road construction is cutting their access to the old growth forests, so they cannot lay more eggs.
You would care about that story, until you found out that this story is a lie, an old April’s fools joke that escaped confinement. The words are the same, but your reactions to the exact words have changed with new information about the source.
When we read personal stories it affects our emotions as we empathize with the author, or otherwise share the feelings that the author is trying to convey. When we find out there is no such author, our empathy and our notion of shared feelings vanishes with the new information even though the words stay the same.
> You do care though. You may not think you care but you absolutely do.
This is something I have a lot of trouble explaining and generally don't try to because I've never actually studied this or anything. So I can only go from my 45 years experiencing of experiencing art along with others.
Of course if you are just putting on music to work to, this isn't going to matter much if at all, but...
Generally people do really seem to care about the person behind something they are experiencing. The simplest example I can give is one of those extremely well shot photos that very few people have taken from a sitting position of their feet dangling off a massive building. I would have a very hard time believing anyone claiming that such a photo wouldn't give them very different feelings if they knew it was a real person v not. Again, this is the simplest example I can think of but I think it goes much deeper with all sorts of art, ie, most people to some degree are attempting experiencing art through the person who created it whether they "know" it or not. This is evident when presented with something they don't like and say something like, "Who would make this?"
Your example hinges on whether a bunch of words is true, not on how they came to be written.
> When we read personal stories it affects our emotions as we empathize with the author, or otherwise share the feelings that the author is trying to convey. When we find out there is no such author, our empathy and our notion of shared feelings vanishes with the new information even though the words stay the same.
This isn't true for me. If I read an incredibly moving poem and later learned that it was written by someone casting the I Ching and picking words out of a hat, it would not affect how I felt about the poem.
I care because it’s not personal, it’s not informed by lived experience, etc. It has no meaning. AI “novels” are just facsimiles of other people’s writings assembled in a way that appears new, but ultimately it’s just an approximation of a person writing a book. Their’s no author intent or context of any kind to consider. It removes core pillars of experiencing literature.
I don’t want a mathematical approximation of writing informed by feelings, knowledge, experiences, etc. anymore than I would want to see an “AI band” perform just because the music is supposedly great. There’s no personality, there’s nothing personal period.
while I understand this argument - I think it consistently falls down when compared to the types of content that are generally purchased and consumed.
I'd argue that most folks get plenty of value from the content itself, entirely separate from the intent, context, or even existence of the author.
I don't care about the motivations of the director to enjoy an action movie. I don't care about the life history of the author to enjoy a good fiction.
I think there is (and should be) space for content where people care, but I'd suggest it's the fringes (ex - majority of music is production grade pop, not meaningful songs, majority of books sales are erotic fiction, etc).
> I don't care about the life history of the author to enjoy a good fiction.
But the fiction is a product of the author's lived experiences. If the author had lived a different life, they would have written a different book. Or none at all. Without life experience, where would stories even come from? Why would they matter at all?
> types of content that are generally purchased and consumed.
This is the entire issue though. We’ve boiled creation down to “consumable content” like we are all in a boardroom talking market strategy. I am reading a book to enrich my life. Yes I enjoy popcorn entertainment and “low brow“ stuff, not everything needs to be Citizen Kane, but what does it say about us if we’re willing to just sit around “consuming” LLM content which is just facsimiles of actual creation by real people? What is the point when we have more “content to consume” than ever before? It’s just saturating us with impersonal stuff lazily achieved by scraping the real thing and prompting until it outputs something acceptable. Why is the person even making it? The answer unfortunately is almost always “a quick buck,” so I’m not sold.
What is the point of reading a middling fantasy book that a person didn’t even create when there are already likely countless fantasy books in existence/being written right now?
> I don't care about the motivations of the director to enjoy an action movie. I don't care about the life history of the author to enjoy a good fiction.
...and you don't need to know those things, that's insane. But you very likely are, even (not so) subconsciously, questioning some of these things.
The calculator argument is interesting because I am for sure slower at arithmetic by hand than when I was in gradeschool able to rattle off times tables and long division and what not, due to calculator use. I know plenty of other people who can’t easily do basic math, like multiplying something by 1.5x as a recent example in my life. Their gradeschool self probably would also run laps around them.
> We have people here who use LLMs to raise their children, to manage relationships, to design products.
My first thoughts were around trying to understand why these people would do that. What I see a lot around me, is people being afraid to fail, as it's some inherently dumb and bad thing, not realizing that sometimes failing is what makes you learn enough so you don't fail later, or builds resilience in other ways that later will be useful, for others or for yourself. Avoiding the path of harsh failure will put you down the path of mediocrity.
It's passing the buck, isn't it, if I ask AI and AI tells me something dumb, I can blame AI and absolve myself of responsibility.
At the same time, the troubling allure is that the machine has ingested a million books and has better knowledge than me (e.g. in things like JavaScript or Japanese), so why not trust it about how to raise kids.
I don't relate with this at all, are you saying that founders who execute through employees are not creating anything by "themselves", they have no "unique contribution"?
Most of us just want to create great products and hopefully get paid for them. And at the end, if I create a great product through this method, I feel equally happy, who wants to grind for a decade over some manual software creation process ?
*some of us. This is one of the differentiators (can’t find a better word right now). Some became software developers because they enjoy(ed) the process. Some became software developers because that was the way to make products that made money. Neither is more correct or genuine than the other, but neither can also claim to be a significant majority.
> So what's your unique contribution to this world - is it the prompt you once wrote
Why not? You're implying writing good prompts doesn't require effort or thought, which is false to me.
> Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?
To write better prompt is a good reason to learn new things. It's so obvious that more knowledge and experience you have, the better results you get from LLMs.
Most people only do the simplest of math on a day to day basis. I’d bet that if you asked most adults to do something even remotely non-trivial (addition, subtraction, multiplication) a lot would have trouble without a calculator and/or make a large number of mistakes. That’s probably fine because it’s unlikely most adults are all of a sudden going to have to do non-trivial math without a calculator. That probably isn’t the case for people who are having agents do effectively everything for them in their lives
"What's your unique contribution to the world" is a good question, and I've been thinking about it a lot. One framing I very much like is thinking about the priceable versus unpriceable contributions of a person. As an example, your software engineering ability is priceable (it's your labor). Your value to your parents isn't priceable. The problem is, right now the default assumed value of the latter is 0. Our society only tries to put a value on priceable facets of a person. There's reasons for that, wisdom of the crowds and whatnot, but of course it also means that if an AI can do that thing, economics drives society to adopt AI over employing people.
It's possible to set that latter value to be nonzero. You can't use free markets to set it because they necessarily cannot see what price to set, so you kinda have to guess, but IMO, it isn't zero, and I'd hazard to say it's positive.
Silly nitpick but the free market absolutely can, and has in the past, put a monetary (positive)value on your person.
We have of course abolished slavery since then.
Which is a weird way of saying I don't think we necessarily should put a monetary value to people. Instead we need to realize that money isn't everything in life. So much of life is unpriceable, and that's as it should be. Attempting to price the unpriceable necessarily makes a mockery of the unpriceable thing.
Why do you think religion existed en large mass? It was a means for self-control.
In a world where you have OF, lots of soft-porn content, content designed to get reactions etc which billions of people are exposed to to - do you not realise humans are implicitly stating there is a price for everything?
That’s essentially what America figured out whilst Europe and the rest have tried to remain ‘true to values’.
The same was probably said when machines started making furniture, cloth, harvesting wheat, etc. A desk made custom by a human to your specs is cool, beautiful, and life giving for both involved. And will cost $10-$50k. Maybe with the AIs and robots doing the grunt work, humans could get back to building things for each other. That would be great but I'm not sure what percentage of people are interested in that though.
> I'm not good at lifting weights or running, so my thinking is the only thing I have
I'm not saying this to be snarky, but maybe it's time to work on lifting weights and running. The worth of a novel is, of course, subjective, but most people would judge it based on the entertainment value it provides and not how much human effort went into it. Hobbies like fitness (which don't have an output that's meant to be consumed) seem like a safer harbor in this new era.
I was going to say, neither of those forms of exercise are expressions of a natural talent... they're things you simply do (at whatever levels you can currently manage) to stay healthy. They're good for your mental state, too! (And I could stand to do both a bit more regularly...)
Just live your life and don't support the AI machine. Nobody is forcing you, your life quality might get much better and you live your life as an intelligent person.
On the other side, people pushing the intelligence is for losers narrative are already too stupid to create anything that benefits humanity or the planet, so might as well ignore them.
Focusing just on AI wrt our work (let’s say software engineering for now) and letting go the other items. I see a like when this comes up the story is if the AI is doing the arch/impl then what are you even doing anymore? That is honestly not where my value add is, many people can do those two things plenty well, it’s basically commoditized at this point (at a certain experience level). What I bring is all the soft skills and intuition to glue it together and ship and maintain it.
> Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?
We had some contractors at the house doing some work. The contractors had been doing this work for a long time. I could tell they were good at what they did because I had some experience with that type of work when I was younger and they were way better than me.
We ran into an unusual situation (old house, old building techniques, overlapping old renovations) that they had never seen before. The contractors pulled up ChatGPT and started chatting with it. Eventually they came up with a solution with some good materials and techniques to make it work without turning it into a bigger tear out and complete rebuild project, and they got quick links to confirm it matched code.
This worked because they had the experience to judge the accuracy of the answers. They still checked the claims against the description of the materials we needed to order. After they did the work they had the experience to do it again, and it didn’t take them any guessing, trial and error, or searching through forums or Facebook groups (which are very popular with contractors) to get advice.
This is the right way to benefit from LLMs in learning something: Get the information then put it into practice so you learn it.
I worry when I see someone claim to learn something from an LLM but then the next turn is having the LLM do the work. It’s like skimming a math textbook but then skipping the exercises and never doing any quizzes. Every student in school learns that you have to practice to really understand something because reading the words is not enough. You have to work through it to internalize it.
LLMs can be very powerful tools in this way. They can also trick people into thinking they’re learning when they’re not. I think everyone has to go through that learning process about how to learn all over again like we all did with calculators in elementary school. There will be a lot of people who get trapped into being LLM promoters, whose output ceiling is limited by what LLMs can do as limited by their ability to recognize what’s right and wrong.
By now, almost everyone can think of someone they know who is happy to turn their brain off and press enter in Claude until it says the task is complete. I worry how this is going to intersect with the Reddit generation who grew up consuming doomer content about how jobs are just “bullshit jobs” and how your goal should be to do as little as possible at work, because there will come a time where those who spent their careers offloading their thinking to LLMs without doing the hard work of learning will not have much compelling reason to be hired over any other intern who can prompt Claude.
On the long term view I can see a lot of the LLM prompters getting replaced by LLMs doing the prompting. On social media like Reddit you can already see a lot of people gloating about how they just copy from Jira into Claude, push a PR, and then have Claude handle code review from their peers while they do nothing. Not hard to see that someone like that isn’t doing much that couldn’t be done by going one level deeper with Claude and letting it automate that person away. You have to put in some work to stay valuable.
> In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share.
.... What?
You open up with the calculator argument. Great. Then state "on the flip side". Ok.
And then declare it very simple in your mind when there is _only_ the flip side.
The whole issue is the not-simple gray area and where each of us believes the line between empowered human and brainless idiot is drawn.
> Many LLM maximalists say they use the tech to learn new things, but to what effect?
"Many book maximalists say they read books to learn new things, but to what effect?"
First we had verbal transmission, then books, then the Internet, now LLMs. And they all kind of do the same thing.
For me, the value of an LLM is it can take my imprecise query, scour its memory AND the Internet, and then return an answer (with citations if you ask) quicker than I can look for it "manually".
I know that the common refrain is “think of yourself as a manager now” but I’ve actually taken the opposite approach and have been telling anyone I train the same.
Diving deeper into technical understanding makes more sense to me at this point both as a way to make yourself more useful in the age of AI and also to use AI more effectively.
I regularly tell the kids to grab a text book on a subject that interests them and I do the same.
I’m willing to bet deep understanding is going to become a commodity soon.
It is easier than ever to learn difficult concepts. It is also easier than ever to produce things that used to require understanding of those concepts without them. Discipline and drive to use these powerful new tools patiently and with purpose is what is required.
> the common refrain is “think of yourself as a manager now”
One of the many reasons I'm determined to remain a luddite wrt AI. I hate the idea of being a manager and have refused promotions to avoid it in the past. I don't want to manage automatons any more than I want to manage people. I want to do stuff, not manage.
I find a lot of people are actually happy that they can offload mundane parts of job to automatons and to be really busy with the interesting parts.
It is not like you have to give them fun parts of the job.
While being a team lead for people I actually have to do the opposite. Get interesting parts to hand over to people to keep them happy and pick mundane not interesting boilerplate myself.
I keep hearing my this very much, so I have a question: what fun parts are there left for me?
It feels like this message of “offloading mundane tasks so you will have more time to do fun parts” is being pushed so hard, but in reality the opposite is happening. Fun parts are being offloaded while I’m left doing the soul sucking and mundane parts
I need a bash script for example. I don’t like dealing with bash or debugging scripts written by others.
I have that part covered by AI so I can get multiple servers up and running and then eventually fix some config or networking here and there which I like.
I wonder if it'd be more effective to teach how to critically use AI than to try to forward people to textbooks.
For myself, I have found that I am better able to learn new topics than ever before because being able to have a conversation with a moderately competent but sometimes catastrophically wrong AI about any new subject is actually the perfect mix of helpful and unhelpful for learning.
I use a loop along these lines:
* Ask a question
* Get an answer
* Be skeptical of the answer
* Investigate/reason about the answer
* Critique the answer
* Rinse and repeat
This kind of loop is far more useful to me than any textbook ever has been, because a textbook just drips information into my head. It's more likely to be accurate, but not guaranteed, and it doesn't encourage me to actually engage with the material in the way that a wrong AI answer does.
That’s the beauty, you can do both. In practice I usually just let the AI know what chapter I’m on and then ask questions or have it ask me questions based on the chapter.
I too give juniors the advice to crack open $textbook. It’s just painful to see the complex things they’ve created with horrible performance and no cohesive data model because they don’t have the requisite academic foundation to hand code the same thing given unlimited time
Managers, good technical ones anyway, also dive deeper into technical understanding. They don't do it to write more code, they do it to figure out what the developer is saying to ensure the developers aren't blowing smoke.
Your argument in that quote is completely unclear. Why do you believe that deep understanding is going to become a commodity soon? How would that happen?
I couldn’t agree more. It seems AI is very good at middle of the distribution tasks where it has access to a lot of training data - enough to be highly reliable at that task.
Which means as a human your only added value is on the edge of the distribution. Which means you need to be learning and doing more complex, deep topics.
Are you really diving into technicals just by asking "sooo what did you do?" to your AI? Without document crawling, debugging and pulling your hair out, how much of it is really a deep dive? I feel like all that effort that goes into generating a mental image from bits and pieces is gone. I'm just grudgingly happy I went through all that before humanity retired.
> What are we automating? Human work or human agency? Human tasks or human thinking?
I find it's so easy to convince oneself they're doing the former when it's increasingly the latter. The thinking part is so often provided by default by the models, or is a single prompt away. The thoughts are so syntactically (though not stylistically) perfect that it's difficult to ignore them and reason greenfield.
What's the solution? Given how keen models are to short-circuit the thinking process it could be the only solution is to silo off tasks/ideas. Choosing which mental tasks to silo off is itself incredibly difficult especially when there's a pressure to deliver rapidly (and in quantity) on those tasks.
It will be interesting to see when ads and propaganda are seamlessly (but purposefully) integrated into the output of LLMs. Who will decide what you should want or think? Will you even notice?
As for now, autocomplete is only as good as the training data. Once humanity collectively stop being autonomous beings and generating novel ideas, it all comes to a halt. LLM suggested ideas and preferences are nothing more than some mashup/average of what came before. The ability to actually think may become a rare treasure.
When I use a calculator, I atleast try to get with in a few digits of what I think the anwser is in my head. Mostly since when I was younger I had a very passionate teacher about how much slower everyone is now because of calculators on simple math. I just apply the same thing with LLMs, just try and think of how and what I would have said and see how close I was. Only thing I change is I don't trust the anwsers and accept some nuance in the given context. It's a double edge sword because then I crash out over it more than if I don't. When it over and under explaining the wrong sections or when it gets to an objectively terrible solution that technically anwsers the question. It feels like a student trying to get brownie points and/or give fluffed anwsers for the sake of not leaving anything blank on a test.
OT, but if we made kids learning math use log tables and slide rules for all their calculations I expect that they would engage their brains more and actually think about what they were doing, ie: form a strategy to solve a problem before they started calculating. Also I think that they would get a better "feel" for working numbers in general. I have no evidence, but I suspect that by abstracting away a lot of the "gruntwork" of calculating, we've really hampered people's development in math.
Unfortunately this adds quite a bit of overhead and would make everything take a lot more time. It might be worth it though.
> Also I think that they would get a better "feel" for working numbers in general.
This is called "number sense". I'm pretty sure we do have evidence under searches for that term, it was well-enough known as a concept when I was in school decades ago and is the reason we don't use calculators when first learning math.
This is really cool! After entering the software field, I felt uncomfortable not having a sense of relative size while working with digital storage units. Piece of mind was achieved after sitting down and mapping the units (Gb, Mb, etc) to common artifacts(audio file, plain text, database).
I've been planning forever to do something similar with length, duration, power, etc.
We had Mental Math sessions in class. The goal was to teach you how to do math without pen/paper, calculators were not even an option. I try to teach some of this to my 6 y o.
> The goal was to teach you how to do math without pen/paper, calculators were not even an option. I try to teach some of this to my 6 y o.
It works quite well. I do the math lessons during bath-time daily with my 6 yo. He's up to the point were he can add multiply pretty much any number by 2, 3 or 4 as long as the product is under 3 digits.
Going from adding single digits to multiplication of random 2-digit numbers by 4 with lessons only during bath-time (no paper or whiteboard) gives a child a great deal of confidence with numbers.
The calculator comparisons are truly meaningless imo, a calculator does nothing if you don’t know how to use it and what to input, an LLM circumvents all that, but a lot of people seem to think it’s the same.
It’s not a perfect analogy. I’m approaching it from perspective of a more universal experience of people losing aspects of a skill over time by using a tool. When ultimately the skill learned with out it allows them to use the tool much better
The question presumes that most of us are "thinking" in the first place, when in actuality, most of us are just acting according to the patterns that have emerged from our encounters with the thoughts of others. We generally adopt them and/or try to hallucinate coherence when they conflict. Very few people actually "think". It's hard work and takes time. We neither have (take) the time nor are we particularly motivated to put in the work because the patterns we have learned from others are useful enough to achieve the low goals we set for ourselves.
IOW - modern AI is simply an extension of the lack of thinking that characterizes the modern life... It just does it faster and uses a hulluva lot more energy.
This is such a strange perspective to me. I wouldn't describe anybody I know in this way. Do you not engage in thoughtful conversation with the people you meet? Do you not know people who make art as a hobby? When techies propound such a dim view of humanity, I truly fear for our species. Nobody will shed a tear when your bodily resources are reallocated to paperclip production.
I'm guessing they're referring to the unexpected finding that most people don't have an "inner voice" - they don't think in words. It at least partially influenced the NPC meme.
Man, I appreciate your optimism, but I don't know how you can spend even 5 minutes on Facebook looking at the absolutely bonkers shit one's own relatives and neighbors post and still come away thinking that most people are "thoughtful".
Are humans capable of profound creativity? Of course. Are they actually doing it? No, not very often.
I don't think modernity caused any sort of degradation.
You said it yourself, "thinking is hard work". It's rational to save energy. This might even have incentivized the emergence of mimesis in humans, which is arguably the foundation of our ability to cooperate at large scale.
Maybe a few of us do the hard work of thinking, and, if we figure out something novel and useful, huge numbers of people ape us uncritically. It's not an inspiring picture of humanity, but it's also not a reason to disparage anyone. More of a fact of life to be dealt with strategically.
It’s bad enough for rational reasoned discourse that we anthropomorphise LLMs, let’s please not then feed those words back into human discourse, further diluting their meaning. No one “hallucinates coherence”, hallucinations are by definition a perception which does not match reality.
> AI is simply an extension of
It may be an extension, but not “simply” as it also creates the problem where it didn’t exist. I’ve seen several reports (both on and offline) of people who used to engage in deep thinking (I’m talking scientists, postgrads, PhDs working at the edge of what we know) now worrying they are losing their ability to properly think due to their LLM use.
> It just does it faster and uses a hulluva lot more energy.
I hope we can agree that’s bad and that we should try to stop and even reverse it, not simply shrug our shoulders and go “ah well, we were already going to shit anyway, might as well fuck everything up faster”.
Even on Hacker News, when you see debates like 'X technology is good' or 'X technology is bad,' most of it seems to be about identity. And that identity often originates from the community they belong to.
The first identity usually starts with a community or the person who created it. Once the community forms, people under it often forget the original reasons and just accept it as their identity.
This is especially true for technology related issues, because the market share of a technology is directly tied to one's career, which makes it even more prone to becoming an identity issue.
I also do some 'thinking' in certain areas, but most of the time I don't. As my field gets deeper, it becomes harder to allocate cognitive resources to other areas. So in general, most people follow the crowd's opinion, but only maintain deep, thoughtful thinking, including 'taste,' in a few specific technical domains.
Clearly "mechanical" thinking has been automated, and you are better off, if not forced to outsource to AI. Humans have the biological needs, consciousness, taste and imagination, that's what we are left with thus far.
And just like people going to the gym to exercise their otherwise economically useless bodies, same thing will happen with the mind.
I thought it was a myth until the junior developer on my own team responded with "I don't know" to a question about why he made a certain computation during a design review. Because the (wrong) computation was fully AI generated and he couldn't even tell the difference.
Most people don't use AI to learn new stuff. They use it to do "the job" for them and they don't even understand the result. What is the point of a person if they don't bring any value to the table other than being a "resource" to generate prompts?
Maybe it's a way of perspective. I adore to use AI to actually learn, so I don't feel like I offload thinking. I use AI to do all the research which earlier I did manual through google search. I still make my own decisions, I just let Claude spoon feed me all infos I want and need. Feels great man
Using AI to actually learn is indeed very helpful I've found... I just don't think most people use it for that. I recently wanted to build and train my own (albeit, small and simple 10M param model) and used AI extensively to explain concepts for me, explain lines of code, and generate in depth visuals I can use to get a visual intuitive understanding of what is happening under the hood. I think people who have a natural curiosity to understand the why and how of things benefit immensely from it, but I do admit it is very easy to just offload all thinking to it instead of asking it why something is happening. I could have just asked it to implement the entire model using PyTorch and just ran the CLI command to begin training on some auto-generated dataset, but intentionally struggled through it. Actually learning from it requires intentionality for sure.
Lol, wdym. I like to prompt it that I want evidence, so when there is any new topic where I currently don't know how to tackle it myself, I ask that it does a research and give me evidence. So i can check the sources myself. Like I said, I let it spoon feed me.
When I know upfront how to do anything, I just give all the instructions. But the OPs point was If we offload thinking too much, so that's why I was just thinking about this example when I need thinking - that's usually when I need to learn something new.
I've found it present wrong, skewed, biased, or old information. In other cases, it seems to ignore the elephant in the room. I find all of these things problematic when trying to learn something new. The less I know about a subject, the more susceptible I am to being mislead.
When I bring these things up, it will apologize, tell me I'm right, and adjust. But what if I didn't know enough to question it? Approaching from the other angle, maybe I'm actually wrong and it's a sycophant, as mentioned, trying to please the user?
On the topics where I'm having to correct it, I probably shouldn't bother asking in the first place. On topics where I'm not correcting it, is it because the AI was right, or I just don't know enough to call it out? This kind of thing worries me about AI being leaned on more and more as a teacher.
This is also now the first thing that I find it truly useful for in a non-coding role: researching how to do things in Azure, which I have not used before.
I am increasingly finding my consulting work to be orientated around clearing up after people who outsourced their thinking to AI.
I'm seeing some incredibly dumb stuff: researchers spending months on Claude trying to do insane deduplication, unrelated to their research question, using regex; whole research methodologies YOLO'd out of ChatGPT.
The results invariably chaotic, resulting in huge amounts of stress and wasted time.
Non-technical people are treating LLMs like an oracle, making big assumptions and decisions with little regard for their implications, because their clanker told them to.
It's scary out there. The lack of critical thinking I'm seeing in some of these projects is horrific. Not unique to the post-AI era, certainly, but on a whole new level. Bad things are undoubtedly happening everywhere, right now, because someone's just like "let's ask Claude".
> The lack of critical thinking I'm seeing in some of these projects is horrific.
Yesterday I noted in another comment that I am having lots of Lawyers and Writers ask me pointed questions about "docker" and "agents" that make them sound more like JR engineers.
It's two groups of people who, as a profession, spend a large amount of time reviewing their own work and the work of others with a critical eye. Writers edit, Lawyers review evidence and every bit of content that their oppositions produces. Both groups (when doing their job well) aren't just critically thinking, they are being diligent and dedicated.
This is actually a very hard question to answer. If you’re truly AGI-pilled and believe in this progress continuing, then you basically have to contend with a scenario where AI is better than you at doing anything by an order of magnitude. And doing things suboptimally just for the sake of some notion of human independence doesn’t feel right to me.
Perhaps the only way forward will be if we figure out how to merge with the AIs so we can keep up. Otherwise, a soma-filled world likely awaits. And unlike Brave New World, I think it might actually be a lot more pleasant, but still one with a different set of tradeoffs.
I've been doing computer stuff all my life. I am learning the construction trades now. LLMs help you learn the rules for doing weird and specific stuff in the trades. One of my favorite LLM prompts is for plumbing tasks. For example: I have 1" pex b and I need to convert to a typical 1/4" RO connection line, what fittings on supplyhouse.com should I buy? So much easier than standing in Home Depot in the plumbing aisle for 25 minutes staring at the wall of fittings.
I think in the software trade you will definitely use your brain less. But in other trades, it removes the time sucks and gets you back to work.
That's pretty much how I use it for software too. I don't want it to make stuff for me. I do want it to unblock me when I feel stuck unraveling something arcane.
I think partially it is because of the amount of data you can get out of an LLM and because it looks pretty good, a lot of people treat it as authoritative.
This means they send it around and then other people have to go through the work of actually validating it before being able to act on it.
So what this really means is that the person you go the information from offloaded their thinking to the AI, while cannibalizing on yours.
The right question to ask is if it is enabling us to do more interesting things or take on harder or bigger problems that seemed too daunting before. That's what all delegation of tasks (to other humans or machines) have enabled humans to do – scale.
Whatever creativity/thinking/effort bandwidth that's available will now get shifted to a different place in the problem-solving effort bottleneck.
That's the hallmark of any delegation being effective. Do we see that happening with AI tools? Personally, I do see that working for me. Is it as good as the hype makes it to be or I wish it to be? maybe not, yet, for me. But that's the case with most things in life.
One major issue that the author ignores is that while we’re all having AI analyze our conversations or when we use it instead of search, there’s a chance it will provide an “answer” that is not correct, and literally drummed up out of thin air, and not even in the source material it’s “synthesizing”.
The article takes a position that assumes hallucinations do not occur, and then posits from that stance the question as to whether we rely on AI too much. We should be taking a step back before even asking that question and focusing on the part where AI does invent answers whole-cloth.
Information is seldom presented in a way that makes sense to me. That’s not quite right. Let me attempt to explain.
I want certain answers that the docs and the code are not giving me yet. Nothing is more irritating than working through a tutorial on a new framework and then throwing all that work out because that’s not really how one should use the framework. Nothing is more frustrating than having to get through a treatise on why this framework is The Solution before I can actually see code that uses the solution. And it’s beyond annoying when this End All Be All framework has a glaring omission that’s not obvious until you’ve built large amounts of your project on top of it.
Hand the docs and the example code to the LLM, and now I can get answers. “How can I do X?” Example code. “Then I need Z” Modified code. “How is this going to handle Q?” Explanation. “That doesn’t seem quite right. Give me a reference to the doc or code showing this.” Links.
Great, in 15 min, I have learned what I need to know, I can see that this solves a problem that I have, and I have discovered that I need an implementation of S to complement this solution.
I have no idea what people are doing. I am thinking more about harder problems than ever before.
It goes like: "Here is this thing I wonder about", and the LLM is like "Yeah sure, consider these things that are super related to what you are doing, that you probably know nothing about yet (but you know... if you are interested...)".
And that goes in any direction, for any depth. Anything that is made trivial now, is just replaced by something more consequential a level or two higher. You can just get much better at things that matter more.
The post is illustrated by a picture of handwritten notes, like that was supposed to shock the reader or something. I find this aesthetic tiring, and it usually comes from AI-maxxers. To me its saying: look at this quaint relic of the past, bereft of day-to-day utility, replaced by superior technology. Its life is now only as a symbol of a time where people actually used their brains.
This is exactly the lens I use myself. I write AI software and I use it in my development process, but I try to use the AI to do things that don't remove my agency, but extend my capabilities:
- Debug things. It knows way more than I do in many areas and it sees things I will miss. If I'm struggling to find the answer, maybe it will succeed.
- Review things. It has a wealth of experience I couldn't possibly have. Ask it to critique my work and provide an alternative perspective that I can't provide myself
- Implement a design. I have already gone through the thoughtful engineering to decide what to do and how to do it. The rest amounts to translating pseudocode to the programming language. Let it type what I would have typed anyway and save me the hassle of typos, looking up function and parameter names, and other such mechanical details. Let me use that time and mental effort to better consider my design, try more alternatives, or build more things, providing more value overall.
- Suggest ideas. Even as a 20 year professional, there are things I don't know or haven't considered. Is there a newer, faster, or more maintainable way of doing this? Is what I wrote clear to anyone other than myself? Before AI, I would ask coworkers, search the web, or reference other sources. Now I can get an immediate suggestion from something with tremendous knowledge at almost no cost. It's full up to be to consider what it suggests, further explore the used and learn about them - I don't take the AI at its word and let it decide what is best for me. But I do use it to gain perspective and explore alternatives.
There are some common traits about the thighs I use AI for. They are this that I either couldn't possibly do myself (because I'm biased, or unfamiliar, or have no access to the expertise) or that I would spend a lot of time while having little agency (mechanical translation). I am not replacing learning, thinking, or deciding. I think this is the key difference.
This is something I see and feel everyday. And it’s very frustrating and annoying.
For example I send some doc asking for a feedback and someone without reading it generate a feedback with llm with so much ambiguity that I have to get back and wait couple of more days to get a reply.
One of the most silliest thing I see is a middle manager feeding Microsoft planner to Claude to generate a report and generate future steps and sometimes it makes no sense what he present couple of weeks ago because what he present today is contradicting to the one before.
At this point I feel it’s cheaper to replace them with AI. They are just physical vessels for AI.
It’s just not that maybe they were not good enough. But now they just fully depend on AI.
Humans have been offloading thinking to something/some entity forever. From gods to influencers.
AI is the current popular way (at least in these circles) and if it's too much or too little it might not matter. What will matter is if this offloading is making people unhappy and having any negative impact in civilization.
I have no idea
The rise of knowledge work made many people far less physically active because moving one's body was no longer a given part of one's job. This led to a lot of people (who assumed sports was exercise on top of one's work, not the only source of exercise) moving very little. This meant we needed to rediscover the importance of exercise as a pillar of health.
I think something similar will happen with knowledge work, where we have to do a lot less cognitive exercise due to AI (as well as the decline of reading and rise of short-form video), which will likely lead to eventual issues and subsequently, a rise in activities designed to replicate the cognitive exercise work used to provide.
Our brains take time to ingest things and learn and using an AI tool to make decisions when taking in vast inputs of data requires then an overview of those decisions and understanding to some degree of care. If you don't take the time to go over these decisions and understand them and weigh them and course correct as needed, then you are offloading too much. You will become this approver buffer in between claude and nothing more if you don't have this methodical checking and understanding.
What is frightening is with something like neuralink that in a future hypothetical time would have very fast capability to keep informed and advised, you could be a zombie decision maker and nobody would really be able to tell. Even when you were pressed to why you made the decision, it's just another AI response, it's like a con artist or imposter dream scenario.
I noticed that atm, before these crazy hypotheticals potentially happen, the people that seem to take the time to understand things deeper are still way more valuable than those that just use tools more than not. Its obvious atm due to the lag in time and the way people respond in meetings, at least for now. :)
I saw a post the other day pulling a Frank Herbert quote from Dune about men offloading their thinking to machines and being controlled by the men who controlled the machines.
To answer the title's question directly? Yes, I think so. Or at least that it is a legitimate hazard.
I think the analogy to hyper-palatable, calorific food works well:
Humans adapted for a world with too little food. Then once there was too much, obesity and overeating became a problem for the first time. Self discipline is the cost we have to pay for this kind of abundance.
We now have a general-purpose way to offload mental effort, and are discovering in real-time the negative consequences of that.
I use AI for coding, but I feel I've moved past the honey-moon period and am now learning how to use it in a way that is not a detriment. If I care about the work I'm doing I don't want the AI to do it for me, even if it could. Deciding what work I want/should be doing myself vs what can be delegated is a new skill I, and I believe we all need to learn.
Just like calculators became crutches for school children memorising their times tables and our phones contacts list a crutch for memorising phone numbers, I see AI becoming a crutch for many, in all the areas we seem important and humanity has spent years refining (communication, thinking, writing, etc)
The reality is that most humans do very little actual thinking of their own anyway, and, if you believe that what LLMs produce constitutes a form of intelligence, it does seem "more intelligent" than most humans.
So: is outsourcing thinking a net improvement for a majority of users?
I use several models, daily, and they seem "reasonably conditioned" that they are only input to my thinking and not "my thinking". I correct them constantly; they are wrong (in reasoning/logic, in actual facts) frequently. They are demonstrably "not smarter" than I am. And yet I know many people who can "do more" with them as a "thinking" tool. I can say that "the problem" is they can't spot the errors, but they can't or won't do that in their ordinary lives, either, so, again, is it a net improvement for them?
Almost definitely Id think - I think we have to treat it something like power tools / chairs and a gym.
AI makes me massively more productive as a quant, and more creative in the sense that it can often find calculations I don't know how to start, BUT the flip side of that is that I can also feel my skills atrophy and as such am trying to make sure I do maths exercises and so on. I don't worry about programming skills because programming isn't about code.
I do feel like I'm offloading thinking to an AI, but I think that's a good thing. I envision a world where users and AI are aligned without corporate interference. AI lets me offload things that I don't need to know and frees up my brain to push farther than I could before. At least that's how it feels to me.
>AI lets me offload things that I don't need to know and frees up my brain to push farther than I could before.
How can you push your brain go farther than ever, when you don't use it for the basic task?
Higher Math does not work without understanding "lower" Math, running long runs does not work without starting on shorter runs. Thinking about complicated staff will probly not work, if you can't think about the easy stuff.
One can not learn a language without vocabulary and skipping learning verbs in a foreign language, because dictionaries exists does not bring one closer to being able to speak.
I agree that when you're building mathematical understanding, you have to do it brick by brick. But the fact of the matter is it's been decades since I did a long division problem by hand. AI lets me jump into areas that I would previously have had to spend a month studying before I could begin building - That's what I mean by it lets me reach farther, faster.
You don't always have to do division by hand. I would make the argument that it's probably good to do it occasionally. At least enough to remember how.
I don't think offloading things that you "think" you don't need to know is a realistic thing. Instead this seems like some slippery slope of intellectual degradation where slowly you'll replace more and more parts of the thought process with AI which ends in some rather sad existence.
I agree, I suspect that when we say "oh, I don't need to know this", what we really mean is we have that unpleasant feeling behind our eyes that says this is not going to gratify my goal
The issue being, gratification is rarely a good guideline. It just means collapsing the gap between doing the thing and the idea of having done the thing. But that gap is where you actually learn things
Is a good thing depending on which ones. And our taste on what to specialize our judgment into varies a lot. One thing I do use as criteria to detect it can be a problem is in lack of understanding and lack of control of the layers that are part of verifications and diagnosing power.
I'm probably not the only person that types math problems into google search because I'm too lazy to open up a calculator (or do the math myself). So yes, given I do that, I probably am susceptible to offloading some of my thinking to AI.
Are CEOs outsourcing too much of their thinking to their employees?
I find I'm not thinking less per say, just thinking about different things. Maybe you could argue there are CEOs who get too far out of touch with the reality on the ground and should get more directly involved in the work. However, I don't know how well one could argue that the CEO should do all of the work.
I see at least the current iteration LLMs and harnesses as me managing and coordinating them and thinking differently, not less.
I've noticed it when interviewing interns. A surprising number seem unable to think on their feet or solve problems without immediately reaching for chatgpt. I don't necessarily expect you to be able to solve a problem entirely without tools, but you should be able to give me the outline of how to go about something and why you would go that way.
After all, if you are just going to spit out AI, I will just get AI to do your job...
It really depends on which angle you look at it. Is it purely to meet a business goal? Or is it also for personal growth? I think it's a mix of both, but for me it's always important that I engage mentally with the process, learn something, and solve puzzles, even if that involves letting the AI take care of the coding, which is an abstraction. You could still code and not think creatively.
Decisions are special things. One of the golden rules of life is that a person (or entity) making decisions is somehow impacted or otherwise getting feedback on the repercussions of those decisions.
When you cognitively surrender to AI, or to another person (be it a leader/manager, or a subordinate/report), you are asking for trouble.
I don't think we are offloading thinking to AI. We just started to use it. AI is a tool useful to write text, for a fast searching, for the boring work. Personally I don't take the first answer, I like to challenge, to ask why, to tell it's wrong.
as a six-figure salary software engineer haver I find that a lot of colleagues take their cushy jobs for granted.
these are the people who are now first against the wall when the revolution comes.
why would I want to hire you to work the three hours you feel like working per day (per week?) when I can have AI with a deeper
knowledge set available 24x7?
Personally, I use AI to learn more about Backend Engineering actually, so it's fine for me. Beside I can also use AI to suggest and it's me verifying the idea so that's a no for me
How much of the thinking is involved in asking the right question, versus coming to the correct answer? I don't have a real answer to that but it does seem to be worth considering.
I know people like that - the amount of inane, obsessive and just strange conversations they have with AI is concerning - there's never any actually useful result or information that they get out of these chats.
Furthermore, there are some clearly wrong questions where person asks AI to make some kind of numerical evaluation of some data. And evaluation is done entirely through inference - essentially a hallucination, instead of some one-off python script which can actually give deterministic calculated evaluation. Yet they accept the answer AI gives them.
Having a very dangerous AI standoff at work, where people are debating wether or not to use a particular connection pooling / threading strategy to fix a production issue, and everyone is unqualified to answer and is instead arguing what their agent said.
They are just straight up admitting they don't know anything, and advocate fiercely for their agent's recommendation.
No one cares, no one tries to stop this behavior. It's seen as good, apparently. I admitted that I don't know enough to have an opinion at the moment, I certainly don't know how to judge the contradictory opinions of multiple frontier AIs, and I fear that just made me look incompetent.
Run both. Benchmark them. Performance is notoriously difficult to predict and much easier to test. If you have a load balancer, run the new strategy on one or two servers and see how their throughput compares to the others.
Exactly! It got so easy nowadays to use AI to setup scenarios since the busy work of writing code/test harnesses and setting things up for the benchmark is done by the machine. Then throw away what does not work.
Some benchmark that would take weeks to plan, code and set up is now hours and days - the time is now spent on the benchmark itself, not on temporary code.
Not an AI defender but this doesn't sound all that different than before where people were just arguing for their understanding of the issue. No one knows everything and has perfect alignment with reality. I'm with the other guy who responded, test it, else y'all just guessing and arguing who's better at guessing.
The answer to this question is: Politicians, not you!
Perhaps the question to ask is: who is making all of the final decisions for the things that really matter to you in your life?
No direct democracy, just people deciding for you. You can choose once every four years. Are we surprised of how easily we delegate decisions? May be AI can do it better
Yes that seems pretty clear. We also need quick adaptation in school and university to avoid cheating. Back to closed book exams I guess, home projects not counting for the final score. It sucks but I don´t see how you can´t avoid removing technology for evaluation. Sure people can use AI to boost their ability to get better results but they will have to prove it the old way with no assisted help and people watching them during exams.
LLMs are strongly mean reverting in their decision making, so heavy users are likely to be readily identified by their unadventurous conformism much as generated media is identified now.
I don't think AI causes people to stop thinking. Rather, I think it biases people toward doing what they want to do more, and that bias is what becomes problematic.
In fact, when I use AI, I don't really use it for the things I actually enjoy doing. For example, I like making UI animations, and I don't use AI for that. I also don't use AI when I'm playing games I enjoy. But when I have to make something tedious like a login screen, I use AI. And after I write the code, I just throw the entire codebase at AI to write the documentation.
The problem is that this only lets me think about things I have a taste for.
Having taste and diving deep into it is good. Immersion is great. But on the flip side, you also need to do things that aren't your taste. That's more cognitively healthy. AI prevents that.
In that sense, I think AI's strength is that it creates an environment where you can dive deeper into the areas you like.
But the real question becomes how you use the cognitive surplus that's left after offloading tasks to AI.
I visit Korean, Chinese, Japanese, and USA sites, and honestly, most people, including myself, only have deep thinking about certain topics. Outside of those, we just follow the prevailing opinion.
So I'm not really sure. I don't think using AI makes me stop thinking. I just think it creates a bias that makes my thinking only focus on the parts I want to focus on.
I’ve found that when I ask AI to do something for me that I know how to do myself - but would rather not spend the time doing - there is a not insignificant chance that the AI will return a subpar result, which I can usually tell rather quickly. Either by glancing at the code, or trying to compile it and getting an error.
This happens frequently enough that it creates a real disincentive for me to use AI for anything that I already know how to do - and use it exclusively for things I don’t know how to do.
It’s deeply frustrating to realize you just wasted 20 minutes posting error messages into Claude when you could’ve just locked in and written it yourself.
Very few actually get to a position of influence and power to actually do anything. Much of economic activity isn’t critical to life and death - it’s just stuff we have to make life more interesting than past generations.
Consider the following scenario.
You're a beginner or with very little experience go on a popular programming forum to ask a question, in the typical way someone with little experience asks, what do you get?
An answer?
No, you get bad manners, insults,
"you don't know how to ask questions",
"RTFM".
At best you'll get some people challenging you to clarify and refine your question,
which you can't do because you don't know the technology well enough.
Annoyed,
you go find a popular programming chatbot,
and ask the question.
The chatbot will give you answer,
no matter how poorly worded or nonsensical your question,
and it will do it cheerfully and confidently.
It may even tell you how great your question or idea is.
Granted,
the answer will be worthless,
both because the question was poorly worded
and because the chatbot is simply spewing statistically probable text,
but you won't know.
You're a beginner, without experience to know correct from bullshit.
You try to use the answer the chatbot gave you,
and when it doesn't work,
you go back to the chatbot.
It will continue to cheerfully answer your questions as long as you have tokens to spend.
The chatbot will never give up on trying to help you,
it will never be rude,
it won't complain.
I don't know if this is a good framing. "Too much" is subjective, and every heavy AI user will assert that they're just unlocking their potential, that calculators didn't make us dumber, etc.
But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you. On the flip side, if you use an LLM do most of your thinking, what's left? We have people here who use LLMs to raise their children, to manage relationships, to design products. So what's your unique contribution to this world - is it the prompt you once wrote? You're standing in front of a token-generating machine, pulling a lever, sometimes receiving gifts. Is that your edge, your unique experience, your purpose in life?
Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?
In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share. I want to support that. And I want to be a human who can write novels, the old-fashioned way. I'm not good at lifting weights or running, so my thinking is the only thing I have.
Have you read the "Whispering earring" essay? I love it for the LLM era.[1]
You can treat AI as a whispering earring - "What should we do now? How do we fix this? What do you think?" Or you can treat it like an exoskelton - "Implement kd-tree with metric space xyz for this problem, mapping this to that blah blah".
That's pre-thought execution automation that makes review much simpler - you already know the shape of the desired output. The whispering earring is atrophy.
1. https://croissanthology.com/earring
Thank you for the surprising read
Something I never considered much is what happens when everyone else is using the Whispering Earring.
You may be more free and independent, but you may also be unable to compete as everyone else easily gains wealth and success. Natural selection doesn't particularly care about freedom of consciousness.
Bleak.
Eventually, that becomes interesting. Before we get to that stage, I think we pass through (almost) everyone following almost the exact same advice as each other.
Such people are extremely predictable.
You may have already noticed, before LLMs, cliché? Talking points that make a group identifiable? Words and phrases that act as applause lights or cognitive stop signs? (That last sentence itself being a pair of clichés that you can use to identify where I hang out online).
Anyway, point is, LLMs will give us a memetic monoculture before they turn us all into a world of correctly personalised Whispering Earring wearers. That makes them predictable, that makes them exploitable. It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves.
(Of course, the fact I've said this in writing means this is in the training data; in the general case this means the LLMs will know that and account for that, but I suspect comments like this won't shift the needle all that much compared to the aggregate output of 3 billion people reacting to short-form emotional manipulation A/B slop)
This is already underway.
My dog often gets misidentified as a restricted breed. This used to make apartment hunting difficult, because occasionally, the property manager would visually ID the dog breed as banned, I’d have to go to the vet and get paperwork, potentially gene testing, arguing she wasn't, it was a whole thing.
But recently, the apartment I moved into had an online portal where I had to upload a photo, and it would identify the breed to determine if it was approved.
I correctly assumed the portal was using an LLM for this purpose. I wrote a script which submitted different photos of my dog to the major LLM providers until it found a photo which all the LLMs identified agreed was the correct breed.
I simply submitted that photo and, as expected, passed with flying colors.
> It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves.
I don't believe this is how chess works, and I don't believe this is how Stockfish works, and I don't believe this is how AI works.
Stockfish isn't winning because it's playing a better sequence of programmed steps, and having access to "the next version of Stockfish" doesn't mean it can "guess the next move" and play against that.
I didn't say "the next version of Stockfish".
You have Stockfish version n, see board state s. I have Stockfish version n, see board state s. I want to know what you're about to do, so I put Stockfish into state s, ask it what the best move is, and I know you'll make that move because I know you'll ask Stockfish version n the same question of the same state. I now know board state s+1.
The steps are not pre-programmed, but the program itself is (modulo hardware imprecision) deterministic. If there's a RNG in there then sure, this doesn't work as easily as I wrote it; and there may be randomness in the thing that this is a metaphor for, regardless of if there's one in Stockfish or not, but that's not hard to work with when you want to win against an aggregate: we invented the field of statistics to deal with random numbers because they come up so often.
"Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?"
Many of the LLM maximalists I know don't have the skills or knowledge to excel in technology and need to use LLMs to do their job. It's seen as a cheat code to get work done.
As an example, A person I went to high school with that could barely figure out how to setup a Drupal site a few years ago, is now a frontier engineer at an AI startup. His Linkedin posts are filled with AI buzz words on a daily basis.
"It's inherently worth more because it's hard-earned and anchored to experiences we share. "
At some point, it will be impossible to tell the difference. Many people already can't tell if something was generated by AI.
> At some point, it will be impossible to tell the difference. Many people already can't tell if something was generated by AI.
So what if its indistinguishable - its not a product of human intellect or effort. I feel there is a large disconnect where people look at artful output such as music or writing as a thing no different than a box of paper clips. To them, "It's just there." They don't care how it got there, they just like the feeling they get from the consumption.
That's fast food thinking; It's engineered to be "tasty" in the sense that they put the right amount of chemicals into the food to tickle the right nerve endings. It's junk food that exists to turn a profit. Whereas even the local diner puts effort into its food and has a damn fine Greek menu and the best mozzarella sticks.
I'm not sure what you're arguing here?
If it's indistinguishable, that could be because the user doesn't care to look closely, or it could be because it's just that well made now. For simple profile pictures, I genuinely stopped being able to tell if I'm looking at a real photo or not last year.
> That's fast food thinking; It's engineered to be "tasty" in the sense that they put the right amount of chemicals into the food to tickle the right nerve endings. It's junk food that exists to turn a profit. Whereas even the local diner puts effort into its food and has a damn fine Greek menu and the best mozzarella sticks.
The former is molecular gastronomy: https://en.wikipedia.org/wiki/Molecular_gastronomy
The digital version fast food we already have and is a little different, in that it's the tuning of "the algorithm" to addict us, while text and image models* seem to be trying to actually fool us.
* I suspect video and music models are trying to addict, but I'm not super sure either way.
If you knew how the modern music industry and local diners work, you might not be so impressed.
It's not all artisanal, made with love goodness just because human hands did it.
But the AI would not be able to perform live on a stage, producing emotions. I can't imagine staying 3 hours in a crowded room looking at a robot or a computer screen, even if it produces great music. Or if we get to that point, what gives the right and permission for humans to exist anymore?
> I can't imagine staying 3 hours in a crowded room looking at a robot or a computer screen, even if it produces great music.
May I introduce you to the runtime of The Lord of the Rings films?
Also the American Federation of Musicians' campaign against "robot" musicians replacing live musicians in movie theatres? https://www.smithsonianmag.com/history/musicians-wage-war-ag...
>That's fast food thinking
You're simply drawing the line where it suits you.
I don't consider it pure human coding if you use anything but Notepad.
What about punch cards?
Real programmers write the code and throw it away after compilation. All the fixes happen in the binary. You are not a real programmer unless you debug hex dumps and add changes directly to the compiled program.
Text editor? No. You punch the program on cards and then wait 1 week to get your turn and get a compiler error.
Nope, I draw my line at Notepad!
Perhaps you underestimated the person you went to high school with? The fact he or she "could barely figure out how to set up a Drupal site a few years ago" doesn't mean much. Lots of people are capable of immense progress when they apply themselves.
>> "It's inherently worth more because it's hard-earned and anchored to experiences we share."
> At some point, it will be impossible to tell the difference. Many people already can't tell if something was generated by AI.
Nit: it won't be impossible, just so hard so most people won't bother or give up if they try, and society will settle into a mediocre, regressed state. Then wait a little while, and the next generation will justify their mediocrity as actually some kind of progress, and the people who knew better will be dead and unable to challenge that.
More technology != more progress. Just look what social media had done. At its best, it's like what came before, just more isolating.
More technology does bring more progress in the long-run.
Control of technology is a key issue. Social media wouldn’t be allowed to exist the way it does if govt’s regulated the internet much earlier. Are you happy with the trade off…?
> More technology does bring more progress in the long-run.
That's an article of faith, and I don't think it's true, or will stop being true at some point.
> Control of technology is a key issue. Social media wouldn’t be allowed to exist the way it does if govt’s regulated the internet much earlier. Are you happy with the trade off…?
Not necessarily. Human psychology needs limits and friction, and technology is almost always about removing limits and friction. We're not wise enough, collectively and individually, to say no in many, many cases when we should.
I don't think what I'm about to say is a black-and-white rule, but I think we get into trouble when technology isn't invented to solve actual problems, but invented as an end in itself.
If we're contributing anecdata I have used LLMs to tremendous effect to learn all kinds of interesting stuff because I like learning interesting stuff and LLMs can tailor the level of instruction to exactly where you're at and don't mind questions interrupting ever 6 seconds which is better than both textbooks and teachers, mostly.
Have you ever tried to do the thing you learned without the LLM holding your hand?
Or are you just assuming you learned something because you made a finished product of some kind?
This is the classic gatekeeping our industry loves:
"This guy couldn't do a thing we found easy, now he can! Boooo!"
Yes, I can now set up a Drupal website in a few hours. That is great for me.
"Gatekeeping" is a derogatory way of saying "maintaining standards of quality", usually by those who stand to benefit from lowered standards. I heartily approve of "gatekeeping" because I like having competent co-workers.
> But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you.
Except calculators have been a problem for decades, it's why they're not allowed in school when you're still learning. Without doing the math yourself and internalizing how it works, you won't develop the number sense to tell if the result makes sense (broken calculator, typo, wrong equation, etc).
I still remember my physics teacher using one of the student's test answers as an example of how he should have known it was wrong and gone back over it (I think it was a pendulum on an elevator, his result had negative gravity (so gravity going upwards)).
> If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share. I want to support that
I'd extend on this as well: the process of creating changes you. In a technical sense, where you approach a problem and the way you solve that problem informs you. Both your problem solving skills, creative skills, but also even understanding how a compromise works.
This is why I have minimal compunction about an experienced engineer using AI-assisted coding ("hey claude, define this data class") versus finding AI-art to be repugnant.
The act of creating an artistic work is both an expression, but also the act of ideating and then executing on that changes you. Experience, emotion, and other more intangible concepts.
I use AI to make songs and pictures for my friends. I barely even listen to music that I didn’t have a hand in making. It’s my favorite genre by far. My kids too. Personalized studio quality songs all day, every day. I suddenly have the power and time to make things I enjoy. I think there’s going to be a hollowing out that’s going to happen where we might not have another Michael Jackson or Taylor Swift or famous musicians because why bother when you can just listen to your own custom tailored tunes. All that matters to me is that I like the song. I don’t care if someone built a custom keyboard or did it on a Stradivarius violin.
I mostly agree with you but, how do you square the use of other types of automation that are not AI? e.g computer animation vs hand-drawn, CAD software for engineering and architecture… Are those lesser art forms or we’re just used to them?
> In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more.
While I appreciate you laying it out so plainly, I disagree. A novel is a bunch of words and I don't care if they were written by one person, five, an AI, or infinite monkeys on typewriters. What's valuable in a novel (or a poem) is in the words.
Wow, I could not disagree with this more. What’s valuable about a novel is the social relationships it fosters, both in relation to the author, and also all the others that have read it. I read a book to better understand what other people are thinking, how they see the world — both directly from the author, and indirectly, in discovering what other readers may have found valuable.
I can maybe understand finding value in a machine-written novel if others also read it and enjoyed it, but having an LLM spit out a novel and reading it in isolation, that would be a complete waste of time to me.
It seems to me that you don't like reading (which is fine). Some people enjoy reading words strung together in a certain way. The value comes from the simple relationship between the text and the reader, not on some kind of social connection.
It seems to be that you don't appreciate the relationship between the author and the reader and the author and the text, which is fine, but is a bit tragic.
Not that I doubt that one day people will simply gather around the AI infinite story creation bot. They just won't know what they're missing. :(
I love reading, and I was explaining precisely why I love it.
Ironic, as you didn’t read their comment…
> The value comes from the simple relationship between the text and the reader
This is not some universal truth, yet you state it as such.
What if the original idea/concept was from a human, who used an LLM to extend and write the book?
LLMs are inherently backward looking; their training corpus consists of existing works and their ability to extrapolate is ultimately constrained by that corpus. However powerful their proponents claim them to be, they aren't powerful enough to predict the evolution of human society and culture as a whole.
Changes to that corpus that reflect the real world are going to come from incorporating future works created by humans. (LLM generated training data can reinforce fidelity to the current corpus but don't alter it.) The future still belongs to human creativity.
novel is a bunch of words
or even a bunch of characters, bunch of pixels and so on.
To me this is the wrong level of abstraction that is not sufficient to encode the meaning of literature.
Generally speaking - I agree with you. Source bias is a real logical fallacy, and failing to evaluate the content itself, regardless of the source, is a problematic view of the world.
In the same way that I don't need the lumber in my house "hand sawed" for it to achieve my goal of creating a habitable space.
---
But more broadly, I do think there's space to at least question the use and role of AI.
Because while content can (and should) be addressed directly, there's a valid meta-conversation about the intent of producing content, and the results producing that content might have.
What goal does producing this content achieve?
What is the role of this content in society?
Is this content, on this scale, an appropriate thing to be making?
These are MUCH harder questions - often because we've shifted from concrete (content) to abstract (value judgements).
To go back to my housing analogy: We're no longer evaluating the benefits of hand-sawed vs power sawed timber. We're discussing whether our housing is built in the right spots, if we're building enough of it, and are we allocating it in the right ways.
> Source bias is a real logical fallacy
This is irrelevant because we’re not using the source to judge the strength of an argument. Logical fallacies have only to do with the strength of a logical argument, nothing else.
To illustrate this, if I sell you some energy and you ask whether the energy comes from burning children or from a solar farm, I can’t say that it’s a logical fallacy for you to care because all energy powers your home just the same. You don’t owe any consideration to the content (energy) at all, because energy is not an argument that is subject to the source fallacy, to treat it as such is a fundamental category error.
Even to take your tack and say “We should consider the energy generated from burning children apart from its source, because we don’t want to fall prey to the source fallacy. However, the societal effects should also be taken into consideration…” shouldn’t be countenanced.
I think there is both value that comes from the work being authored by a human, and value which does not?
You do care though. You may not think you care but you absolutely do. If somebody tells you that in the Hood canal west of Seattle, there is a population of endangered tree octopuses, who are unique among octopuses in that they climb up to land and lay their eggs under fallen tree trunks in old growth forests, and that they are now endangered because road construction is cutting their access to the old growth forests, so they cannot lay more eggs.
You would care about that story, until you found out that this story is a lie, an old April’s fools joke that escaped confinement. The words are the same, but your reactions to the exact words have changed with new information about the source.
When we read personal stories it affects our emotions as we empathize with the author, or otherwise share the feelings that the author is trying to convey. When we find out there is no such author, our empathy and our notion of shared feelings vanishes with the new information even though the words stay the same.
> You do care though. You may not think you care but you absolutely do.
This is something I have a lot of trouble explaining and generally don't try to because I've never actually studied this or anything. So I can only go from my 45 years experiencing of experiencing art along with others.
Of course if you are just putting on music to work to, this isn't going to matter much if at all, but...
Generally people do really seem to care about the person behind something they are experiencing. The simplest example I can give is one of those extremely well shot photos that very few people have taken from a sitting position of their feet dangling off a massive building. I would have a very hard time believing anyone claiming that such a photo wouldn't give them very different feelings if they knew it was a real person v not. Again, this is the simplest example I can think of but I think it goes much deeper with all sorts of art, ie, most people to some degree are attempting experiencing art through the person who created it whether they "know" it or not. This is evident when presented with something they don't like and say something like, "Who would make this?"
Your example hinges on whether a bunch of words is true, not on how they came to be written.
> When we read personal stories it affects our emotions as we empathize with the author, or otherwise share the feelings that the author is trying to convey. When we find out there is no such author, our empathy and our notion of shared feelings vanishes with the new information even though the words stay the same.
This isn't true for me. If I read an incredibly moving poem and later learned that it was written by someone casting the I Ching and picking words out of a hat, it would not affect how I felt about the poem.
I care because it’s not personal, it’s not informed by lived experience, etc. It has no meaning. AI “novels” are just facsimiles of other people’s writings assembled in a way that appears new, but ultimately it’s just an approximation of a person writing a book. Their’s no author intent or context of any kind to consider. It removes core pillars of experiencing literature.
I don’t want a mathematical approximation of writing informed by feelings, knowledge, experiences, etc. anymore than I would want to see an “AI band” perform just because the music is supposedly great. There’s no personality, there’s nothing personal period.
while I understand this argument - I think it consistently falls down when compared to the types of content that are generally purchased and consumed.
I'd argue that most folks get plenty of value from the content itself, entirely separate from the intent, context, or even existence of the author.
I don't care about the motivations of the director to enjoy an action movie. I don't care about the life history of the author to enjoy a good fiction.
I think there is (and should be) space for content where people care, but I'd suggest it's the fringes (ex - majority of music is production grade pop, not meaningful songs, majority of books sales are erotic fiction, etc).
> I don't care about the life history of the author to enjoy a good fiction.
But the fiction is a product of the author's lived experiences. If the author had lived a different life, they would have written a different book. Or none at all. Without life experience, where would stories even come from? Why would they matter at all?
> types of content that are generally purchased and consumed.
This is the entire issue though. We’ve boiled creation down to “consumable content” like we are all in a boardroom talking market strategy. I am reading a book to enrich my life. Yes I enjoy popcorn entertainment and “low brow“ stuff, not everything needs to be Citizen Kane, but what does it say about us if we’re willing to just sit around “consuming” LLM content which is just facsimiles of actual creation by real people? What is the point when we have more “content to consume” than ever before? It’s just saturating us with impersonal stuff lazily achieved by scraping the real thing and prompting until it outputs something acceptable. Why is the person even making it? The answer unfortunately is almost always “a quick buck,” so I’m not sold.
What is the point of reading a middling fantasy book that a person didn’t even create when there are already likely countless fantasy books in existence/being written right now?
> I don't care about the motivations of the director to enjoy an action movie. I don't care about the life history of the author to enjoy a good fiction.
...and you don't need to know those things, that's insane. But you very likely are, even (not so) subconsciously, questioning some of these things.
The calculator argument is interesting because I am for sure slower at arithmetic by hand than when I was in gradeschool able to rattle off times tables and long division and what not, due to calculator use. I know plenty of other people who can’t easily do basic math, like multiplying something by 1.5x as a recent example in my life. Their gradeschool self probably would also run laps around them.
> We have people here who use LLMs to raise their children, to manage relationships, to design products.
My first thoughts were around trying to understand why these people would do that. What I see a lot around me, is people being afraid to fail, as it's some inherently dumb and bad thing, not realizing that sometimes failing is what makes you learn enough so you don't fail later, or builds resilience in other ways that later will be useful, for others or for yourself. Avoiding the path of harsh failure will put you down the path of mediocrity.
It's passing the buck, isn't it, if I ask AI and AI tells me something dumb, I can blame AI and absolve myself of responsibility.
At the same time, the troubling allure is that the machine has ingested a million books and has better knowledge than me (e.g. in things like JavaScript or Japanese), so why not trust it about how to raise kids.
I don't relate with this at all, are you saying that founders who execute through employees are not creating anything by "themselves", they have no "unique contribution"? Most of us just want to create great products and hopefully get paid for them. And at the end, if I create a great product through this method, I feel equally happy, who wants to grind for a decade over some manual software creation process ?
> most of us
*some of us. This is one of the differentiators (can’t find a better word right now). Some became software developers because they enjoy(ed) the process. Some became software developers because that was the way to make products that made money. Neither is more correct or genuine than the other, but neither can also claim to be a significant majority.
> So what's your unique contribution to this world - is it the prompt you once wrote
Why not? You're implying writing good prompts doesn't require effort or thought, which is false to me.
> Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?
To write better prompt is a good reason to learn new things. It's so obvious that more knowledge and experience you have, the better results you get from LLMs.
Most people only do the simplest of math on a day to day basis. I’d bet that if you asked most adults to do something even remotely non-trivial (addition, subtraction, multiplication) a lot would have trouble without a calculator and/or make a large number of mistakes. That’s probably fine because it’s unlikely most adults are all of a sudden going to have to do non-trivial math without a calculator. That probably isn’t the case for people who are having agents do effectively everything for them in their lives
"What's your unique contribution to the world" is a good question, and I've been thinking about it a lot. One framing I very much like is thinking about the priceable versus unpriceable contributions of a person. As an example, your software engineering ability is priceable (it's your labor). Your value to your parents isn't priceable. The problem is, right now the default assumed value of the latter is 0. Our society only tries to put a value on priceable facets of a person. There's reasons for that, wisdom of the crowds and whatnot, but of course it also means that if an AI can do that thing, economics drives society to adopt AI over employing people.
It's possible to set that latter value to be nonzero. You can't use free markets to set it because they necessarily cannot see what price to set, so you kinda have to guess, but IMO, it isn't zero, and I'd hazard to say it's positive.
Silly nitpick but the free market absolutely can, and has in the past, put a monetary (positive)value on your person.
We have of course abolished slavery since then.
Which is a weird way of saying I don't think we necessarily should put a monetary value to people. Instead we need to realize that money isn't everything in life. So much of life is unpriceable, and that's as it should be. Attempting to price the unpriceable necessarily makes a mockery of the unpriceable thing.
This is nonsense though.
Why do you think religion existed en large mass? It was a means for self-control.
In a world where you have OF, lots of soft-porn content, content designed to get reactions etc which billions of people are exposed to to - do you not realise humans are implicitly stating there is a price for everything?
That’s essentially what America figured out whilst Europe and the rest have tried to remain ‘true to values’.
You guys have to look up where the concept of Dignity came from.
>We have people here who use LLMs to raise their children, to manage relationships, to design products
It's not either/or. I think about designing a product in conjunction with AI "thinking" about it. Two heads are better than one.
LLMs are bicycles for the mind, they do not think instead of you
The same was probably said when machines started making furniture, cloth, harvesting wheat, etc. A desk made custom by a human to your specs is cool, beautiful, and life giving for both involved. And will cost $10-$50k. Maybe with the AIs and robots doing the grunt work, humans could get back to building things for each other. That would be great but I'm not sure what percentage of people are interested in that though.
> I'm not good at lifting weights or running, so my thinking is the only thing I have
I'm not saying this to be snarky, but maybe it's time to work on lifting weights and running. The worth of a novel is, of course, subjective, but most people would judge it based on the entertainment value it provides and not how much human effort went into it. Hobbies like fitness (which don't have an output that's meant to be consumed) seem like a safer harbor in this new era.
I was going to say, neither of those forms of exercise are expressions of a natural talent... they're things you simply do (at whatever levels you can currently manage) to stay healthy. They're good for your mental state, too! (And I could stand to do both a bit more regularly...)
Just live your life and don't support the AI machine. Nobody is forcing you, your life quality might get much better and you live your life as an intelligent person.
On the other side, people pushing the intelligence is for losers narrative are already too stupid to create anything that benefits humanity or the planet, so might as well ignore them.
Focusing just on AI wrt our work (let’s say software engineering for now) and letting go the other items. I see a like when this comes up the story is if the AI is doing the arch/impl then what are you even doing anymore? That is honestly not where my value add is, many people can do those two things plenty well, it’s basically commoditized at this point (at a certain experience level). What I bring is all the soft skills and intuition to glue it together and ship and maintain it.
Let the impl go.
> Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?
We had some contractors at the house doing some work. The contractors had been doing this work for a long time. I could tell they were good at what they did because I had some experience with that type of work when I was younger and they were way better than me.
We ran into an unusual situation (old house, old building techniques, overlapping old renovations) that they had never seen before. The contractors pulled up ChatGPT and started chatting with it. Eventually they came up with a solution with some good materials and techniques to make it work without turning it into a bigger tear out and complete rebuild project, and they got quick links to confirm it matched code.
This worked because they had the experience to judge the accuracy of the answers. They still checked the claims against the description of the materials we needed to order. After they did the work they had the experience to do it again, and it didn’t take them any guessing, trial and error, or searching through forums or Facebook groups (which are very popular with contractors) to get advice.
This is the right way to benefit from LLMs in learning something: Get the information then put it into practice so you learn it.
I worry when I see someone claim to learn something from an LLM but then the next turn is having the LLM do the work. It’s like skimming a math textbook but then skipping the exercises and never doing any quizzes. Every student in school learns that you have to practice to really understand something because reading the words is not enough. You have to work through it to internalize it.
LLMs can be very powerful tools in this way. They can also trick people into thinking they’re learning when they’re not. I think everyone has to go through that learning process about how to learn all over again like we all did with calculators in elementary school. There will be a lot of people who get trapped into being LLM promoters, whose output ceiling is limited by what LLMs can do as limited by their ability to recognize what’s right and wrong.
By now, almost everyone can think of someone they know who is happy to turn their brain off and press enter in Claude until it says the task is complete. I worry how this is going to intersect with the Reddit generation who grew up consuming doomer content about how jobs are just “bullshit jobs” and how your goal should be to do as little as possible at work, because there will come a time where those who spent their careers offloading their thinking to LLMs without doing the hard work of learning will not have much compelling reason to be hired over any other intern who can prompt Claude.
On the long term view I can see a lot of the LLM prompters getting replaced by LLMs doing the prompting. On social media like Reddit you can already see a lot of people gloating about how they just copy from Jira into Claude, push a PR, and then have Claude handle code review from their peers while they do nothing. Not hard to see that someone like that isn’t doing much that couldn’t be done by going one level deeper with Claude and letting it automate that person away. You have to put in some work to stay valuable.
> In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share.
.... What?
You open up with the calculator argument. Great. Then state "on the flip side". Ok.
And then declare it very simple in your mind when there is _only_ the flip side.
The whole issue is the not-simple gray area and where each of us believes the line between empowered human and brainless idiot is drawn.
> Many LLM maximalists say they use the tech to learn new things, but to what effect?
"Many book maximalists say they read books to learn new things, but to what effect?"
First we had verbal transmission, then books, then the Internet, now LLMs. And they all kind of do the same thing.
For me, the value of an LLM is it can take my imprecise query, scour its memory AND the Internet, and then return an answer (with citations if you ask) quicker than I can look for it "manually".
And I'm far from an LLM maximalist...
I know that the common refrain is “think of yourself as a manager now” but I’ve actually taken the opposite approach and have been telling anyone I train the same.
Diving deeper into technical understanding makes more sense to me at this point both as a way to make yourself more useful in the age of AI and also to use AI more effectively.
I regularly tell the kids to grab a text book on a subject that interests them and I do the same.
I’m willing to bet deep understanding is going to become a commodity soon.
It is easier than ever to learn difficult concepts. It is also easier than ever to produce things that used to require understanding of those concepts without them. Discipline and drive to use these powerful new tools patiently and with purpose is what is required.
> the common refrain is “think of yourself as a manager now”
One of the many reasons I'm determined to remain a luddite wrt AI. I hate the idea of being a manager and have refused promotions to avoid it in the past. I don't want to manage automatons any more than I want to manage people. I want to do stuff, not manage.
I find a lot of people are actually happy that they can offload mundane parts of job to automatons and to be really busy with the interesting parts.
It is not like you have to give them fun parts of the job.
While being a team lead for people I actually have to do the opposite. Get interesting parts to hand over to people to keep them happy and pick mundane not interesting boilerplate myself.
I keep hearing my this very much, so I have a question: what fun parts are there left for me?
It feels like this message of “offloading mundane tasks so you will have more time to do fun parts” is being pushed so hard, but in reality the opposite is happening. Fun parts are being offloaded while I’m left doing the soul sucking and mundane parts
I need a bash script for example. I don’t like dealing with bash or debugging scripts written by others.
I have that part covered by AI so I can get multiple servers up and running and then eventually fix some config or networking here and there which I like.
Instead of bashing my head against the wall.
Different strokes I guess. I love using bash.
I wonder if it'd be more effective to teach how to critically use AI than to try to forward people to textbooks.
For myself, I have found that I am better able to learn new topics than ever before because being able to have a conversation with a moderately competent but sometimes catastrophically wrong AI about any new subject is actually the perfect mix of helpful and unhelpful for learning.
I use a loop along these lines:
* Ask a question * Get an answer * Be skeptical of the answer * Investigate/reason about the answer * Critique the answer * Rinse and repeat
This kind of loop is far more useful to me than any textbook ever has been, because a textbook just drips information into my head. It's more likely to be accurate, but not guaranteed, and it doesn't encourage me to actually engage with the material in the way that a wrong AI answer does.
That’s the beauty, you can do both. In practice I usually just let the AI know what chapter I’m on and then ask questions or have it ask me questions based on the chapter.
I too give juniors the advice to crack open $textbook. It’s just painful to see the complex things they’ve created with horrible performance and no cohesive data model because they don’t have the requisite academic foundation to hand code the same thing given unlimited time
a commodity is something of low value unless in a large aggregate.
Managers, good technical ones anyway, also dive deeper into technical understanding. They don't do it to write more code, they do it to figure out what the developer is saying to ensure the developers aren't blowing smoke.
Deep knowledge is going to be the opposite of a commodity.
> deep understanding is going to become a commodity soon
How does the text generated by LLM make “our” understanding deeper compared to text written in the books?
I think you have my argument backwards
Your argument in that quote is completely unclear. Why do you believe that deep understanding is going to become a commodity soon? How would that happen?
Yup, my mistake.
No worries :)
I couldn’t agree more. It seems AI is very good at middle of the distribution tasks where it has access to a lot of training data - enough to be highly reliable at that task.
Which means as a human your only added value is on the edge of the distribution. Which means you need to be learning and doing more complex, deep topics.
Are you really diving into technicals just by asking "sooo what did you do?" to your AI? Without document crawling, debugging and pulling your hair out, how much of it is really a deep dive? I feel like all that effort that goes into generating a mental image from bits and pieces is gone. I'm just grudgingly happy I went through all that before humanity retired.
> What are we automating? Human work or human agency? Human tasks or human thinking?
I find it's so easy to convince oneself they're doing the former when it's increasingly the latter. The thinking part is so often provided by default by the models, or is a single prompt away. The thoughts are so syntactically (though not stylistically) perfect that it's difficult to ignore them and reason greenfield.
What's the solution? Given how keen models are to short-circuit the thinking process it could be the only solution is to silo off tasks/ideas. Choosing which mental tasks to silo off is itself incredibly difficult especially when there's a pressure to deliver rapidly (and in quantity) on those tasks.
It will be interesting to see when ads and propaganda are seamlessly (but purposefully) integrated into the output of LLMs. Who will decide what you should want or think? Will you even notice?
As for now, autocomplete is only as good as the training data. Once humanity collectively stop being autonomous beings and generating novel ideas, it all comes to a halt. LLM suggested ideas and preferences are nothing more than some mashup/average of what came before. The ability to actually think may become a rare treasure.
When I use a calculator, I atleast try to get with in a few digits of what I think the anwser is in my head. Mostly since when I was younger I had a very passionate teacher about how much slower everyone is now because of calculators on simple math. I just apply the same thing with LLMs, just try and think of how and what I would have said and see how close I was. Only thing I change is I don't trust the anwsers and accept some nuance in the given context. It's a double edge sword because then I crash out over it more than if I don't. When it over and under explaining the wrong sections or when it gets to an objectively terrible solution that technically anwsers the question. It feels like a student trying to get brownie points and/or give fluffed anwsers for the sake of not leaving anything blank on a test.
OT, but if we made kids learning math use log tables and slide rules for all their calculations I expect that they would engage their brains more and actually think about what they were doing, ie: form a strategy to solve a problem before they started calculating. Also I think that they would get a better "feel" for working numbers in general. I have no evidence, but I suspect that by abstracting away a lot of the "gruntwork" of calculating, we've really hampered people's development in math.
Unfortunately this adds quite a bit of overhead and would make everything take a lot more time. It might be worth it though.
> Also I think that they would get a better "feel" for working numbers in general.
This is called "number sense". I'm pretty sure we do have evidence under searches for that term, it was well-enough known as a concept when I was in school decades ago and is the reason we don't use calculators when first learning math.
I'm working on this! https://magworld.pw
This is really cool! After entering the software field, I felt uncomfortable not having a sense of relative size while working with digital storage units. Piece of mind was achieved after sitting down and mapping the units (Gb, Mb, etc) to common artifacts(audio file, plain text, database).
I've been planning forever to do something similar with length, duration, power, etc.
Looks interesting, I'll give this a listen and a read. :)
We had Mental Math sessions in class. The goal was to teach you how to do math without pen/paper, calculators were not even an option. I try to teach some of this to my 6 y o.
> The goal was to teach you how to do math without pen/paper, calculators were not even an option. I try to teach some of this to my 6 y o.
It works quite well. I do the math lessons during bath-time daily with my 6 yo. He's up to the point were he can add multiply pretty much any number by 2, 3 or 4 as long as the product is under 3 digits.
Going from adding single digits to multiplication of random 2-digit numbers by 4 with lessons only during bath-time (no paper or whiteboard) gives a child a great deal of confidence with numbers.
The calculator comparisons are truly meaningless imo, a calculator does nothing if you don’t know how to use it and what to input, an LLM circumvents all that, but a lot of people seem to think it’s the same.
It’s not a perfect analogy. I’m approaching it from perspective of a more universal experience of people losing aspects of a skill over time by using a tool. When ultimately the skill learned with out it allows them to use the tool much better
Ah yes, I very much agree on that framing.
The question presumes that most of us are "thinking" in the first place, when in actuality, most of us are just acting according to the patterns that have emerged from our encounters with the thoughts of others. We generally adopt them and/or try to hallucinate coherence when they conflict. Very few people actually "think". It's hard work and takes time. We neither have (take) the time nor are we particularly motivated to put in the work because the patterns we have learned from others are useful enough to achieve the low goals we set for ourselves.
IOW - modern AI is simply an extension of the lack of thinking that characterizes the modern life... It just does it faster and uses a hulluva lot more energy.
This is such a strange perspective to me. I wouldn't describe anybody I know in this way. Do you not engage in thoughtful conversation with the people you meet? Do you not know people who make art as a hobby? When techies propound such a dim view of humanity, I truly fear for our species. Nobody will shed a tear when your bodily resources are reallocated to paperclip production.
I'm guessing they're referring to the unexpected finding that most people don't have an "inner voice" - they don't think in words. It at least partially influenced the NPC meme.
Man, I appreciate your optimism, but I don't know how you can spend even 5 minutes on Facebook looking at the absolutely bonkers shit one's own relatives and neighbors post and still come away thinking that most people are "thoughtful".
Are humans capable of profound creativity? Of course. Are they actually doing it? No, not very often.
> the modern life
I don't think modernity caused any sort of degradation.
You said it yourself, "thinking is hard work". It's rational to save energy. This might even have incentivized the emergence of mimesis in humans, which is arguably the foundation of our ability to cooperate at large scale.
https://en.wikipedia.org/wiki/Mimesis
Maybe a few of us do the hard work of thinking, and, if we figure out something novel and useful, huge numbers of people ape us uncritically. It's not an inspiring picture of humanity, but it's also not a reason to disparage anyone. More of a fact of life to be dealt with strategically.
Have you ever read Kurt Vonnegut's Timequake? I think it's very applicable to the average human experience.
> and/or try to hallucinate coherence
It’s bad enough for rational reasoned discourse that we anthropomorphise LLMs, let’s please not then feed those words back into human discourse, further diluting their meaning. No one “hallucinates coherence”, hallucinations are by definition a perception which does not match reality.
> AI is simply an extension of
It may be an extension, but not “simply” as it also creates the problem where it didn’t exist. I’ve seen several reports (both on and offline) of people who used to engage in deep thinking (I’m talking scientists, postgrads, PhDs working at the edge of what we know) now worrying they are losing their ability to properly think due to their LLM use.
> It just does it faster and uses a hulluva lot more energy.
I hope we can agree that’s bad and that we should try to stop and even reverse it, not simply shrug our shoulders and go “ah well, we were already going to shit anyway, might as well fuck everything up faster”.
I agree with many points.
Even on Hacker News, when you see debates like 'X technology is good' or 'X technology is bad,' most of it seems to be about identity. And that identity often originates from the community they belong to.
The first identity usually starts with a community or the person who created it. Once the community forms, people under it often forget the original reasons and just accept it as their identity.
This is especially true for technology related issues, because the market share of a technology is directly tied to one's career, which makes it even more prone to becoming an identity issue.
I also do some 'thinking' in certain areas, but most of the time I don't. As my field gets deeper, it becomes harder to allocate cognitive resources to other areas. So in general, most people follow the crowd's opinion, but only maintain deep, thoughtful thinking, including 'taste,' in a few specific technical domains.
Everyone is always thinking, just many of us not about what we're doing! Sorry
if we were really thinking then llm wouldn't have been able to compress all knoweldge into few gbs.
everyone is just thinking about how to recall, remix and repeat.
Clearly "mechanical" thinking has been automated, and you are better off, if not forced to outsource to AI. Humans have the biological needs, consciousness, taste and imagination, that's what we are left with thus far.
And just like people going to the gym to exercise their otherwise economically useless bodies, same thing will happen with the mind.
Folks, get over it.
I thought it was a myth until the junior developer on my own team responded with "I don't know" to a question about why he made a certain computation during a design review. Because the (wrong) computation was fully AI generated and he couldn't even tell the difference.
Most people don't use AI to learn new stuff. They use it to do "the job" for them and they don't even understand the result. What is the point of a person if they don't bring any value to the table other than being a "resource" to generate prompts?
Maybe it's a way of perspective. I adore to use AI to actually learn, so I don't feel like I offload thinking. I use AI to do all the research which earlier I did manual through google search. I still make my own decisions, I just let Claude spoon feed me all infos I want and need. Feels great man
Using AI to actually learn is indeed very helpful I've found... I just don't think most people use it for that. I recently wanted to build and train my own (albeit, small and simple 10M param model) and used AI extensively to explain concepts for me, explain lines of code, and generate in depth visuals I can use to get a visual intuitive understanding of what is happening under the hood. I think people who have a natural curiosity to understand the why and how of things benefit immensely from it, but I do admit it is very easy to just offload all thinking to it instead of asking it why something is happening. I could have just asked it to implement the entire model using PyTorch and just ran the CLI command to begin training on some auto-generated dataset, but intentionally struggled through it. Actually learning from it requires intentionality for sure.
It’s just so sycophantic though.
Lol, wdym. I like to prompt it that I want evidence, so when there is any new topic where I currently don't know how to tackle it myself, I ask that it does a research and give me evidence. So i can check the sources myself. Like I said, I let it spoon feed me.
When I know upfront how to do anything, I just give all the instructions. But the OPs point was If we offload thinking too much, so that's why I was just thinking about this example when I need thinking - that's usually when I need to learn something new.
I've found it present wrong, skewed, biased, or old information. In other cases, it seems to ignore the elephant in the room. I find all of these things problematic when trying to learn something new. The less I know about a subject, the more susceptible I am to being mislead.
When I bring these things up, it will apologize, tell me I'm right, and adjust. But what if I didn't know enough to question it? Approaching from the other angle, maybe I'm actually wrong and it's a sycophant, as mentioned, trying to please the user?
On the topics where I'm having to correct it, I probably shouldn't bother asking in the first place. On topics where I'm not correcting it, is it because the AI was right, or I just don't know enough to call it out? This kind of thing worries me about AI being leaned on more and more as a teacher.
If you build stuff with AI it's different. It's very tempting to defer many (too many?) decisions to AI.
I do daily. So far I am doing fine, not sure what exactly your point is, sorry.
Same here.
If anything, I think most here outsource too little thinking to AI.
What am I supposed to be afraid of? Losing skills I no longer need to get the job done?
This is also now the first thing that I find it truly useful for in a non-coding role: researching how to do things in Azure, which I have not used before.
I am increasingly finding my consulting work to be orientated around clearing up after people who outsourced their thinking to AI.
I'm seeing some incredibly dumb stuff: researchers spending months on Claude trying to do insane deduplication, unrelated to their research question, using regex; whole research methodologies YOLO'd out of ChatGPT.
The results invariably chaotic, resulting in huge amounts of stress and wasted time.
Non-technical people are treating LLMs like an oracle, making big assumptions and decisions with little regard for their implications, because their clanker told them to.
It's scary out there. The lack of critical thinking I'm seeing in some of these projects is horrific. Not unique to the post-AI era, certainly, but on a whole new level. Bad things are undoubtedly happening everywhere, right now, because someone's just like "let's ask Claude".
> The lack of critical thinking I'm seeing in some of these projects is horrific.
Yesterday I noted in another comment that I am having lots of Lawyers and Writers ask me pointed questions about "docker" and "agents" that make them sound more like JR engineers.
It's two groups of people who, as a profession, spend a large amount of time reviewing their own work and the work of others with a critical eye. Writers edit, Lawyers review evidence and every bit of content that their oppositions produces. Both groups (when doing their job well) aren't just critically thinking, they are being diligent and dedicated.
Lots of people are treating LLM's like an oracle (technical ones as well). Because our culture values moving bricks faster, with out taking into account if they are going to the right place, or even if they are the right bricks. (See: https://www.business.com/articles/management-theory-of-frank... and https://en.wikipedia.org/wiki/Time_and_motion_study for why "bricks" matter).
This is actually a very hard question to answer. If you’re truly AGI-pilled and believe in this progress continuing, then you basically have to contend with a scenario where AI is better than you at doing anything by an order of magnitude. And doing things suboptimally just for the sake of some notion of human independence doesn’t feel right to me.
Perhaps the only way forward will be if we figure out how to merge with the AIs so we can keep up. Otherwise, a soma-filled world likely awaits. And unlike Brave New World, I think it might actually be a lot more pleasant, but still one with a different set of tradeoffs.
I've been doing computer stuff all my life. I am learning the construction trades now. LLMs help you learn the rules for doing weird and specific stuff in the trades. One of my favorite LLM prompts is for plumbing tasks. For example: I have 1" pex b and I need to convert to a typical 1/4" RO connection line, what fittings on supplyhouse.com should I buy? So much easier than standing in Home Depot in the plumbing aisle for 25 minutes staring at the wall of fittings.
I think in the software trade you will definitely use your brain less. But in other trades, it removes the time sucks and gets you back to work.
That's pretty much how I use it for software too. I don't want it to make stuff for me. I do want it to unblock me when I feel stuck unraveling something arcane.
I'm going to say: yes.
I think partially it is because of the amount of data you can get out of an LLM and because it looks pretty good, a lot of people treat it as authoritative.
This means they send it around and then other people have to go through the work of actually validating it before being able to act on it.
So what this really means is that the person you go the information from offloaded their thinking to the AI, while cannibalizing on yours.
The right question to ask is if it is enabling us to do more interesting things or take on harder or bigger problems that seemed too daunting before. That's what all delegation of tasks (to other humans or machines) have enabled humans to do – scale.
Whatever creativity/thinking/effort bandwidth that's available will now get shifted to a different place in the problem-solving effort bottleneck.
That's the hallmark of any delegation being effective. Do we see that happening with AI tools? Personally, I do see that working for me. Is it as good as the hype makes it to be or I wish it to be? maybe not, yet, for me. But that's the case with most things in life.
One major issue that the author ignores is that while we’re all having AI analyze our conversations or when we use it instead of search, there’s a chance it will provide an “answer” that is not correct, and literally drummed up out of thin air, and not even in the source material it’s “synthesizing”.
The article takes a position that assumes hallucinations do not occur, and then posits from that stance the question as to whether we rely on AI too much. We should be taking a step back before even asking that question and focusing on the part where AI does invent answers whole-cloth.
I'm not personally, since I don't use GenAI at all.
Especially given the comments I see here and on other tech and programming forums, I hate the direction things are going.
I still have some hope this will all fade, but the damage done will be worse the longer it goes on, I think.
Information is seldom presented in a way that makes sense to me. That’s not quite right. Let me attempt to explain.
I want certain answers that the docs and the code are not giving me yet. Nothing is more irritating than working through a tutorial on a new framework and then throwing all that work out because that’s not really how one should use the framework. Nothing is more frustrating than having to get through a treatise on why this framework is The Solution before I can actually see code that uses the solution. And it’s beyond annoying when this End All Be All framework has a glaring omission that’s not obvious until you’ve built large amounts of your project on top of it.
Hand the docs and the example code to the LLM, and now I can get answers. “How can I do X?” Example code. “Then I need Z” Modified code. “How is this going to handle Q?” Explanation. “That doesn’t seem quite right. Give me a reference to the doc or code showing this.” Links.
Great, in 15 min, I have learned what I need to know, I can see that this solves a problem that I have, and I have discovered that I need an implementation of S to complement this solution.
That is usefulness. And it requires experience.
I have no idea what people are doing. I am thinking more about harder problems than ever before.
It goes like: "Here is this thing I wonder about", and the LLM is like "Yeah sure, consider these things that are super related to what you are doing, that you probably know nothing about yet (but you know... if you are interested...)".
And that goes in any direction, for any depth. Anything that is made trivial now, is just replaced by something more consequential a level or two higher. You can just get much better at things that matter more.
The post is illustrated by a picture of handwritten notes, like that was supposed to shock the reader or something. I find this aesthetic tiring, and it usually comes from AI-maxxers. To me its saying: look at this quaint relic of the past, bereft of day-to-day utility, replaced by superior technology. Its life is now only as a symbol of a time where people actually used their brains.
This is exactly the lens I use myself. I write AI software and I use it in my development process, but I try to use the AI to do things that don't remove my agency, but extend my capabilities: - Debug things. It knows way more than I do in many areas and it sees things I will miss. If I'm struggling to find the answer, maybe it will succeed. - Review things. It has a wealth of experience I couldn't possibly have. Ask it to critique my work and provide an alternative perspective that I can't provide myself - Implement a design. I have already gone through the thoughtful engineering to decide what to do and how to do it. The rest amounts to translating pseudocode to the programming language. Let it type what I would have typed anyway and save me the hassle of typos, looking up function and parameter names, and other such mechanical details. Let me use that time and mental effort to better consider my design, try more alternatives, or build more things, providing more value overall. - Suggest ideas. Even as a 20 year professional, there are things I don't know or haven't considered. Is there a newer, faster, or more maintainable way of doing this? Is what I wrote clear to anyone other than myself? Before AI, I would ask coworkers, search the web, or reference other sources. Now I can get an immediate suggestion from something with tremendous knowledge at almost no cost. It's full up to be to consider what it suggests, further explore the used and learn about them - I don't take the AI at its word and let it decide what is best for me. But I do use it to gain perspective and explore alternatives.
There are some common traits about the thighs I use AI for. They are this that I either couldn't possibly do myself (because I'm biased, or unfamiliar, or have no access to the expertise) or that I would spend a lot of time while having little agency (mechanical translation). I am not replacing learning, thinking, or deciding. I think this is the key difference.
This is something I see and feel everyday. And it’s very frustrating and annoying.
For example I send some doc asking for a feedback and someone without reading it generate a feedback with llm with so much ambiguity that I have to get back and wait couple of more days to get a reply.
One of the most silliest thing I see is a middle manager feeding Microsoft planner to Claude to generate a report and generate future steps and sometimes it makes no sense what he present couple of weeks ago because what he present today is contradicting to the one before.
At this point I feel it’s cheaper to replace them with AI. They are just physical vessels for AI.
It’s just not that maybe they were not good enough. But now they just fully depend on AI.
Humans have been offloading thinking to something/some entity forever. From gods to influencers.
AI is the current popular way (at least in these circles) and if it's too much or too little it might not matter. What will matter is if this offloading is making people unhappy and having any negative impact in civilization. I have no idea
I think this is absolutely an issue.
The rise of knowledge work made many people far less physically active because moving one's body was no longer a given part of one's job. This led to a lot of people (who assumed sports was exercise on top of one's work, not the only source of exercise) moving very little. This meant we needed to rediscover the importance of exercise as a pillar of health.
I think something similar will happen with knowledge work, where we have to do a lot less cognitive exercise due to AI (as well as the decline of reading and rise of short-form video), which will likely lead to eventual issues and subsequently, a rise in activities designed to replicate the cognitive exercise work used to provide.
Our brains take time to ingest things and learn and using an AI tool to make decisions when taking in vast inputs of data requires then an overview of those decisions and understanding to some degree of care. If you don't take the time to go over these decisions and understand them and weigh them and course correct as needed, then you are offloading too much. You will become this approver buffer in between claude and nothing more if you don't have this methodical checking and understanding.
What is frightening is with something like neuralink that in a future hypothetical time would have very fast capability to keep informed and advised, you could be a zombie decision maker and nobody would really be able to tell. Even when you were pressed to why you made the decision, it's just another AI response, it's like a con artist or imposter dream scenario.
I noticed that atm, before these crazy hypotheticals potentially happen, the people that seem to take the time to understand things deeper are still way more valuable than those that just use tools more than not. Its obvious atm due to the lag in time and the way people respond in meetings, at least for now. :)
I saw a post the other day pulling a Frank Herbert quote from Dune about men offloading their thinking to machines and being controlled by the men who controlled the machines.
Somebody asked an AI how to interpret it.
To answer the title's question directly? Yes, I think so. Or at least that it is a legitimate hazard.
I think the analogy to hyper-palatable, calorific food works well:
Humans adapted for a world with too little food. Then once there was too much, obesity and overeating became a problem for the first time. Self discipline is the cost we have to pay for this kind of abundance.
We now have a general-purpose way to offload mental effort, and are discovering in real-time the negative consequences of that.
I use AI for coding, but I feel I've moved past the honey-moon period and am now learning how to use it in a way that is not a detriment. If I care about the work I'm doing I don't want the AI to do it for me, even if it could. Deciding what work I want/should be doing myself vs what can be delegated is a new skill I, and I believe we all need to learn.
Maybe, let me ask my coding agent what he thinks about this.
Just like calculators became crutches for school children memorising their times tables and our phones contacts list a crutch for memorising phone numbers, I see AI becoming a crutch for many, in all the areas we seem important and humanity has spent years refining (communication, thinking, writing, etc)
It's a well done and thought-provoking article.
The reality is that most humans do very little actual thinking of their own anyway, and, if you believe that what LLMs produce constitutes a form of intelligence, it does seem "more intelligent" than most humans.
So: is outsourcing thinking a net improvement for a majority of users?
I use several models, daily, and they seem "reasonably conditioned" that they are only input to my thinking and not "my thinking". I correct them constantly; they are wrong (in reasoning/logic, in actual facts) frequently. They are demonstrably "not smarter" than I am. And yet I know many people who can "do more" with them as a "thinking" tool. I can say that "the problem" is they can't spot the errors, but they can't or won't do that in their ordinary lives, either, so, again, is it a net improvement for them?
Interesting times and all that.
Almost definitely Id think - I think we have to treat it something like power tools / chairs and a gym.
AI makes me massively more productive as a quant, and more creative in the sense that it can often find calculations I don't know how to start, BUT the flip side of that is that I can also feel my skills atrophy and as such am trying to make sure I do maths exercises and so on. I don't worry about programming skills because programming isn't about code.
I do feel like I'm offloading thinking to an AI, but I think that's a good thing. I envision a world where users and AI are aligned without corporate interference. AI lets me offload things that I don't need to know and frees up my brain to push farther than I could before. At least that's how it feels to me.
>AI lets me offload things that I don't need to know and frees up my brain to push farther than I could before.
How can you push your brain go farther than ever, when you don't use it for the basic task?
Higher Math does not work without understanding "lower" Math, running long runs does not work without starting on shorter runs. Thinking about complicated staff will probly not work, if you can't think about the easy stuff.
One can not learn a language without vocabulary and skipping learning verbs in a foreign language, because dictionaries exists does not bring one closer to being able to speak.
I agree that when you're building mathematical understanding, you have to do it brick by brick. But the fact of the matter is it's been decades since I did a long division problem by hand. AI lets me jump into areas that I would previously have had to spend a month studying before I could begin building - That's what I mean by it lets me reach farther, faster.
You don't always have to do division by hand. I would make the argument that it's probably good to do it occasionally. At least enough to remember how.
I don't think offloading things that you "think" you don't need to know is a realistic thing. Instead this seems like some slippery slope of intellectual degradation where slowly you'll replace more and more parts of the thought process with AI which ends in some rather sad existence.
I agree, I suspect that when we say "oh, I don't need to know this", what we really mean is we have that unpleasant feeling behind our eyes that says this is not going to gratify my goal
The issue being, gratification is rarely a good guideline. It just means collapsing the gap between doing the thing and the idea of having done the thing. But that gap is where you actually learn things
Is a good thing depending on which ones. And our taste on what to specialize our judgment into varies a lot. One thing I do use as criteria to detect it can be a problem is in lack of understanding and lack of control of the layers that are part of verifications and diagnosing power.
Related discussion: Outsourcing thinking (270 points, 5 months ago) https://news.ycombinator.com/item?id=46840865
I'm probably not the only person that types math problems into google search because I'm too lazy to open up a calculator (or do the math myself). So yes, given I do that, I probably am susceptible to offloading some of my thinking to AI.
Are CEOs outsourcing too much of their thinking to their employees?
I find I'm not thinking less per say, just thinking about different things. Maybe you could argue there are CEOs who get too far out of touch with the reality on the ground and should get more directly involved in the work. However, I don't know how well one could argue that the CEO should do all of the work.
I see at least the current iteration LLMs and harnesses as me managing and coordinating them and thinking differently, not less.
Some of us are, yes.
I've noticed it when interviewing interns. A surprising number seem unable to think on their feet or solve problems without immediately reaching for chatgpt. I don't necessarily expect you to be able to solve a problem entirely without tools, but you should be able to give me the outline of how to go about something and why you would go that way.
After all, if you are just going to spit out AI, I will just get AI to do your job...
It really depends on which angle you look at it. Is it purely to meet a business goal? Or is it also for personal growth? I think it's a mix of both, but for me it's always important that I engage mentally with the process, learn something, and solve puzzles, even if that involves letting the AI take care of the coding, which is an abstraction. You could still code and not think creatively.
The value of thinking in language is going down, the value of acting on your dreams is going up
Decisions are special things. One of the golden rules of life is that a person (or entity) making decisions is somehow impacted or otherwise getting feedback on the repercussions of those decisions.
When you cognitively surrender to AI, or to another person (be it a leader/manager, or a subordinate/report), you are asking for trouble.
This is the kind of headline I would expect from one of those news that appear as you progress in cookie clicker.
I don't think we are offloading thinking to AI. We just started to use it. AI is a tool useful to write text, for a fast searching, for the boring work. Personally I don't take the first answer, I like to challenge, to ask why, to tell it's wrong.
as a six-figure salary software engineer haver I find that a lot of colleagues take their cushy jobs for granted.
these are the people who are now first against the wall when the revolution comes.
why would I want to hire you to work the three hours you feel like working per day (per week?) when I can have AI with a deeper knowledge set available 24x7?
Yes, I think we are . And because of super competition, most of us are trying to make AI (agentic) do the work.
Personally, I use AI to learn more about Backend Engineering actually, so it's fine for me. Beside I can also use AI to suggest and it's me verifying the idea so that's a no for me
How much of the thinking is involved in asking the right question, versus coming to the correct answer? I don't have a real answer to that but it does seem to be worth considering.
imo no way
But this varies from person to person
Some of us overthink already and offloading to AI just enables us to overthink more in other directions than we would if we didn't have ai
I know people like that - the amount of inane, obsessive and just strange conversations they have with AI is concerning - there's never any actually useful result or information that they get out of these chats.
Furthermore, there are some clearly wrong questions where person asks AI to make some kind of numerical evaluation of some data. And evaluation is done entirely through inference - essentially a hallucination, instead of some one-off python script which can actually give deterministic calculated evaluation. Yet they accept the answer AI gives them.
Ironically I just caught myself offloading the thinking about this article to the comment section before I read it
Having a very dangerous AI standoff at work, where people are debating wether or not to use a particular connection pooling / threading strategy to fix a production issue, and everyone is unqualified to answer and is instead arguing what their agent said.
They are just straight up admitting they don't know anything, and advocate fiercely for their agent's recommendation.
No one cares, no one tries to stop this behavior. It's seen as good, apparently. I admitted that I don't know enough to have an opinion at the moment, I certainly don't know how to judge the contradictory opinions of multiple frontier AIs, and I fear that just made me look incompetent.
Run both. Benchmark them. Performance is notoriously difficult to predict and much easier to test. If you have a load balancer, run the new strategy on one or two servers and see how their throughput compares to the others.
Exactly! It got so easy nowadays to use AI to setup scenarios since the busy work of writing code/test harnesses and setting things up for the benchmark is done by the machine. Then throw away what does not work.
Some benchmark that would take weeks to plan, code and set up is now hours and days - the time is now spent on the benchmark itself, not on temporary code.
Seriously its easy to build prototypes with AI and benchmark...
It's not quite as easy to simulate weird behaviors that emerge across clouds and on prem data centers.
I don’t know where all these people work that think simulating production load is easy
Not an AI defender but this doesn't sound all that different than before where people were just arguing for their understanding of the issue. No one knows everything and has perfect alignment with reality. I'm with the other guy who responded, test it, else y'all just guessing and arguing who's better at guessing.
honestly sounds like you have too many unqualified employees. best case scenario though, they all come out of this having learned a little bit more.
This is a load-bearing title.
The answer to this question is: Politicians, not you!
Perhaps the question to ask is: who is making all of the final decisions for the things that really matter to you in your life?
No direct democracy, just people deciding for you. You can choose once every four years. Are we surprised of how easily we delegate decisions? May be AI can do it better
You’d lose your ability to interview at a different workplace.
I'd rather hire a clankers these days
If you are then yes. See also Am I relying on the recommendation algorithm too much to decide for me on <platform>
I like the colonialism conversation.
@grok is this true?
I am reminded of the old Zizek remark about letting two sex toys fuck each other so you and your partner are freed up to go about and do other things.
If you offload any of your thinking to AI, you're offloading too much of your thinking to AI.
Offload your execution, not your thinking.
I liked the ending well said
Yes.
Honestly, using AI helps me get more done in a day because I can delegate some decision making to it, usually inconsequential stuff.
Yes that seems pretty clear. We also need quick adaptation in school and university to avoid cheating. Back to closed book exams I guess, home projects not counting for the final score. It sucks but I don´t see how you can´t avoid removing technology for evaluation. Sure people can use AI to boost their ability to get better results but they will have to prove it the old way with no assisted help and people watching them during exams.
> Side note: his startup is replacing human engineers by capturing their every input and operation, but without their explicit consent.
...huh. It's a "startup", so it's not Meta capturing their employees' inputs. I wonder what it could be.
LLMs are strongly mean reverting in their decision making, so heavy users are likely to be readily identified by their unadventurous conformism much as generated media is identified now.
I don't think AI causes people to stop thinking. Rather, I think it biases people toward doing what they want to do more, and that bias is what becomes problematic.
In fact, when I use AI, I don't really use it for the things I actually enjoy doing. For example, I like making UI animations, and I don't use AI for that. I also don't use AI when I'm playing games I enjoy. But when I have to make something tedious like a login screen, I use AI. And after I write the code, I just throw the entire codebase at AI to write the documentation.
The problem is that this only lets me think about things I have a taste for.
Having taste and diving deep into it is good. Immersion is great. But on the flip side, you also need to do things that aren't your taste. That's more cognitively healthy. AI prevents that.
In that sense, I think AI's strength is that it creates an environment where you can dive deeper into the areas you like.
But the real question becomes how you use the cognitive surplus that's left after offloading tasks to AI.
I visit Korean, Chinese, Japanese, and USA sites, and honestly, most people, including myself, only have deep thinking about certain topics. Outside of those, we just follow the prevailing opinion.
So I'm not really sure. I don't think using AI makes me stop thinking. I just think it creates a bias that makes my thinking only focus on the parts I want to focus on.
Absolutely. Calculators made us lazy. GPS made us dumb. LLMs are turning us into idiots. Because idiots have the power to damage themselves and others
I’ve found that when I ask AI to do something for me that I know how to do myself - but would rather not spend the time doing - there is a not insignificant chance that the AI will return a subpar result, which I can usually tell rather quickly. Either by glancing at the code, or trying to compile it and getting an error.
This happens frequently enough that it creates a real disincentive for me to use AI for anything that I already know how to do - and use it exclusively for things I don’t know how to do.
It’s deeply frustrating to realize you just wasted 20 minutes posting error messages into Claude when you could’ve just locked in and written it yourself.
Yes
Who is "we"? Is there a mouse in your pocket?
The royal "we"! You know, the editorial "we"!
"we" no. just allows me to think about the stuff that matters.
and work on things that would usually be out of my element.
if you aren't thinking more than ever, you're using ai wrong.
The normal application of Betteridge's law of headlines is "no" but I think this is one of the rare instances where it's "yes."
yes
Yawn who cares.
Most people just wanna enjoy life and then die.
Very few actually get to a position of influence and power to actually do anything. Much of economic activity isn’t critical to life and death - it’s just stuff we have to make life more interesting than past generations.
is the pope catholic?
does a bear shit in the woods?
does rust have the worst community of all time?
> does rust have the worst community of all time?
Of all time? Nah.
That would be either emacs of vi. I'll let them fight with each other over which!
Looking at your posting history, you really dislike Rust, huh?
Consider the following scenario. You're a beginner or with very little experience go on a popular programming forum to ask a question, in the typical way someone with little experience asks, what do you get? An answer? No, you get bad manners, insults, "you don't know how to ask questions", "RTFM". At best you'll get some people challenging you to clarify and refine your question, which you can't do because you don't know the technology well enough.
Annoyed, you go find a popular programming chatbot, and ask the question. The chatbot will give you answer, no matter how poorly worded or nonsensical your question, and it will do it cheerfully and confidently. It may even tell you how great your question or idea is. Granted, the answer will be worthless, both because the question was poorly worded and because the chatbot is simply spewing statistically probable text, but you won't know. You're a beginner, without experience to know correct from bullshit. You try to use the answer the chatbot gave you, and when it doesn't work, you go back to the chatbot. It will continue to cheerfully answer your questions as long as you have tokens to spend. The chatbot will never give up on trying to help you, it will never be rude, it won't complain.
And people wonder why chatbots are so popular.