This is a must-read series of articles, and I think Kyle is very much correct.
The comparison to the adoption of automobiles is apt, and something I've thought about before as well. Just because a technology can be useful doesn't mean it will have positive effects on society.
That said, I'm more open to using LLMs in constrained scenarios, in cases where they're an appropriate tool for the job and the downsides can be reasonably mitigated. The equivalent position in 1920 would not be telling individuals "don't ever drive a car," but rather extrapolating critically about the negative social and environmental effects (many of which were predictable) and preventing the worst outcomes via policy.
But this requires understanding the actual limits and possibilities of the technology. In my opinion, it's important for technologists who actually see the downsides to stay aware and involved, and even be experts and leaders in the field. I want to be in a position to say "no" to the worst excesses of AI, from a position of credible authority.
> Just because a technology can be useful doesn't mean it will have positive effects on society.
You say it in a way that it sounds like automobiles don't have a positive effect. I don't agree - they have some negative effects but overall they have a vast net positive effect for everyone.
Their negative effects are much more vast, subtle, and cultural. You could say many of the broad and widespread mental issues we have in the US is the result of automobiles leading to suburbanization and thus isolation of people. It has created an expensive barrier of entry for existing in society and added a ton of friction to doing anything and everything, especially with people. That's not even getting into the climate effects.
The upsides of automobiles generally all exist outside of the 'personal automobile', i.e. logistics. These upsides and downsides don't need to coexist. We could reap the benefits without needing to suffer for it, but here we are.
> You could say many of the broad and widespread mental issues we have in the US is the result of automobiles leading to suburbanization and thus isolation of people.
Yes, you could say that, though I'm not sure who would actually say that seriously.
Respectfully, without judgement, your perspective may be wildly skewed because you’re American (going by your post history). I suspect the negative externalities in a society built around cars don’t register with you because to you it is the normal state of the world. As a Dutchman, I grew up in a built world that is based around the human scale and to me your parent’s claim comes across as astonishingly obvious.
This is a willfully ignorant and wildly incorrect take. Your isolation argument completely neglects socialization with family and friends that is supported via automotive mobility. Do you also somehow have the impression that automobiles somehow forced suburbanization? I think not- you don't want others to have the freedom to choose anything other than some industrialized urban existence. The effects of the automobile are vast, subtle, and cultural- and overwhelmingly positive
The upsides of automobiles, or personal mobility in general, are enormous. I can go wherever I want, whenever I want along with other people and cargo. I don't have to wait for a schedule set by someone else, or worry about union strikes. I love my cars!
this is so funny, even when trying to talk positively about cars you cant help but throw in a 'fuck you, I got mine'. Unions are cool, and good for workers. Enjoy your weekend! Thank a union.
Those are all enormous benefits to you and you alone. The greatest thing about cars are the things they do for you.
In order for someone else to have those benefits, they also need a car.
If as a society, if we could feel the same way about public transit, bike lanes, sidewalks, that you do about your own personal vehicle - we'd be better off.
I'm hardly alone, there are millions and millions of us. But the HN bubble skews toward affluent childless male urbanites, so discussions here tend to be weirdly disconnected from the real world that regular middle-class Americans experience.
They have a net positive effect for every owner, except that they seem to facilitate and encourage ways of living that require automobile ownership as a condition of adulthood in most places. So I'm not entirely sure they're a vast net positive in every value system. In yours, yes, but not in mine.
There is very little connection between ownership and who does the driving. I still want to own my own cars even if a computer does most of the driving. That way it's always available, and more importantly I can keep my own stuff in it.
This is such a fascist take- "they seem to facilitate and encourage ways of living that require automobile ownership as a condition of adulthood" i.e. I don't agree with that way of living so I wish others didn't have the freedom to choose it
It's not fine if that choice denies other people the choice not to.
And there seems to be a lot of the latter.
For example, when shopping facilities or hospitals are built so as to be, de-facto, only accessible by automobile, that locks people out of the choice to say no thanks.
I don't follow, are people then not able to choose to live somewhere that has shopping facilities or hospitals that are built so as not to be only accessible by automobile?
Automobiles are one of a key pillar of logistics. Getting things (food, medicine, construction materials, etc. etc.) to and from backbones like rail, harbors, airports etc. So even for those who don't own a vehicle or even want to own a vehicle, automobiles are still a vast net positive.
I'm not sure what the alternative would be. Maybe everyone lives in giant 10 million+ population cities that are all connected to each other by rail (and rail connects all airports, harbors, etc.) and then you have to show up at rail station to get your groceries or whatever else?
Personal cars are not the same as using them for logistics.
Yes cars/trucks/busses are still useful overall and are an incredible last-mile solution for freight.
But on a personal level, it means we all must live far apart and maintain our own individual vehicles, along with the average total costs of $11,500/year PER CAR. [0]
I’m not saying they should’ve even been banned for personal use - owning a car and living in a rural suburb should still be an option, but it is very expensive to choose that lifestyle.
However the auto companies on the early to mid 1900s had heavy influence on policy, even buying and shutting down their public transit competitors, converting cities into “car cities”. This is where it drove into “negatives outweigh the positives” territory. Everything before that was more positive, but this was a massive negative on society and continues to handicap cities today, making them expensive and even just dangerous to walk around (due to high speed roads and limited sidewalks)
The amount of space in US cities (broadly, out into their sprawl) that is used up by cars is incredible and serves to make other modes of transportation (to include things like busses, even) less-useful and make cars on-par with or worse than things like bicycles once you take out the time spent traveling these inflated distances, ~50% of which distance typically exists because of cars, and the time spent working to pay for your car, to say nothing of then needing to dedicate more time specifically to working out (or just accept being less healthy) because you're not walking or bicycling as much as you could be in a world where cars hadn't sprawled everything really far apart with gigantic parking lots, half-mile-diameter highway interchanges, large barely-used front lawns to provide distance from unpleasant and loud roads, big unusable "green space" buffers from highways, et c.
Once you start really marking how much nothing you're driving by even in many cities, where that "nothing" is one or another use of land that exists solely because of cars, it's a bit of a shock. "Wait, work would only be 8 miles away instead of 15 if not for the effects of widespread private car ownership? The grocery store could be 1 mile instead of 3? And I spend how much time a week bicycling to nowhere in particular to make up for sitting all day long? And this car & gas & insurance costs me how many of my work-hours per week, just to pay for it? Hm... am I... losing time to cars!?"
You don't get highways and the interstate system if vehicles are not for personal use. And if you don't get those, you don't get the modern logistics system.
I guess what I don't understand is, given the current state, 1) what do you want? 2) how much will it cost? (and how will we pay for it?) and 3) what are the tradeoffs?
On a related note, it seems like a lot of the anti-car/urban planning wonks have a belief that everyone really wants to walk, ride bikes, or take mass transit everywhere, and I think they're wrong. Most people want to drive personal vehicles.
Maybe if we lived in a world where mass transit had very strictly enforced behavioral norms, more would consider it. But even then, I still think most people prefer the many conveniences afforded by personal vehicles.
> Personal cars are not the same as using them for logistics.
Yes, they are in fact, the same. You cannot introduce such massively useful technology into the world and then say that it would be used only for logistics and not for personal transportation. Short of a worldwide totalitarian government, such arbitrary restriction would be completely unenforceable.
It is possible to shape things with regulation, but only to some degree. With any great technology, you have to take the good with the bad. And the good outweights the bad in any historical technology. AI will be no exception.
Sure, on your own land, just like you can drive more-or-less whatever you want as long as you stick to your own property, today, including vehicles that aren't "street legal".
On public roads? No reason we'd have to license private cars for that, at least not for just any purpose.
How about the fact that any country that tries to ban private ownership of cars would completely fall behind in all car-related technologies, infrastructure and services, which would very soon negatively affect all those commercial or logistical use cases that our civilization vitally depends on?
Trying to ban all private cars while keeping our car-dependent civilization working is unrealistic, no matter how you look at it.
Exactly. These arguments are all buttressed by the "if everyone would just..." argument [1]. In fact, everyone will not just. And so if you want to build your Utopia, it will have to be compelled by force.
I've always lived in walkable cities. I don't own a car and with pollution, congestion, accident risk, pavement obstruction, etc. other people's cars unequivocally make my life worse.
We can argue about whether this is a good trade off, but the claim that cars make everyone's life better is straightforwardly false.
Troll post? No, they do not "unequivocally" make your life worse. "Other people's cars" facilitate thousands of aspects of modern living and society that you apparently take for granted. You can choose to ONLY look at the negative impacts, but the comment as stated is ridiculous.
The only way you receive food (except from your backyard inner-city garden?) is through people DRIVING. The way you receive packages is by DRIVING. They city infrastructure you enjoy is maintained through skilled laborers and tradespeople DRIVING.
Troll post? You state that "other people's cars" facilitate thousands of aspects of modern living, then go on to talk about things that trucks do, not personal vehicles
I don't think it's possible to clearly separate personal vehicles from commercial ones. The technology is the same. Any regulation that tries to ban the one while allowing the other would be a huuuge clusterfuck.
I mean sure, they both have engines and wheels, but they're already distinguishable in the eyes of the law. Commercial and personal vehicles are registered separately
Anyway, I don't think anyone is proposing banning cars. Just would be good to provide alternatives
There's a difference between personal vehicles and special purpose vehicles like ambulances and delivery trucks. I don't think anyone in this thread is saying all automobiles are bad, but car-centric development is definitely bad. You don't have to theorize from first principles about this. There are many places around the world that aren't as locked into the personal car as the US is, and they are still functioning societies where you can receive food, packages, medicine, workers maintain infrastructure, etc.
> "Other people's cars" facilitate thousands of aspects of modern living and society that you apparently take for granted. You can choose to ONLY look at the negative impacts, but the comment as stated is ridiculous.
THIS! I am shocked that some people don't realize that modern civilization and our modern quality of life depends on cars to a huge degree, even for people don't personally drive. Such a lack of knowledge about modern industry and logistics..
In aggregate, benefits of cars outweight the cons for 99% of people. Perhaps if you live right next to a busy highway, you might the the exception..
> THIS! I am shocked that some people don't realize that modern civilization and our modern quality of life depends on cars to a huge degree, even for people don't personally drive. Such a lack of knowledge about modern industry and logistics..
I'm more shocked that somebody thinks that modern civilization and logistics depend on personal cars. Can ypu expand on your statement that modern industry and logistic depend on persobal cars?
The positive effects were immediate, and measurable. The negative effects are delayed, and hard to quantify without all the advancement in climate research since then. If everyone in 1920 knew a 100 years from now there would be climate crisis to reckon with, perhaps a few things would have changed along the way.
Today we have a much better understanding of the world, so we have the means to think down the line of what the negative effects of LLMs and course correct if needed.
The problem is we are numb to it. 40,000+ people are killed in car accidents every year in just the USA. Wars are started over oil and accepted by the people so they can keep paying less at the pump. Microplastics entering the environment each day along with particulate from brakes, and exhaust. Speaking of exhaust: global warming. Even going electric just shifts the problems as we need to dig up lithium, the new oil. We still have to drill for oil for plastics and metal refining, recycling and fabrication.
I think it's most obvious in hindsight, probably it was a long time (some decades) before cars were understood to have much of a negative effect at all. Nobody* thought much about air pollution (even adding lead to the gasoline) or climate effects, or what would happen when cities were built enough that they were then dependent on cars, or what happens when gas or cars gets expensive.
All they saw was that trips taking a day could now be done in an hour and produced no manure, and that meant suddenly you could reasonably go to many more places. What's not to like? A model T was cheap, and you didn't even need to worry about insurance or having a driver's license. Surely nobody would drive so carelessly as to crash.
*well, not technically nobody, but nobody important.
If you read the period news, pretty much everything except lead poisoning and climate change was well known by the 1920s. Rich people wanted cars but a ton of places had resistance from everyone else to what they correctly recognized as removing the public spaces they used and shifting externalities to, for example, the people being hit by cars.
What’s really interesting is that you can find newspaper columns in the 1920s recognizing what we now call induced demand as even by then it was clear that adding road capacity simply inspired more people to drive.
Cars I'd argue are a net negative for everyone. In the article it goes over this pretty well.
The automobile was a revolutionary tool, but I think it has been overprescribed as a solution for the problem of transportation.
The grips of capitalism and consumerism have allowed for automobiles to become a requirement for living nearly everywhere in America except for the densest of areas.
I love cars, I enjoy working on them, driving them, the way they look, the way they sound and feel. They do offer a freedom that is unparalleled, and offer many benefits to those who truly need those guarantees.
Ultimately, to me they are a symbol of toxic individualism. I would be happy if we could move on from them as a society.
The term "toxic individualism" doesn't mean that individualism is inherently toxic, like "toxic masculinity" doesn't mean that about masculinity in the general case. These terms mean the over-expression of their worst aspects.
You are taking the statement of "toxic individualism" to mean "all individualism is toxic" rather than "certain parts of individualism can become toxic if not followed."
It is possible to say "some things could be done better" without meaning "throw it all away."
I fear that outside of cataclysmic global warfare or some sort of butlerian jihad (which amounts to the same) this genie is not going back into the bottle.
This tech is 100% aligned with the goals of the 0.001% that own and control it, and almost all of the negatives cited by Kyle and likeminded (such as myself) are in fact positives for them in context of massive population reduction to eliminate "useless eaters" and technological societal control over the "NPCs" of the world that remain since they will likely be programmed by their peered AI that will do the thinking for them.
So what to do entirely depends on whether you feel we are responsible to the future generations or not. If the answer is no, then what to do is scoped to the personal concerns. If yes, we need a revolution and it needs to be global.
It can't. It can't even deal with emails without randomly deleting your email folder [1]. Saying that it can make decisions and replace humans is akin of saying that random number generator can make decisions and can replace people.
It's just an automation tool, and just like all automation tools before it it will create more jobs than destroy. All the CEOs' talks about labor replacement are a fuss, a pile of lies to justify layoffs and worsening financial situation.
Yes, but that isn’t the question as long as those wealthy people control most of the system: companies aren’t going to lose executives, they’ll shed the jobs which they don’t respect. Someone wealthy does not need to accept a bad deal to avoid sleeping on thr street. It’s everyone who isn’t insulated who has to actually compete for work.
I'm tempted to (bitterly) point out that feeling responsible for future generations was already off the table decades ago when we decided to ignore our ecological footprints.
Not to collapse the economic system, but to present a credible threat of collapsing the economic system which AI development, as these elite and their platforms know it, relies on. When they're freaking out, we call for negotiations.
This only works if people with "secure" livelihoods not just participate, but drive the effort. Getting paid six figures or more in a layoff-proof position? Cool, you need to be the first person walking out the door on May 1st (or whenever this happens), and the first person at the bank counter requesting your max withdrawal.
You're free to take a vacation or quit working if you want to. Go ahead.
As for bank runs, no one cares. The big banks no longer need retail customer deposits as a source of capital for fractional reserve lending. Modern bank funding mechanisms are more sophisticated than that.
Geopolitical realities and considerations require that the effort is synchronized and global. Assume great power X's society revolts and decides to reign in the financial and technological barons and lords, and do away with such things. Meanwhile, great powers Y, Z etc. are not doing this and one day people in X will wake up to AI drone swarms of these powers taking them over and they're back to square 1 and now not even a great power.
Collective humanity needs to think this matter through and take global action. This is the only way I fear, short of natural calamities (act of God) that unplugs humanity from advanced tech for a few generations again.
This reminds me a bit of the ending of In the Beginning Was the Command Line:
> The people who brought us this operating system would have to provide templates and wizards, giving us a few default lives that we could use as starting places for designing our own. Chances are that these default lives would actually look pretty damn good to most people, good enough, anyway, that they'd be reluctant to tear them open and mess around with them for fear of making them worse. So after a few releases the software would begin to look even simpler: you would boot it up and it would present you with a dialog box with a single large button in the middle labeled: LIVE. Once you had clicked that button, your life would begin. If anything got out of whack, or failed to meet your expectations, you could complain about it to Microsoft's Customer Support Department. If you got a flack on the line, he or she would tell you that your life was actually fine, that there was not a thing wrong with it, and in any event it would be a lot better after the next upgrade was rolled out. But if you persisted, and identified yourself as Advanced, you might get through to an actual engineer.
> What would the engineer say, after you had explained your problem, and enumerated all of the dissatisfactions in your life? He would probably tell you that life is a very hard and complicated thing; that no interface can change that; that anyone who believes otherwise is a sucker; and that if you don't like having choices made for you, you should start making your own.
Two years ago, I was enjoying a drink with my wite, her friend, a very senior female VC partner, and another friend.
Somehow we talked AI in some depth, and the VC at one point said (about AI): “I don’t know what are kids are going to do for work. I don’t know what jobs there will be to do.”
That same VC invests in AI companies and by what I heard about her, has done phenomenally well.
I think about that exchange all the time. Worried about your own kids but acting against their interests. It unsettled me, and Kyle’s excellent articles brought that back to a boiling point in my mind.
To take the car analogy: it matters how we use the car.
The car in itself can be used to save time and energy that would otherwise be used to walk to places. That extra time and energy can be used well, or poorly.
- It can be squandered by having a longer commute that defeats the point
- Alternatively, it can be wasted by sitting on a couch consuming Netflix or TikTok
- Alternatively, it can be used productively, by playing team sports with friends, or chasing your kids through the park, or building a chicken coop in your back yard
It’s all about wise usage. Yes it can be used as a way to destroy your own body and waste your time and attention, but also it can be used as a tool to deploy your resources better, for example in physical activities that are fun and social rather than required drudgery.
I think it’s the same for LLMs. Managers and executives have always delegated the engineering work, and even researching and writing reports. It matters whether we find places to continue to challenge and deploy our cognition, or completely settle back, delegate everything to the LLM and scroll TikTok while it works.
I agree with you that the majority of people will use it to feed their attention and energy to the attention economy. Meta will be more profitable than ever, as will TikTok, Netflix, YouTube
But the majority have always chosen the path of least resistance. This is not new! Socrates’ famous exhortation is “the unexamined life is not worth living”. People were living mindlessly on autopilot before TikTok.
I think if you want to give a call to action, as this piece does, the right call to action is “think carefully about how you can make a good use of your time and energy, now that the default path has changed.” I know it’s not as simple or emotionally powerful as “go down kicking and screaming, stick it to the man”, but as a rule of thumb, the less fiercely emotional path is usually the right one.
I agree with the general sentiment that the structure of society is going to change, but I don't know what the satisfying solution is. It's hard to imagine not participating will work, or even be financially viable for me, for long.
I agree. I'm the AI luddite on my team of red team security engineers, but I'm still using it in very limited use cases. As much as I disagree with how the guardrails around AI are being handled, I still need to use it to stay relevant in my field and not get canned.
I'm using claude but then refuse to do much cleaning up of what it spews. Im leaving that for the PR reviewers who love AI and going through slop. If they want slop, I'll give them the slop they want.
Not advocating that people should follow this but:
As someone that loves cleaning up code, I'm actually asking the vibe coders in the team (designer, PM and SEO guy) to just give me small PRs and then I clean up instead of reviewing. I know they will just put the text back in code anyway, so it's less work for me to refactor it.
With a caveat: if they give me >1000 lines or too many features in the same PR, I ask them to reduce the scope, sometimes to start from scratch.
And I also started doing this with another engineer: no review cycle, we just clean up each other's code and merge.
I'm honestly surprised at how much I prefer this to the traditional structure of code reviews.
Additionally, I don't have to follow Jira tickets with lengthy SEO specs or "please change this according to Figma". They just the changes themselves and we go on with our lives.
Just started work on a project. Greenfield and "AI accelerated". PRs diffs are in the range of 10s of thousands of lines. In the PR, it is suggested to not actually read all the code as it would take too long.
If you push a change, or you approve, you're responsible for the change and its effects later. Regardless of size. If change is too big, tell your teammates its too big to review and to refactor to bite-size with their great coding agents. Use AI models also for review of large changes, consider a checklist . Setup CI and integration tests (also can be AI assisted)
based. our CEO has made it clear that we're expected to use LLMs to shit out as many features as we can as quickly as we can, so that's exactly what I'm doing. Can't wait to watch leadership flail around in a year or two when the long term consequences start to become apparent
That's exactly it. This person does not understand the coercive competition of the market. If you don't use new tech, you are going to be undercut by people who do. And every HR dept is going to expect to to have experience with AI even if the department that’s hiring doesn't really use it. If the author's supposed solution to the problem has negative personal consequences, why would you do it? To be nice?
Because I don't like the feeling my conscience gives me by doing something I think is evil and bad. Some people have moral lines that they won't cross when finding jobs.
If my competitors are filling their flour with sawdust, guess I got to just do the same?
No. I'm doing it because I care more whether I can live with myself than whether I impress people with the name of who I work for. Hence much of my recent comment history here, for example. I don't want any of these people getting the idea they should want me to work with them, either. I do want my name on every industry blacklist I can possibly get it on. Those will eventually be revealed - remember Franklin's dictum, fellas! That shit always comes out in the end - and I look forward to that day with pleased and eager anticipation.
At the moment I'm more looking at menial work for one of the local universities. Money is money, and my needs are small; the work is honest, I still should have a decade or so of physical labor left in me, and it carries the perk of free tuition for the degree I never had time for. I would have the time and energy to write, perhaps, even! And, however badly the people in charge are running things lately, the world will always need someone good at cleaning a toilet. (And I am already pretty good at cleaning a toilet!)
I've been thinking about this a lot recently, and I don't know if it is possible to stop. I've been thinking the most impactful thing would be to create open-source tools to make it easier to build agents on top of open-source models. We have a few open-source models now, maybe not as good as Gemini, but if the agent were sufficiently good, could that compensate?
I think that would democratize some of the power. Then again, I haven't been super impressed with humanity lately and wonder if that sort of democratization of power would actually be a good thing. Over the last few years, I've come to realize that a lot of people want to watch the world burn, way more than I had imagined. It is much easier to destroy than to build. If we make it easier for people to build agents, is that a net positive overall?
> If we make it easier for people to build agents, is that a net positive overall?
If we make it easier for people to drive and have cars, isn't that a net positive? If we make it easier for X, isn't that better? No, not necessarily, that's the entire point of this series of essays. Friction is good in some cases! You can't learn without friction. You can't have sex without friction.
> ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand: the cultivation of what James C. Scott would call
Imagine being starting university now... I can't imagine to have learned what I did at engineering school if it wasn't for all the time lost on projects, on errors. And I can't really think that I would have had the mental strength required to not use LLMs on course projects (or side projects) when I had deadlines, exams coming, yet also want to be with friends and enjoy those years of your life.
The epilogue looked weak to me. The previous sections explored why it was essentially wrong to use current LLM technology, the answers can be wrong, or not even wrong, and why it has to be that way. The epilogue focus more in (our) obsolescence in a paradigm shift towards widespread LLM use scenario and not in them doing their work right or wrong.
And that should be the core. There is a new, emergent technology, should we throw everything away and embrace it or there are structural reasons on why is something to be taken with big warning labels? Avoiding them because they do their work too well may be a global system approach, but decision makers optimize locally, their own budget/productivity/profit. But if they are perceived risks, because they are not perfect, that is another thing.
The reasons laid out in this article are why it's so important to share how we are using AI and what we are getting in return. I've been trying to contribute towards a positive outcome for AI by tracking how well the big AI companies are doing at being used to solve humanitarian problems. I can't really do most of the suggestions the article, they seem like a way to slow progress. I don't want to slow AI progress, I want the technology we already have to be deployed for useful and helpful things.
The comparison to automobiles changing streets is thrown around a lot. But I feel AI is fundamentally different. It is not a technological change like the internet which brought us huge amounts of opportunities in so many different directions. AI’s goal is to automate (in other words, replace) us.
the epilogue is what speaks to me most. all of the work I've done with llms takes that same kind of approach. I never link them to a git repo and I only ever ask them to make specific, well-formatted changes so that I can pick up where they left off. my general feelings are that LLMs make the bullshit I hate doing a lot easier - project setup, integrate themeing, prepare/package resources for installability/portability, basic dependency preparation (vite for js/ts, ui libs for c#, stuff like that), ui layout scaffolding (main panel, menu panel, theme variables), auto-update fetch and execute loops, etc...
and while I know they can do the nitty gritty ui work fine, I feel like I can work just as fast, or faster, on UI without them than I can with them. with them it's a lot of "no, not that, you changed too much/too little/the wrong thing", but without them I just execute because it's a domain I'm familiar with.
So my general idea of them is that they are "90% machines". Great at doing all of the "heavy lifting" bullshit of initial setup or large structural refactoring (that doesn't actually change functionality, just prepares for it) that I never want to do anyway, but not necessary and often unhelpful for filling in that last 10% of the project just the way I want it.
of course, since any good PM knows that 90% of the code written only means 50% of the project finished (at best), it still feels like a hollow win. So I often consider the situation in the same way as that last paragraph. Am I letting the ease of the initial setup degrade my ability to setup projects without these tools? does it matter, since project setup and refactoring are one-and-done, project-specific, configuration-specific quagmires where the less thought about fiddly perfect text-matching, the better? can I use these things and still be able to use them well (direct them on architechture/structure) if I keep using them and lose grounded concepts of what the underlying work is? good questions, as far as I'm concerned.
As a consequentialist who shares the author's concerns, I feel fine (ethically) using AI without advancing it. Foregoing opportunities meaningful to yourself for deontological reasons when it won't have any impact on society is pointless.
Rudolph built his engine, Henry built his car, Popular Mechanics published it. 2000 biofueling stations across the nation. All made illegal by special interests months before the article was published. Information didn't move fast enough to let the editors know that innovation was illegal.
From the article: "I’ve thought about this a lot over the last few years, and I think the best response is to stop. ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand: the cultivation of what James C. Scott would call metis."
"What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking-there's the real danger" - Frank Herbert, God Emperor of Dune
> ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand
On one hand I intuitively think this is correct, on the other hand these very concerns about technology have been around since the invention of... writing.
Here is an excerpt of Socrates speaking on the written word, as recorded in Plato's dialogue Phaedrus - "For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom"
And you know, Socrates was right. We did lose our memory with writing! How many phone numbers do you remember now that you have a phonebook in your phone? Humans will lose skills due to LLMs. That's just obvious on its face by the fact that if you don't do a skill regularly, you will lose it (or lose to do it as well as you once had).
There's a distinction to be made between "worse off" and "worse". Socrates was arguing that writing-users would be worse as people, not that they would experience lives they didn't like as much.
> "What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking-there's the real danger" - Frank Herbert, God Emperor of Dune
I always preferred this take:
“Civilization advances by extending the number of important operations which we can perform without thinking of them.”
― Alfred North Whitehead
It's both opposite and complementary to your Frank Herbert quote.
I think it's important that we recognize and understand how those operations are being done, and ignorance of the complexity of all the parts of our lives leads to the death of expertise. People who would learn a lot just from reading the course description of a 100 level class in a field are assuming their lack of knowledge means there's no complexity there.
> “There is a cult of ignorance in the United States, and there has always been. The strain of anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge.” ― Isaac Asimov
The easier society makes it to be unaware of the complexity of everything around us, the easier it becomes to assume everything is actually as simple as their surface-level understanding.
Also Frank Herbert: "Once, men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."
I mean, people are talking about the butlerian jihad without any sense of irony or subtext. Dune is literally a feudal hellscape that takes place in the wake of that event. It didn’t make things better. Lmao
The idea that Claude might be able to help you change the color of your led lighting as a legitimate counter to things like a less usable world wide web, worse government services, the loss of human ability, etc. is excellent parody.
I read that as an example of how we're seduced into using things - we start small because surely this one small thing won't hurt. And then it becomes one more thing. And one more. It'll start with him using it to change the color of his lights and 5 years from now AI will be embedded in his life.
completely fair, and I agree. but let's talk 6 months/a year down the line - when a local LLM will be able to offer what claude code does only slower and a smaller context window. then do you whip out the local llm to handle the project, or is it still objectionable?
The front page is currently home to the announcement of Qwen 3.6 35B, which has comparable performance to the flagship coding models of a few months ago, and can be run at home by those with a gaming computer or MBP from the last five years. It is happening, but there will always be some lag.
Yes, but every time the capabilities, security, accuracy, or any other quality of LLMs is challenged, the default answer is that we'll essentially have AGI in a quarter or two. It's very tiring to try to argue with people about current quality, when the argument is always to wait and/or pay for a super expensive model.
That's not what the grandparent poster was saying, but sure. They have been steadily improving across those metrics, as Opus 4.6 / 4.7 / Mythos demonstrate. They're certainly not perfect, and I understand your fatigue (it is certainly fatiguing to follow, even if interested!), but each new release pushes it that bit further, and the improvements percolate downwards to the cheaper models.
The Industrial Revolution - the greatest thing ever to happen - required the British govt to deploy more troops against Luddites than they had fighting Napoleon at the same time.
Damaging machinery was made a capital offense and they had dozens of executions, hundreds of deportations.
At every stage, the steady progress of civilization is fragile and in danger of being suffocated. Its opponents cloak themselves in moral righteousness, call themselves luddites, the green party, or AI safety rationalists. Its all the same corrosive thing underneath.
This kind of black and white moral thinking is corrosive to one's intelligence. You're allowed to talk about who benefits from massive society change and who suffers. You are allowed to talk about the ways that technology is implemented and how that leads to pros and cons. An attitude of "if we ever stop moving forward and think then the evil bad people win" is deeply anti-intellectual.
I read couple of articles in the series and I still couldn't get what was the point author is trying to make. Reads like, "let me give you 100 arguments why I think this is bad".
Do LLMs lie? Of course not, they are just programs. Do the make mistakes or get the facts wrong? Of course they do, not more often then a human does. So what is the point of that article? Why my future is particularly bad now because of LLMs?
The argument isn't that LLMs are bad because they can hallucinate. Author (clearly) argues that LLM use has negative cognitive effects on their users and on society as a whole. Plus, the technology would wipe out a large, large number of jobs.
How can you argue they don't lie, as if they have any idea of correct vs wrong? There is no brain there. When statistics overwhelmingly say "yes" is the correct answer to something, it will say "yes" -- completely independent of whether that's the correct answer.
Complaining about AI slop is starting to become its own kind of slop. There isn't anything novel in this little essay. It might as well have been written by AI because I've seen this type of dude complain about this exact type of thing countless times at this point, and none of them have a solution other than empty moralizing or call your representative or whatever. None of that’s going to work. Fortune, Gizmodo, The Verge,Ars Technica, etc. all circulate the same negative headlines and none of them have a solution, and their writers are probably going to be totally replaced by AI so what difference does it make? They're just capitalizing on the negative sentiment and they have no intention to come up with a solution. At that point it's just complaining and I'm sick of it.
Agreed, and I think if you asked most people in the developed world, they'd say the invention of automobiles has been a net positive (to say the least) despite all the very real negatives. Stopped reading the article after that. It seems like the people expressing these sentiments are a loud minority, and I know from having spent way too much time online that if LLMs didn't exist in their current form, they'd be angry about something else. Then again, Maybe I'm just out of touch. It's a distinct possibility.
Frankly I think it’s kind of childish to just put up a massive Uk wide block on your website. “Call your representatives”, ok dude, can I give you a list of things I want to change about your country’s policies?
A statement that can be reversed onto the speaker without effort is meaningless. It has no content. It just means, "I am rational and you are not." Ok, then.
This is a must-read series of articles, and I think Kyle is very much correct.
The comparison to the adoption of automobiles is apt, and something I've thought about before as well. Just because a technology can be useful doesn't mean it will have positive effects on society.
That said, I'm more open to using LLMs in constrained scenarios, in cases where they're an appropriate tool for the job and the downsides can be reasonably mitigated. The equivalent position in 1920 would not be telling individuals "don't ever drive a car," but rather extrapolating critically about the negative social and environmental effects (many of which were predictable) and preventing the worst outcomes via policy.
But this requires understanding the actual limits and possibilities of the technology. In my opinion, it's important for technologists who actually see the downsides to stay aware and involved, and even be experts and leaders in the field. I want to be in a position to say "no" to the worst excesses of AI, from a position of credible authority.
> Just because a technology can be useful doesn't mean it will have positive effects on society.
You say it in a way that it sounds like automobiles don't have a positive effect. I don't agree - they have some negative effects but overall they have a vast net positive effect for everyone.
Their negative effects are much more vast, subtle, and cultural. You could say many of the broad and widespread mental issues we have in the US is the result of automobiles leading to suburbanization and thus isolation of people. It has created an expensive barrier of entry for existing in society and added a ton of friction to doing anything and everything, especially with people. That's not even getting into the climate effects.
The upsides of automobiles generally all exist outside of the 'personal automobile', i.e. logistics. These upsides and downsides don't need to coexist. We could reap the benefits without needing to suffer for it, but here we are.
> You could say many of the broad and widespread mental issues we have in the US is the result of automobiles leading to suburbanization and thus isolation of people.
Yes, you could say that, though I'm not sure who would actually say that seriously.
Respectfully, without judgement, your perspective may be wildly skewed because you’re American (going by your post history). I suspect the negative externalities in a society built around cars don’t register with you because to you it is the normal state of the world. As a Dutchman, I grew up in a built world that is based around the human scale and to me your parent’s claim comes across as astonishingly obvious.
I would say that seriously, so there you go, theres two.
This is a willfully ignorant and wildly incorrect take. Your isolation argument completely neglects socialization with family and friends that is supported via automotive mobility. Do you also somehow have the impression that automobiles somehow forced suburbanization? I think not- you don't want others to have the freedom to choose anything other than some industrialized urban existence. The effects of the automobile are vast, subtle, and cultural- and overwhelmingly positive
The upsides of automobiles, or personal mobility in general, are enormous. I can go wherever I want, whenever I want along with other people and cargo. I don't have to wait for a schedule set by someone else, or worry about union strikes. I love my cars!
this is so funny, even when trying to talk positively about cars you cant help but throw in a 'fuck you, I got mine'. Unions are cool, and good for workers. Enjoy your weekend! Thank a union.
Private employee unions are cool. Public employee unions are a cancer on society.
Those are all enormous benefits to you and you alone. The greatest thing about cars are the things they do for you.
In order for someone else to have those benefits, they also need a car.
If as a society, if we could feel the same way about public transit, bike lanes, sidewalks, that you do about your own personal vehicle - we'd be better off.
I'm hardly alone, there are millions and millions of us. But the HN bubble skews toward affluent childless male urbanites, so discussions here tend to be weirdly disconnected from the real world that regular middle-class Americans experience.
[dead]
They have a net positive effect for every owner, except that they seem to facilitate and encourage ways of living that require automobile ownership as a condition of adulthood in most places. So I'm not entirely sure they're a vast net positive in every value system. In yours, yes, but not in mine.
Ironically, AI facilitates self-driving cars, which promise to _reduce_ the need for private automobile ownership.
There is very little connection between ownership and who does the driving. I still want to own my own cars even if a computer does most of the driving. That way it's always available, and more importantly I can keep my own stuff in it.
This is such a fascist take- "they seem to facilitate and encourage ways of living that require automobile ownership as a condition of adulthood" i.e. I don't agree with that way of living so I wish others didn't have the freedom to choose it
It's fine if people choose it.
It's not fine if that choice denies other people the choice not to.
And there seems to be a lot of the latter.
For example, when shopping facilities or hospitals are built so as to be, de-facto, only accessible by automobile, that locks people out of the choice to say no thanks.
I don't follow, are people then not able to choose to live somewhere that has shopping facilities or hospitals that are built so as not to be only accessible by automobile?
Automobiles are one of a key pillar of logistics. Getting things (food, medicine, construction materials, etc. etc.) to and from backbones like rail, harbors, airports etc. So even for those who don't own a vehicle or even want to own a vehicle, automobiles are still a vast net positive.
I'm not sure what the alternative would be. Maybe everyone lives in giant 10 million+ population cities that are all connected to each other by rail (and rail connects all airports, harbors, etc.) and then you have to show up at rail station to get your groceries or whatever else?
Personal cars are not the same as using them for logistics.
Yes cars/trucks/busses are still useful overall and are an incredible last-mile solution for freight.
But on a personal level, it means we all must live far apart and maintain our own individual vehicles, along with the average total costs of $11,500/year PER CAR. [0]
I’m not saying they should’ve even been banned for personal use - owning a car and living in a rural suburb should still be an option, but it is very expensive to choose that lifestyle.
However the auto companies on the early to mid 1900s had heavy influence on policy, even buying and shutting down their public transit competitors, converting cities into “car cities”. This is where it drove into “negatives outweigh the positives” territory. Everything before that was more positive, but this was a massive negative on society and continues to handicap cities today, making them expensive and even just dangerous to walk around (due to high speed roads and limited sidewalks)
[0] https://www.nerdwallet.com/auto-loans/learn/total-cost-ownin...
The amount of space in US cities (broadly, out into their sprawl) that is used up by cars is incredible and serves to make other modes of transportation (to include things like busses, even) less-useful and make cars on-par with or worse than things like bicycles once you take out the time spent traveling these inflated distances, ~50% of which distance typically exists because of cars, and the time spent working to pay for your car, to say nothing of then needing to dedicate more time specifically to working out (or just accept being less healthy) because you're not walking or bicycling as much as you could be in a world where cars hadn't sprawled everything really far apart with gigantic parking lots, half-mile-diameter highway interchanges, large barely-used front lawns to provide distance from unpleasant and loud roads, big unusable "green space" buffers from highways, et c.
Once you start really marking how much nothing you're driving by even in many cities, where that "nothing" is one or another use of land that exists solely because of cars, it's a bit of a shock. "Wait, work would only be 8 miles away instead of 15 if not for the effects of widespread private car ownership? The grocery store could be 1 mile instead of 3? And I spend how much time a week bicycling to nowhere in particular to make up for sitting all day long? And this car & gas & insurance costs me how many of my work-hours per week, just to pay for it? Hm... am I... losing time to cars!?"
You don't get highways and the interstate system if vehicles are not for personal use. And if you don't get those, you don't get the modern logistics system.
I guess what I don't understand is, given the current state, 1) what do you want? 2) how much will it cost? (and how will we pay for it?) and 3) what are the tradeoffs?
On a related note, it seems like a lot of the anti-car/urban planning wonks have a belief that everyone really wants to walk, ride bikes, or take mass transit everywhere, and I think they're wrong. Most people want to drive personal vehicles.
Maybe if we lived in a world where mass transit had very strictly enforced behavioral norms, more would consider it. But even then, I still think most people prefer the many conveniences afforded by personal vehicles.
> Personal cars are not the same as using them for logistics.
Yes, they are in fact, the same. You cannot introduce such massively useful technology into the world and then say that it would be used only for logistics and not for personal transportation. Short of a worldwide totalitarian government, such arbitrary restriction would be completely unenforceable.
It is possible to shape things with regulation, but only to some degree. With any great technology, you have to take the good with the bad. And the good outweights the bad in any historical technology. AI will be no exception.
Sure, on your own land, just like you can drive more-or-less whatever you want as long as you stick to your own property, today, including vehicles that aren't "street legal".
On public roads? No reason we'd have to license private cars for that, at least not for just any purpose.
How about the fact that any country that tries to ban private ownership of cars would completely fall behind in all car-related technologies, infrastructure and services, which would very soon negatively affect all those commercial or logistical use cases that our civilization vitally depends on?
Trying to ban all private cars while keeping our car-dependent civilization working is unrealistic, no matter how you look at it.
We pretty much did with aviation.
Exactly. These arguments are all buttressed by the "if everyone would just..." argument [1]. In fact, everyone will not just. And so if you want to build your Utopia, it will have to be compelled by force.
[1] https://x.com/eperea/status/1803815983154434435
I've always lived in walkable cities. I don't own a car and with pollution, congestion, accident risk, pavement obstruction, etc. other people's cars unequivocally make my life worse.
We can argue about whether this is a good trade off, but the claim that cars make everyone's life better is straightforwardly false.
Troll post? No, they do not "unequivocally" make your life worse. "Other people's cars" facilitate thousands of aspects of modern living and society that you apparently take for granted. You can choose to ONLY look at the negative impacts, but the comment as stated is ridiculous.
The only way you receive food (except from your backyard inner-city garden?) is through people DRIVING. The way you receive packages is by DRIVING. They city infrastructure you enjoy is maintained through skilled laborers and tradespeople DRIVING.
Troll post? You state that "other people's cars" facilitate thousands of aspects of modern living, then go on to talk about things that trucks do, not personal vehicles
I don't think it's possible to clearly separate personal vehicles from commercial ones. The technology is the same. Any regulation that tries to ban the one while allowing the other would be a huuuge clusterfuck.
> The technology is the same
I mean sure, they both have engines and wheels, but they're already distinguishable in the eyes of the law. Commercial and personal vehicles are registered separately
Anyway, I don't think anyone is proposing banning cars. Just would be good to provide alternatives
There's a difference between personal vehicles and special purpose vehicles like ambulances and delivery trucks. I don't think anyone in this thread is saying all automobiles are bad, but car-centric development is definitely bad. You don't have to theorize from first principles about this. There are many places around the world that aren't as locked into the personal car as the US is, and they are still functioning societies where you can receive food, packages, medicine, workers maintain infrastructure, etc.
> "Other people's cars" facilitate thousands of aspects of modern living and society that you apparently take for granted. You can choose to ONLY look at the negative impacts, but the comment as stated is ridiculous.
THIS! I am shocked that some people don't realize that modern civilization and our modern quality of life depends on cars to a huge degree, even for people don't personally drive. Such a lack of knowledge about modern industry and logistics..
In aggregate, benefits of cars outweight the cons for 99% of people. Perhaps if you live right next to a busy highway, you might the the exception..
> THIS! I am shocked that some people don't realize that modern civilization and our modern quality of life depends on cars to a huge degree, even for people don't personally drive. Such a lack of knowledge about modern industry and logistics..
I'm more shocked that somebody thinks that modern civilization and logistics depend on personal cars. Can ypu expand on your statement that modern industry and logistic depend on persobal cars?
I said cars not driving. Yes, the supermarket needs trucks to deliver the food. It doesn't need cars.
Ambulances: good. Trucks: good. Busses, even: good.
Cars? Waaaay less clear they're net-beneficial.
The positive effects were immediate, and measurable. The negative effects are delayed, and hard to quantify without all the advancement in climate research since then. If everyone in 1920 knew a 100 years from now there would be climate crisis to reckon with, perhaps a few things would have changed along the way.
Today we have a much better understanding of the world, so we have the means to think down the line of what the negative effects of LLMs and course correct if needed.
> I don't agree - they have some negative effects
The problem is we are numb to it. 40,000+ people are killed in car accidents every year in just the USA. Wars are started over oil and accepted by the people so they can keep paying less at the pump. Microplastics entering the environment each day along with particulate from brakes, and exhaust. Speaking of exhaust: global warming. Even going electric just shifts the problems as we need to dig up lithium, the new oil. We still have to drill for oil for plastics and metal refining, recycling and fabrication.
I think it's most obvious in hindsight, probably it was a long time (some decades) before cars were understood to have much of a negative effect at all. Nobody* thought much about air pollution (even adding lead to the gasoline) or climate effects, or what would happen when cities were built enough that they were then dependent on cars, or what happens when gas or cars gets expensive.
All they saw was that trips taking a day could now be done in an hour and produced no manure, and that meant suddenly you could reasonably go to many more places. What's not to like? A model T was cheap, and you didn't even need to worry about insurance or having a driver's license. Surely nobody would drive so carelessly as to crash.
*well, not technically nobody, but nobody important.
If you read the period news, pretty much everything except lead poisoning and climate change was well known by the 1920s. Rich people wanted cars but a ton of places had resistance from everyone else to what they correctly recognized as removing the public spaces they used and shifting externalities to, for example, the people being hit by cars.
What’s really interesting is that you can find newspaper columns in the 1920s recognizing what we now call induced demand as even by then it was clear that adding road capacity simply inspired more people to drive.
Cars I'd argue are a net negative for everyone. In the article it goes over this pretty well.
The automobile was a revolutionary tool, but I think it has been overprescribed as a solution for the problem of transportation.
The grips of capitalism and consumerism have allowed for automobiles to become a requirement for living nearly everywhere in America except for the densest of areas.
I love cars, I enjoy working on them, driving them, the way they look, the way they sound and feel. They do offer a freedom that is unparalleled, and offer many benefits to those who truly need those guarantees.
Ultimately, to me they are a symbol of toxic individualism. I would be happy if we could move on from them as a society.
We need to replace the frigidity of collectivism with the warmth of rugged individualism.
OK Ayn Rand.
I never want to live in a society that views individualism as toxic
Nobody said that that individualism is toxic
The term "toxic individualism" doesn't mean that individualism is inherently toxic, like "toxic masculinity" doesn't mean that about masculinity in the general case. These terms mean the over-expression of their worst aspects.
You are taking the statement of "toxic individualism" to mean "all individualism is toxic" rather than "certain parts of individualism can become toxic if not followed."
It is possible to say "some things could be done better" without meaning "throw it all away."
You’re equivocating, your parent specifically named an example of toxic individualism, they did not say or imply that individualism is toxic.
I guess, if you feel that freedom of movement is insignificant
Saying that a type of individualism is toxic, doesn't mean that all individualism is toxic. Did adjectives change somehow?
[dead]
All blocked in the UK, sadly.
I fear that outside of cataclysmic global warfare or some sort of butlerian jihad (which amounts to the same) this genie is not going back into the bottle.
This tech is 100% aligned with the goals of the 0.001% that own and control it, and almost all of the negatives cited by Kyle and likeminded (such as myself) are in fact positives for them in context of massive population reduction to eliminate "useless eaters" and technological societal control over the "NPCs" of the world that remain since they will likely be programmed by their peered AI that will do the thinking for them.
So what to do entirely depends on whether you feel we are responsible to the future generations or not. If the answer is no, then what to do is scoped to the personal concerns. If yes, we need a revolution and it needs to be global.
> to eliminate "useless eaters"
It can't. It can't even deal with emails without randomly deleting your email folder [1]. Saying that it can make decisions and replace humans is akin of saying that random number generator can make decisions and can replace people.
It's just an automation tool, and just like all automation tools before it it will create more jobs than destroy. All the CEOs' talks about labor replacement are a fuss, a pile of lies to justify layoffs and worsening financial situation.
[1] https://www.pcmag.com/news/meta-security-researchers-opencla...
> This tech is 100% aligned with the goals of the 0.001% that own and control it
If AI is smart enough to replace the 99.999% it's also smart enough to replace the 0.001%.
Yes, but that isn’t the question as long as those wealthy people control most of the system: companies aren’t going to lose executives, they’ll shed the jobs which they don’t respect. Someone wealthy does not need to accept a bad deal to avoid sleeping on thr street. It’s everyone who isn’t insulated who has to actually compete for work.
No because the technology will be used against you.
That fact doesn’t prevent the 0.001% from continuing to control it.
I have given this serious thought over the years. I even have an unfinished novel exactly around that topic.
Energy. The key is controlling their access to energy.
I'm tempted to (bitterly) point out that feeling responsible for future generations was already off the table decades ago when we decided to ignore our ecological footprints.
Gonna beat this drum till it breaks:
Not to collapse the economic system, but to present a credible threat of collapsing the economic system which AI development, as these elite and their platforms know it, relies on. When they're freaking out, we call for negotiations.This only works if people with "secure" livelihoods not just participate, but drive the effort. Getting paid six figures or more in a layoff-proof position? Cool, you need to be the first person walking out the door on May 1st (or whenever this happens), and the first person at the bank counter requesting your max withdrawal.
You're free to take a vacation or quit working if you want to. Go ahead.
As for bank runs, no one cares. The big banks no longer need retail customer deposits as a source of capital for fractional reserve lending. Modern bank funding mechanisms are more sophisticated than that.
Geopolitical realities and considerations require that the effort is synchronized and global. Assume great power X's society revolts and decides to reign in the financial and technological barons and lords, and do away with such things. Meanwhile, great powers Y, Z etc. are not doing this and one day people in X will wake up to AI drone swarms of these powers taking them over and they're back to square 1 and now not even a great power.
Collective humanity needs to think this matter through and take global action. This is the only way I fear, short of natural calamities (act of God) that unplugs humanity from advanced tech for a few generations again.
This reminds me a bit of the ending of In the Beginning Was the Command Line:
> The people who brought us this operating system would have to provide templates and wizards, giving us a few default lives that we could use as starting places for designing our own. Chances are that these default lives would actually look pretty damn good to most people, good enough, anyway, that they'd be reluctant to tear them open and mess around with them for fear of making them worse. So after a few releases the software would begin to look even simpler: you would boot it up and it would present you with a dialog box with a single large button in the middle labeled: LIVE. Once you had clicked that button, your life would begin. If anything got out of whack, or failed to meet your expectations, you could complain about it to Microsoft's Customer Support Department. If you got a flack on the line, he or she would tell you that your life was actually fine, that there was not a thing wrong with it, and in any event it would be a lot better after the next upgrade was rolled out. But if you persisted, and identified yourself as Advanced, you might get through to an actual engineer.
> What would the engineer say, after you had explained your problem, and enumerated all of the dissatisfactions in your life? He would probably tell you that life is a very hard and complicated thing; that no interface can change that; that anyone who believes otherwise is a sucker; and that if you don't like having choices made for you, you should start making your own.
Two years ago, I was enjoying a drink with my wite, her friend, a very senior female VC partner, and another friend.
Somehow we talked AI in some depth, and the VC at one point said (about AI): “I don’t know what are kids are going to do for work. I don’t know what jobs there will be to do.”
That same VC invests in AI companies and by what I heard about her, has done phenomenally well.
I think about that exchange all the time. Worried about your own kids but acting against their interests. It unsettled me, and Kyle’s excellent articles brought that back to a boiling point in my mind.
I don’t think this is the right take.
To take the car analogy: it matters how we use the car.
The car in itself can be used to save time and energy that would otherwise be used to walk to places. That extra time and energy can be used well, or poorly.
- It can be squandered by having a longer commute that defeats the point
- Alternatively, it can be wasted by sitting on a couch consuming Netflix or TikTok
- Alternatively, it can be used productively, by playing team sports with friends, or chasing your kids through the park, or building a chicken coop in your back yard
It’s all about wise usage. Yes it can be used as a way to destroy your own body and waste your time and attention, but also it can be used as a tool to deploy your resources better, for example in physical activities that are fun and social rather than required drudgery.
I think it’s the same for LLMs. Managers and executives have always delegated the engineering work, and even researching and writing reports. It matters whether we find places to continue to challenge and deploy our cognition, or completely settle back, delegate everything to the LLM and scroll TikTok while it works.
While I agree in principle, I don’t know how much faith is warranted in humans using it wisely in practice.
I agree with you that the majority of people will use it to feed their attention and energy to the attention economy. Meta will be more profitable than ever, as will TikTok, Netflix, YouTube
But the majority have always chosen the path of least resistance. This is not new! Socrates’ famous exhortation is “the unexamined life is not worth living”. People were living mindlessly on autopilot before TikTok.
I think if you want to give a call to action, as this piece does, the right call to action is “think carefully about how you can make a good use of your time and energy, now that the default path has changed.” I know it’s not as simple or emotionally powerful as “go down kicking and screaming, stick it to the man”, but as a rule of thumb, the less fiercely emotional path is usually the right one.
I have a lot of faith they will use it unwisely.
Indeed.
I agree with the general sentiment that the structure of society is going to change, but I don't know what the satisfying solution is. It's hard to imagine not participating will work, or even be financially viable for me, for long.
I agree. I'm the AI luddite on my team of red team security engineers, but I'm still using it in very limited use cases. As much as I disagree with how the guardrails around AI are being handled, I still need to use it to stay relevant in my field and not get canned.
I'm already adding "Agentic Workflows" as a skill in my LinkedIn profile. Cringed hard at that, but oh well...
What if the hiring managers at the jobs you'd actually prefer to work at also cringe when they see it on your profile?
It's becoming so ubiquitous, I highly doubt it. At worst I think a manager would just see it as fluff, but not a negative.
I hope the hiring managers I would actually want to work for would see it as a red flag on resumes
That's actually a really good point.
I'm using claude but then refuse to do much cleaning up of what it spews. Im leaving that for the PR reviewers who love AI and going through slop. If they want slop, I'll give them the slop they want.
Not advocating that people should follow this but:
As someone that loves cleaning up code, I'm actually asking the vibe coders in the team (designer, PM and SEO guy) to just give me small PRs and then I clean up instead of reviewing. I know they will just put the text back in code anyway, so it's less work for me to refactor it.
With a caveat: if they give me >1000 lines or too many features in the same PR, I ask them to reduce the scope, sometimes to start from scratch.
And I also started doing this with another engineer: no review cycle, we just clean up each other's code and merge.
I'm honestly surprised at how much I prefer this to the traditional structure of code reviews.
Additionally, I don't have to follow Jira tickets with lengthy SEO specs or "please change this according to Figma". They just the changes themselves and we go on with our lives.
Just started work on a project. Greenfield and "AI accelerated". PRs diffs are in the range of 10s of thousands of lines. In the PR, it is suggested to not actually read all the code as it would take too long.
If you push a change, or you approve, you're responsible for the change and its effects later. Regardless of size. If change is too big, tell your teammates its too big to review and to refactor to bite-size with their great coding agents. Use AI models also for review of large changes, consider a checklist . Setup CI and integration tests (also can be AI assisted)
I thought the de facto policy was that the individual remains responsible in a team context.
Sweet summer child.
based. our CEO has made it clear that we're expected to use LLMs to shit out as many features as we can as quickly as we can, so that's exactly what I'm doing. Can't wait to watch leadership flail around in a year or two when the long term consequences start to become apparent
> when the long term consequences start to become apparent
Choose your own story!
and then a) programmers become relevant again and slowly fix all this crap, b) Claude 7.16 waltz through fixing things as it goes.
You'll just get laid off and they'll be onto the next hype cycle as visionaries.
That's exactly it. This person does not understand the coercive competition of the market. If you don't use new tech, you are going to be undercut by people who do. And every HR dept is going to expect to to have experience with AI even if the department that’s hiring doesn't really use it. If the author's supposed solution to the problem has negative personal consequences, why would you do it? To be nice?
Because I don't like the feeling my conscience gives me by doing something I think is evil and bad. Some people have moral lines that they won't cross when finding jobs.
If my competitors are filling their flour with sawdust, guess I got to just do the same?
No. I'm doing it because I care more whether I can live with myself than whether I impress people with the name of who I work for. Hence much of my recent comment history here, for example. I don't want any of these people getting the idea they should want me to work with them, either. I do want my name on every industry blacklist I can possibly get it on. Those will eventually be revealed - remember Franklin's dictum, fellas! That shit always comes out in the end - and I look forward to that day with pleased and eager anticipation.
At the moment I'm more looking at menial work for one of the local universities. Money is money, and my needs are small; the work is honest, I still should have a decade or so of physical labor left in me, and it carries the perk of free tuition for the degree I never had time for. I would have the time and energy to write, perhaps, even! And, however badly the people in charge are running things lately, the world will always need someone good at cleaning a toilet. (And I am already pretty good at cleaning a toilet!)
That's nice for you but other people have kids to feed and don't particularly care about your little crusade, which will fail.
Go look in a mirror, not at me. That's where the argument you're feeling urged toward is waiting.
I've been thinking about this a lot recently, and I don't know if it is possible to stop. I've been thinking the most impactful thing would be to create open-source tools to make it easier to build agents on top of open-source models. We have a few open-source models now, maybe not as good as Gemini, but if the agent were sufficiently good, could that compensate?
I think that would democratize some of the power. Then again, I haven't been super impressed with humanity lately and wonder if that sort of democratization of power would actually be a good thing. Over the last few years, I've come to realize that a lot of people want to watch the world burn, way more than I had imagined. It is much easier to destroy than to build. If we make it easier for people to build agents, is that a net positive overall?
> If we make it easier for people to build agents, is that a net positive overall?
If we make it easier for people to drive and have cars, isn't that a net positive? If we make it easier for X, isn't that better? No, not necessarily, that's the entire point of this series of essays. Friction is good in some cases! You can't learn without friction. You can't have sex without friction.
> "Unavailable Due to the UK Online Safety Act. Now might be a good time to call your representatives."
Having the "call your representatives" link be to your website as well isn't particularly helpful... I already can't get to it
https://web.archive.org/web/20260416134911/https://aphyr.com...
OP should link to https://www.writetothem.com/
I love it when Americans take the moral high ground
> ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand: the cultivation of what James C. Scott would call
Imagine being starting university now... I can't imagine to have learned what I did at engineering school if it wasn't for all the time lost on projects, on errors. And I can't really think that I would have had the mental strength required to not use LLMs on course projects (or side projects) when I had deadlines, exams coming, yet also want to be with friends and enjoy those years of your life.
The epilogue looked weak to me. The previous sections explored why it was essentially wrong to use current LLM technology, the answers can be wrong, or not even wrong, and why it has to be that way. The epilogue focus more in (our) obsolescence in a paradigm shift towards widespread LLM use scenario and not in them doing their work right or wrong.
And that should be the core. There is a new, emergent technology, should we throw everything away and embrace it or there are structural reasons on why is something to be taken with big warning labels? Avoiding them because they do their work too well may be a global system approach, but decision makers optimize locally, their own budget/productivity/profit. But if they are perceived risks, because they are not perfect, that is another thing.
The reasons laid out in this article are why it's so important to share how we are using AI and what we are getting in return. I've been trying to contribute towards a positive outcome for AI by tracking how well the big AI companies are doing at being used to solve humanitarian problems. I can't really do most of the suggestions the article, they seem like a way to slow progress. I don't want to slow AI progress, I want the technology we already have to be deployed for useful and helpful things.
The comparison to automobiles changing streets is thrown around a lot. But I feel AI is fundamentally different. It is not a technological change like the internet which brought us huge amounts of opportunities in so many different directions. AI’s goal is to automate (in other words, replace) us.
the epilogue is what speaks to me most. all of the work I've done with llms takes that same kind of approach. I never link them to a git repo and I only ever ask them to make specific, well-formatted changes so that I can pick up where they left off. my general feelings are that LLMs make the bullshit I hate doing a lot easier - project setup, integrate themeing, prepare/package resources for installability/portability, basic dependency preparation (vite for js/ts, ui libs for c#, stuff like that), ui layout scaffolding (main panel, menu panel, theme variables), auto-update fetch and execute loops, etc...
and while I know they can do the nitty gritty ui work fine, I feel like I can work just as fast, or faster, on UI without them than I can with them. with them it's a lot of "no, not that, you changed too much/too little/the wrong thing", but without them I just execute because it's a domain I'm familiar with.
So my general idea of them is that they are "90% machines". Great at doing all of the "heavy lifting" bullshit of initial setup or large structural refactoring (that doesn't actually change functionality, just prepares for it) that I never want to do anyway, but not necessary and often unhelpful for filling in that last 10% of the project just the way I want it.
of course, since any good PM knows that 90% of the code written only means 50% of the project finished (at best), it still feels like a hollow win. So I often consider the situation in the same way as that last paragraph. Am I letting the ease of the initial setup degrade my ability to setup projects without these tools? does it matter, since project setup and refactoring are one-and-done, project-specific, configuration-specific quagmires where the less thought about fiddly perfect text-matching, the better? can I use these things and still be able to use them well (direct them on architechture/structure) if I keep using them and lose grounded concepts of what the underlying work is? good questions, as far as I'm concerned.
As a consequentialist who shares the author's concerns, I feel fine (ethically) using AI without advancing it. Foregoing opportunities meaningful to yourself for deontological reasons when it won't have any impact on society is pointless.
Some people like roasting marshmallows. Others think that setting the house on fire may have downsides.
If there's too many lies, "source or gtfo" becomes more important
you would have to trust that the person listening to the lies would know the difference, and that's the rub...
that's the neat part, the source is also going to be bullshit slop!
Therefore, you can dismiss whatwever claim is being made. That's the reason to ask for the source: so you can judge whether it's reliable.
Rudolph built his engine, Henry built his car, Popular Mechanics published it. 2000 biofueling stations across the nation. All made illegal by special interests months before the article was published. Information didn't move fast enough to let the editors know that innovation was illegal.
I'm genuinely trying to understand this comment. Can you /explain
From the article: "I’ve thought about this a lot over the last few years, and I think the best response is to stop. ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand: the cultivation of what James C. Scott would call metis."
"What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking-there's the real danger" - Frank Herbert, God Emperor of Dune
> ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand
On one hand I intuitively think this is correct, on the other hand these very concerns about technology have been around since the invention of... writing.
Here is an excerpt of Socrates speaking on the written word, as recorded in Plato's dialogue Phaedrus - "For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom"
And you know, Socrates was right. We did lose our memory with writing! How many phone numbers do you remember now that you have a phonebook in your phone? Humans will lose skills due to LLMs. That's just obvious on its face by the fact that if you don't do a skill regularly, you will lose it (or lose to do it as well as you once had).
The real question is whether we're worse off or better off overall than we were in Socrates' Athens.
There's a distinction to be made between "worse off" and "worse". Socrates was arguing that writing-users would be worse as people, not that they would experience lives they didn't like as much.
> "What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking-there's the real danger" - Frank Herbert, God Emperor of Dune
I always preferred this take:
“Civilization advances by extending the number of important operations which we can perform without thinking of them.” ― Alfred North Whitehead
It's both opposite and complementary to your Frank Herbert quote.
It's very clear to me that many people have achieved peak civilization, and stopped thinking.
I guess it hinges on your definition of "civilization".
I think it's important that we recognize and understand how those operations are being done, and ignorance of the complexity of all the parts of our lives leads to the death of expertise. People who would learn a lot just from reading the course description of a 100 level class in a field are assuming their lack of knowledge means there's no complexity there.
> “There is a cult of ignorance in the United States, and there has always been. The strain of anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge.” ― Isaac Asimov
The easier society makes it to be unaware of the complexity of everything around us, the easier it becomes to assume everything is actually as simple as their surface-level understanding.
Also Frank Herbert: "Once, men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."
I mean, people are talking about the butlerian jihad without any sense of irony or subtext. Dune is literally a feudal hellscape that takes place in the wake of that event. It didn’t make things better. Lmao
The idea that Claude might be able to help you change the color of your led lighting as a legitimate counter to things like a less usable world wide web, worse government services, the loss of human ability, etc. is excellent parody.
It's way too real, that's just how humans tend to work. Short-term personal benefit almost always outweighs long-term societal cost.
I read that as an example of how we're seduced into using things - we start small because surely this one small thing won't hurt. And then it becomes one more thing. And one more. It'll start with him using it to change the color of his lights and 5 years from now AI will be embedded in his life.
It's the first step on the road to hell.
completely fair, and I agree. but let's talk 6 months/a year down the line - when a local LLM will be able to offer what claude code does only slower and a smaller context window. then do you whip out the local llm to handle the project, or is it still objectionable?
It's already YEARS down the line from when this was promised, we can't keep saying "but in a couple more quarters it'll all be different!".
The front page is currently home to the announcement of Qwen 3.6 35B, which has comparable performance to the flagship coding models of a few months ago, and can be run at home by those with a gaming computer or MBP from the last five years. It is happening, but there will always be some lag.
Yes, but every time the capabilities, security, accuracy, or any other quality of LLMs is challenged, the default answer is that we'll essentially have AGI in a quarter or two. It's very tiring to try to argue with people about current quality, when the argument is always to wait and/or pay for a super expensive model.
That's not what the grandparent poster was saying, but sure. They have been steadily improving across those metrics, as Opus 4.6 / 4.7 / Mythos demonstrate. They're certainly not perfect, and I understand your fatigue (it is certainly fatiguing to follow, even if interested!), but each new release pushes it that bit further, and the improvements percolate downwards to the cheaper models.
Claude Code was released in February 2025, how can it have been years since we were promised competitive local models?
The Industrial Revolution - the greatest thing ever to happen - required the British govt to deploy more troops against Luddites than they had fighting Napoleon at the same time.
Damaging machinery was made a capital offense and they had dozens of executions, hundreds of deportations.
At every stage, the steady progress of civilization is fragile and in danger of being suffocated. Its opponents cloak themselves in moral righteousness, call themselves luddites, the green party, or AI safety rationalists. Its all the same corrosive thing underneath.
This kind of black and white moral thinking is corrosive to one's intelligence. You're allowed to talk about who benefits from massive society change and who suffers. You are allowed to talk about the ways that technology is implemented and how that leads to pros and cons. An attitude of "if we ever stop moving forward and think then the evil bad people win" is deeply anti-intellectual.
We've recreated pre-enlightenment intellectual culture. Authority and logical consistency matter. Reality doesn't.
I read couple of articles in the series and I still couldn't get what was the point author is trying to make. Reads like, "let me give you 100 arguments why I think this is bad".
Do LLMs lie? Of course not, they are just programs. Do the make mistakes or get the facts wrong? Of course they do, not more often then a human does. So what is the point of that article? Why my future is particularly bad now because of LLMs?
The argument isn't that LLMs are bad because they can hallucinate. Author (clearly) argues that LLM use has negative cognitive effects on their users and on society as a whole. Plus, the technology would wipe out a large, large number of jobs.
How can you argue they don't lie, as if they have any idea of correct vs wrong? There is no brain there. When statistics overwhelmingly say "yes" is the correct answer to something, it will say "yes" -- completely independent of whether that's the correct answer.
Complaining about AI slop is starting to become its own kind of slop. There isn't anything novel in this little essay. It might as well have been written by AI because I've seen this type of dude complain about this exact type of thing countless times at this point, and none of them have a solution other than empty moralizing or call your representative or whatever. None of that’s going to work. Fortune, Gizmodo, The Verge,Ars Technica, etc. all circulate the same negative headlines and none of them have a solution, and their writers are probably going to be totally replaced by AI so what difference does it make? They're just capitalizing on the negative sentiment and they have no intention to come up with a solution. At that point it's just complaining and I'm sick of it.
If you’re not an AI yourself it’s weird how you’re so offended by this stuff.
Spotting a problem is relatively easy. Coming up with a solution, not so much. But it is still worth pointing out that there is a problem.
I mean, it has been exhaustively discussed at this stage. Everyone who cares knows all of this stuff already.
The solution is obviously some form of socialism but a lot of tech people are blinkered libertarians who refuse to put two and two together.
Agreed, and I think if you asked most people in the developed world, they'd say the invention of automobiles has been a net positive (to say the least) despite all the very real negatives. Stopped reading the article after that. It seems like the people expressing these sentiments are a loud minority, and I know from having spent way too much time online that if LLMs didn't exist in their current form, they'd be angry about something else. Then again, Maybe I'm just out of touch. It's a distinct possibility.
The conclusion was the takeaway. Everyone is getting bumped up a skill notch, not just bozo liars.
Frankly I think it’s kind of childish to just put up a massive Uk wide block on your website. “Call your representatives”, ok dude, can I give you a list of things I want to change about your country’s policies?
I don't think you can. The comments section of the page is also behind the block for you, no?
This article is a good example of how ideology can can lead people down irrational paths.
A statement that can be reversed onto the speaker without effort is meaningless. It has no content. It just means, "I am rational and you are not." Ok, then.