LLMs can't be strategic because they do not understand the big picture -- that the real work of good software is balancing a hundred different constraints in a way that produces the optimal result for the humans who use it.
It's not all that different from the state of big corp software today! Large organizations with layers of management tend to lose all abiliy to keep a consistent strategy. They tend to go all in on a single dimension such as ROI for the next quarter, but it misses the bigger picture. Good software is about creating longer term value and takes consistent skill & vision to execute.
Those software engineers who focus on this big picture thinking are going to be more valuable than ever.
>Good software is about creating longer term value and takes consistent skill & vision to execute.
>Those software engineers who focus on this big picture thinking are going to be more valuable than ever.
Not to rain on our hopes, but AI can give us some options and we can pick the best. I think this eliminates all middle level positions. Newbies are low cost and make decisions that are low stakes. The most senior or seniors can make 30 major decisions per day when AI lays them out.
I own a software shop and my hires have been: Interns and people with the specific skill of my industry(Mechanical engineers).
2 years ago, I hired experienced programmers. Now I turn my mechanical engineers into programmers.
So what you are a saying is that you removed the people who can make the decisions that keep your software maintainable and kept the people who will slowly over time cause your software to become less maintainable? I'm not sure that tradeoff is a a good one.
"Im not even sure we will need maintain software" (sic) - I'm not sure what your specific background is, but with a statement like that you lose all legitimacy to me.
And the human downstream of this random reorganization of things at will, how do they manage it?
If its AI agents all the way down its commoditization all the way down, if humans have to deal with it there's some sort of cost for change even if its 0 for code.
Not to rain on our hopes, but AI can give us some options and we can pick the best.
a.k.a. greedy algorithms, a subject those of us on HN should be well-acquainted with. You can watch the horizon effect frequently play out in corporate decisionmaking.
> Not to rain on our hopes, but AI can give us some options and we can pick the best.
But that's kind of my point. A bunch of decisions like that tend to end up with a "random walk" effect. It's a bunch of tactical choices which don't add up to something strategic. It could be, but it takes the human in the loop to hold onto that overall strategy.
AI often simply does not offer the best options, does not think strategically, and if you constrained to its choices you will often make silly mistakes.
This is why all the arguments about context windows and RAG exist, because at the end of the day even if you asked the question of a human with all the context there's such a thing as opinions, stated vs unstated goals, requirements vs non functional requirements, etc which will give you wildly different answers.
Most of the time people don't even know the questions they want to ask.
Why can’t LLMs understand the big picture? I mean, a lot of companies have most of their information available in a digital form at this point, so it could be consumed by the LLM.
I think if anything, we have a better chance in the little picture: you can go to lunch with your engineering coworkers or talk to somebody on the factory floor and get insights that will never touch the computers.
Giant systems of constraints, optimizing many-dimensional user metrics: eventually we will hit the wall where it is easier to add RAM to machines than humans.
Most senior could make sense (although I’d like to see a collection of independent guilds coordinated by an LLM “CEO” just to see how it could work—might not be good enough yet, but it’d be an interesting experiment).
Ultimately, I suspect “AI” (although, maybe much more advanced than current LLMs) will be able to do just about any information based task. But in the end only humans can actually be responsible/accountable.
> Because LLMs don't understand things to begin with.
Ok, that’s fair. But I think the comment was making a distinction between the big picture and other types of “understanding.” I agree that it is incorrect to say LLMs understand anything, but I think that was just an informal turn of phrase. I’m saying I don’t think there’s something special about “big picture” information processing tasks, compared to in-detail information processing tasks, that makes them uniquely impossible for LLM.
The other objections seem mostly to be issues with current tooling, or the sort of capacity problems that the LLM developers are constantly overcoming.
I would say that it's very germane to my original statement. Understanding is absolutely fundamental to strategy and it is pretty much why I can say LLMs can't be strategic.
To really strategize you have to have mental model of well, everything and be able to sift through that model to know what elements are critical or not. And it includes absolutely everything -- human psychology to understand how people might feel about certain features or usage models, the future outlook for what popular framework to choose and will it as viable next year as it is today. The geographic and geopolitics of which cloud provider to use. The knowledge of human sentiment around ethical or moral concerns. The financial outlook for VC funding and interest rates. The list goes on and on. The scope of what information may be relevant is unlimited in time and space. It needs creativity, imagination, intuition, inventiveness, discernment.
> I’m saying I don’t think there’s something special about “big picture” information processing tasks, compared to in-detail information processing tasks, that makes them uniquely impossible for LLM.
LLMs can do neither reliably because to do that you need understanding which LLMs don't have. You need to learn from the codebase and the project, which LLMs can't do.
On top of that, to have the big picture LLMs have to be inside your mind. To know and correlate the various Google Docs and Figma files, the Slack discussions, the various notes scattered on your system etc.
They can't do that either because, well, they don't understand or learn (and no, clawdbot will not help you with that).
> The other objections seem mostly to be issues with current tooling, or the sort of capacity problems that the LLM developers are constantly overcoming.
These are not limitations of tooling, and no, LLM developers are not even close to overcoming, especially not "constantly". The only "overcoming" has been the gimmicky "1 million token context" which doesn't really work.
Task Time horizons are improving exponentially with doubling times around 4 months per METR. At what timescale would you accept that they "can be strategic"? Theres little reason to think they wont be at multi week or month time horizons very soon. Do you need to be strategic to complete multi month tasks?
Prompt-injection in all its forms. If the hyper-mad-libs machine doesn't reliably "understand" and model the difference between internal and external words, how can we trust them to model fancier stuff?
> LLMs can't be strategic because they do not understand the big picture -- that the real work of good software is balancing a hundred different constraints in a way that produces the optimal result for the humans who use it.
There’s good reason to think that they could understand the big picture just fine, even today, except that they’re currently severely constrained by what we choose, or have time, to tell them. They can already easily give a much more comprehensive survey of suitable options for solving a given problem than most humans can.
If they had more direct access to the information we have access to, that we currently grudgingly dole out to them in dribs and drabs, they would be much more capable.
I don't feel like the abstraction away from assembly language resulted in fewer software engineering jobs. Nor do I feel like Java's virtual machine resulted in fewer systems engineering jobs. Somehow I don't feel that writing in English rather than pure logic will result in fewer engineering problems either. A lot more actually. But at least we'll get the requirements out of users into something concrete faster.
What is definitely going to be abundantly clear is just how much better machines can get at creating correct code and how bad each of us truly is at this. That's an ego hit.
The loving effort an artisan puts into a perfect pot still has wabi sabi from the human error; whereas a factory produced pot is way more perfect and possesses both a Quality from closeness to Idealism and an eerieness from its unnaturalness.
However, the demand for artisan pottery has niched out compared to Ikea bowls, so that's just how it is.
> I’m starting to think that’s going to be personality and feel and polish, but turned up a notch. That’s what I used to do when I started writing apps, but in some ways I have really toned it down in favor of OS alignment.
That's all that is really required. I mean look at the Microslop fiasco. They ruined a perfectly good editor: Notepad with AI slop. But this is not reflecting in their sales. They still are showing record revenues.
Just because a competing product exists does not mean your product is suddenly obsolete. There will always be people who will want to buy (provided the market is not oversaturated). Because that is how humans do things. AI won't change that behavior overnight [1]. Look around you and you will see every product you hold in your hand has at least 5-10 competitors.
[1] Think about all the things that are still not computerized and which requires you to fill some or the other form of paperwork. We have had computers for over nearly 6 decades now. We STILL have physical forms that we fill from time to time. Computerization was touted to revolutionize this and yet here we are. Still not achieved 100% digitalization. The same will happen with AI as well. There is this initial burst of excitement (which is the phase we are in) until reality sets in and that's when people will learn how to best use the technology. What you are seeing today (vibe coding et all) is NOT IT.
> It was a nice little feature that I knew exactly how to do, but I hadn’t prioritized getting done yet because there were a bunch of other things on my plate. But with a little assist, it was quick to implement.
Exactly how I feel. AI has allowed me to work on projects that I've wanted to work on but didn't have the time/energy for.
This song bangs, looping this all day. What a throwback. Reminds me of ytcracker/digitalgangster days. Also reminds me of Das Racist. Thanks for posting this lol. Love that grime-ish electro beat.
Can you email me? Would love to chat/get more tunes from you.
LLMs can't be strategic because they do not understand the big picture -- that the real work of good software is balancing a hundred different constraints in a way that produces the optimal result for the humans who use it.
It's not all that different from the state of big corp software today! Large organizations with layers of management tend to lose all abiliy to keep a consistent strategy. They tend to go all in on a single dimension such as ROI for the next quarter, but it misses the bigger picture. Good software is about creating longer term value and takes consistent skill & vision to execute.
Those software engineers who focus on this big picture thinking are going to be more valuable than ever.
>Good software is about creating longer term value and takes consistent skill & vision to execute.
>Those software engineers who focus on this big picture thinking are going to be more valuable than ever.
Not to rain on our hopes, but AI can give us some options and we can pick the best. I think this eliminates all middle level positions. Newbies are low cost and make decisions that are low stakes. The most senior or seniors can make 30 major decisions per day when AI lays them out.
I own a software shop and my hires have been: Interns and people with the specific skill of my industry(Mechanical engineers).
2 years ago, I hired experienced programmers. Now I turn my mechanical engineers into programmers.
So what you are a saying is that you removed the people who can make the decisions that keep your software maintainable and kept the people who will slowly over time cause your software to become less maintainable? I'm not sure that tradeoff is a a good one.
This might have been true pre-agent AI programming, but honestly the code seems better than ever. It finds edge cases better than me.
I know... I know buddy. The world changed and I don't know if I'm going to have a job.
Finding edge cases is completely orthogonal to creating maintainable software. Finding edge cases ~= identifying test suites. Making software maintainable ~= minimizing future cost of effective changes.
Ignoring future maintenance cost because test suites are easier to create seems like disjointed logic.
Im not even sure we will need maintain software. I can basically have AI rewrite entire code bases in an hour including testing.
Have you use AI Agents? Specifically with SOTA models like Opus.
I talked like you 3 weeks ago. But the world changed.
"Im not even sure we will need maintain software" (sic) - I'm not sure what your specific background is, but with a statement like that you lose all legitimacy to me.
And the human downstream of this random reorganization of things at will, how do they manage it?
If its AI agents all the way down its commoditization all the way down, if humans have to deal with it there's some sort of cost for change even if its 0 for code.
> Not to rain on our hopes, but AI can give us some options and we can pick the best.
But that's kind of my point. A bunch of decisions like that tend to end up with a "random walk" effect. It's a bunch of tactical choices which don't add up to something strategic. It could be, but it takes the human in the loop to hold onto that overall strategy.
AI often simply does not offer the best options, does not think strategically, and if you constrained to its choices you will often make silly mistakes.
This is why all the arguments about context windows and RAG exist, because at the end of the day even if you asked the question of a human with all the context there's such a thing as opinions, stated vs unstated goals, requirements vs non functional requirements, etc which will give you wildly different answers.
Most of the time people don't even know the questions they want to ask.
Why can’t LLMs understand the big picture? I mean, a lot of companies have most of their information available in a digital form at this point, so it could be consumed by the LLM.
I think if anything, we have a better chance in the little picture: you can go to lunch with your engineering coworkers or talk to somebody on the factory floor and get insights that will never touch the computers.
Giant systems of constraints, optimizing many-dimensional user metrics: eventually we will hit the wall where it is easier to add RAM to machines than humans.
I basically just posted the same response. I generally agree with everything you said.
Only thing to add, maybe we have the most senior of seniors verifying the decisions of AI.
Most senior could make sense (although I’d like to see a collection of independent guilds coordinated by an LLM “CEO” just to see how it could work—might not be good enough yet, but it’d be an interesting experiment).
Ultimately, I suspect “AI” (although, maybe much more advanced than current LLMs) will be able to do just about any information based task. But in the end only humans can actually be responsible/accountable.
> Why can’t LLMs understand the big picture?
Because LLMs don't understand things to begin with.
Because LLMs only have access to aource code and whatever .md files you've given them.
Because they have biases in their training data that overfit them on certain solutions.
Because LLMs have a tiny context window.
Because LLMs largely suck at UI/UX/design especially when they don't have referense images.
Because...
> Because LLMs don't understand things to begin with.
Ok, that’s fair. But I think the comment was making a distinction between the big picture and other types of “understanding.” I agree that it is incorrect to say LLMs understand anything, but I think that was just an informal turn of phrase. I’m saying I don’t think there’s something special about “big picture” information processing tasks, compared to in-detail information processing tasks, that makes them uniquely impossible for LLM.
The other objections seem mostly to be issues with current tooling, or the sort of capacity problems that the LLM developers are constantly overcoming.
I would say that it's very germane to my original statement. Understanding is absolutely fundamental to strategy and it is pretty much why I can say LLMs can't be strategic.
To really strategize you have to have mental model of well, everything and be able to sift through that model to know what elements are critical or not. And it includes absolutely everything -- human psychology to understand how people might feel about certain features or usage models, the future outlook for what popular framework to choose and will it as viable next year as it is today. The geographic and geopolitics of which cloud provider to use. The knowledge of human sentiment around ethical or moral concerns. The financial outlook for VC funding and interest rates. The list goes on and on. The scope of what information may be relevant is unlimited in time and space. It needs creativity, imagination, intuition, inventiveness, discernment.
LLMs are fundamentally incapable of this.
> I’m saying I don’t think there’s something special about “big picture” information processing tasks, compared to in-detail information processing tasks, that makes them uniquely impossible for LLM.
LLMs can do neither reliably because to do that you need understanding which LLMs don't have. You need to learn from the codebase and the project, which LLMs can't do.
On top of that, to have the big picture LLMs have to be inside your mind. To know and correlate the various Google Docs and Figma files, the Slack discussions, the various notes scattered on your system etc.
They can't do that either because, well, they don't understand or learn (and no, clawdbot will not help you with that).
> The other objections seem mostly to be issues with current tooling, or the sort of capacity problems that the LLM developers are constantly overcoming.
These are not limitations of tooling, and no, LLM developers are not even close to overcoming, especially not "constantly". The only "overcoming" has been the gimmicky "1 million token context" which doesn't really work.
Yeah, it's strange to me that the default assumption is that current LLMs are already human-level AGI.
>LLMs can't be strategic because they do not understand the big picture
While I do tend to believe you, what evidence based data do you have to prove this is true?
> While I do tend to believe you, what evidence based data do you have to prove this is true?
IMO the onus is to prove that they can be strategic. Otherwise you're asking me to prove a negative.
Saying the tiger has to prove it can eat you is not a great strategy to survive a tiger attack.
Task Time horizons are improving exponentially with doubling times around 4 months per METR. At what timescale would you accept that they "can be strategic"? Theres little reason to think they wont be at multi week or month time horizons very soon. Do you need to be strategic to complete multi month tasks?
Can an LLM give you an upfront estimate that a task will take multiple months?
Can it decide intelligently what it would have to change if you said "do what you can to have it ready in half the time?"
Prompt-injection in all its forms. If the hyper-mad-libs machine doesn't reliably "understand" and model the difference between internal and external words, how can we trust them to model fancier stuff?
> LLMs can't be strategic because they do not understand the big picture -- that the real work of good software is balancing a hundred different constraints in a way that produces the optimal result for the humans who use it.
There’s good reason to think that they could understand the big picture just fine, even today, except that they’re currently severely constrained by what we choose, or have time, to tell them. They can already easily give a much more comprehensive survey of suitable options for solving a given problem than most humans can.
If they had more direct access to the information we have access to, that we currently grudgingly dole out to them in dribs and drabs, they would be much more capable.
I don't feel like the abstraction away from assembly language resulted in fewer software engineering jobs. Nor do I feel like Java's virtual machine resulted in fewer systems engineering jobs. Somehow I don't feel that writing in English rather than pure logic will result in fewer engineering problems either. A lot more actually. But at least we'll get the requirements out of users into something concrete faster.
What is definitely going to be abundantly clear is just how much better machines can get at creating correct code and how bad each of us truly is at this. That's an ego hit.
The loving effort an artisan puts into a perfect pot still has wabi sabi from the human error; whereas a factory produced pot is way more perfect and possesses both a Quality from closeness to Idealism and an eerieness from its unnaturalness.
However, the demand for artisan pottery has niched out compared to Ikea bowls, so that's just how it is.
> I’m starting to think that’s going to be personality and feel and polish, but turned up a notch. That’s what I used to do when I started writing apps, but in some ways I have really toned it down in favor of OS alignment.
That's all that is really required. I mean look at the Microslop fiasco. They ruined a perfectly good editor: Notepad with AI slop. But this is not reflecting in their sales. They still are showing record revenues.
Just because a competing product exists does not mean your product is suddenly obsolete. There will always be people who will want to buy (provided the market is not oversaturated). Because that is how humans do things. AI won't change that behavior overnight [1]. Look around you and you will see every product you hold in your hand has at least 5-10 competitors.
[1] Think about all the things that are still not computerized and which requires you to fill some or the other form of paperwork. We have had computers for over nearly 6 decades now. We STILL have physical forms that we fill from time to time. Computerization was touted to revolutionize this and yet here we are. Still not achieved 100% digitalization. The same will happen with AI as well. There is this initial burst of excitement (which is the phase we are in) until reality sets in and that's when people will learn how to best use the technology. What you are seeing today (vibe coding et all) is NOT IT.
So now we're supposed to become the idea guy? Goddamnit.
> It was a nice little feature that I knew exactly how to do, but I hadn’t prioritized getting done yet because there were a bunch of other things on my plate. But with a little assist, it was quick to implement.
Exactly how I feel. AI has allowed me to work on projects that I've wanted to work on but didn't have the time/energy for.
> I don't feel like the abstraction away from assembly language resulted in fewer software engineering jobs.
Given the models are unlikely to stop getting better, I think it is fair to say the human contribution is going to keep getting "leaner".
That is going to change the job, but also head count.
But I agree harnessing models opens up opportunities for better product design, ... but only ... everywhere.
The people who design the most usable software have always been in a minority. They will be valuable for some time.
Are you the wallflower from the legendary fugg efnet rap video
https://www.youtube.com/watch?v=evBderwVKKE
This song bangs, looping this all day. What a throwback. Reminds me of ytcracker/digitalgangster days. Also reminds me of Das Racist. Thanks for posting this lol. Love that grime-ish electro beat.
Can you email me? Would love to chat/get more tunes from you.