I liked the commentary and the metaphor with the Nyquist theorem. Mathematically speaking it's invalid but the problem being discussed is real - when the velocity of code generation (accelerated by LLMs) outpaces human cognitive capacity to review, test, and mentally model the system, structural coherence collapses.
And we're already there, it's beyond our cognitive capacity to fully understand the (AI) produced volume of work for given unit of time.
In order to really mitigate the human bottleneck, we will either have to reach the AGI, which might not come in the next 10 years or so, or change the form of how we're doing work, review included.
Funnily enough, there are techniques that allows to go under the Nyquist limit, eg compressive sensing.
I don't like the analogy, it supposed that AI generated code will always be badly organized underneath if it is automatic, however, as time goes, I find the code better and better..
Agreed that it is an analogy, not necessarily a law of nature. However, my takeaway had nothing to do with the quality or organization of the generated code. The "signal" fidelity, so to speak, is the human understanding of the system, which I presume we still value since we would like human evaluation and decision making to drive the direction of change. If we're not making decisions from a foundation of understanding, we may as well include some dice rolls and ouija board advice.
The author assumes that human understanding and modeling is essential. However, this no longer appears to be the expressed preference of the industry.
AI advocates nowadays are calling for abolishing human code review. So code moves from read-often-write-rarely to read-once-write-once to read-never-write-often. The assumption is that we can rely on the agents to decipher the code if needed.
My feeling is that software "engineering" in the future will be basically a cargo cult or the Adeptus Mechanicus from Warhammer 40k: all detailed technical knowledge comes from the gods, humans only manage the rituals around technology.
I have to say I mainly agree in a way I wouldn’t have 1 month ago. We are now starting to see bugs creep above our baseline. This is only what we are detecting. At least in certain critical areas we are not removing strict human review.
One theory is our codebase (which is large and mature) was very coherent. As we moved to more autonomy and needed generated code in the base has grown and is less mature now and likely not as coherent. The LLM has to cope with this.
Seems like a forced analogy. Is the signal a periodic and regular frequency?
If its a straight line you need 2 samples.
If its not you probably need more.
Id argue codebase understanding is neither of these simplifications
And the sampling theorem only applies when you only know that the signal is bandlimited. The more you know can assume, the fewer samples you need.
I liked the commentary and the metaphor with the Nyquist theorem. Mathematically speaking it's invalid but the problem being discussed is real - when the velocity of code generation (accelerated by LLMs) outpaces human cognitive capacity to review, test, and mentally model the system, structural coherence collapses.
And we're already there, it's beyond our cognitive capacity to fully understand the (AI) produced volume of work for given unit of time.
In order to really mitigate the human bottleneck, we will either have to reach the AGI, which might not come in the next 10 years or so, or change the form of how we're doing work, review included.
[delayed]
Funnily enough, there are techniques that allows to go under the Nyquist limit, eg compressive sensing.
I don't like the analogy, it supposed that AI generated code will always be badly organized underneath if it is automatic, however, as time goes, I find the code better and better..
Agreed that it is an analogy, not necessarily a law of nature. However, my takeaway had nothing to do with the quality or organization of the generated code. The "signal" fidelity, so to speak, is the human understanding of the system, which I presume we still value since we would like human evaluation and decision making to drive the direction of change. If we're not making decisions from a foundation of understanding, we may as well include some dice rolls and ouija board advice.
You change what the signal is. Simple as.
The amount of app updates - I don't have auto update turned on - is now completely ridiculous.
LLMs plus continuous delivery - what a gruesome twosome.
It's like you've ended up inside your own OODA loop.
The author assumes that human understanding and modeling is essential. However, this no longer appears to be the expressed preference of the industry.
AI advocates nowadays are calling for abolishing human code review. So code moves from read-often-write-rarely to read-once-write-once to read-never-write-often. The assumption is that we can rely on the agents to decipher the code if needed.
My feeling is that software "engineering" in the future will be basically a cargo cult or the Adeptus Mechanicus from Warhammer 40k: all detailed technical knowledge comes from the gods, humans only manage the rituals around technology.
He talks about the problem with this empirically.
I have to say I mainly agree in a way I wouldn’t have 1 month ago. We are now starting to see bugs creep above our baseline. This is only what we are detecting. At least in certain critical areas we are not removing strict human review.
One theory is our codebase (which is large and mature) was very coherent. As we moved to more autonomy and needed generated code in the base has grown and is less mature now and likely not as coherent. The LLM has to cope with this.