Last year during the summer I had a bunch of things I wanted to do to improve on the trading system I was working on. A bunch of things that were mostly fiddly, which I had estimated would take me 6 months.
Someone told me to try out Claude Code during my holiday.
I learned the backstory of Sauron with a friend while getting the entire backlog cleaned up. Things worked, better and faster and more maintainable.
Did I run out of things to do? Well, the firm I was with didn't know what they wanted built, so it was a lot of not getting anything done while waiting for them. But the things they had previously wanted, yes, they were done.
I feel like you could only ask this question if you don't really understand software development. If you're building features, you're building bugs. Nobody ever said AI produced bug free code. But it does make it possible to evaluate code extensively to find bugs before you ship it. I believe the code I'm shipping now has far fewer bugs than ever before, but bug free? I don't even know if that makes sense. it all depends on what ends up counting as a bug, in the end.
For my team projects, I still need my teammates to review code, so that bottlenecks how quickly we can burn down the backlog. If I didn't need code review I would have the backlog burned down in weeks, but I'd create a bunch of outages in the process (we do catch things in code review!)
Also, some of my solo projects are gamedev-related, and gamedev is also bottlenecked on actually playing your game to see if its fun :) but it is great to get out of a 2 hour playtest session with notes and have a new build in like 15 minutes with the feedback incorporated.
I am building Flightpad (https://flightpad.net) and absolutely wouldn't without AI tools.
I have several validation loops that run full test harnesses over live (and ever-changing) data. AI brings some subjective evaluation, since data output is all deterministic there are bugs that pop up.
My bug list is nearly empty always. It takes some focus to solve a lot of the data bugs, because I need to understand the root causes (or I feel I need to).
AI has been an incredibly useful addition to my solo work.
It pretty much is? Every easily-described bug has been fixed. The only things that remain are "this isn't wrong per se, it just violates customer expectations" and someone needs to figure out what the product should do in this case.
We burnt through our backlog of "obviously we should do this when we have time" stuff in the spring. It was pretty fun to kill pretty much all of my multi-year refactor and cleanup list.
Now we are pretty much bottlenecked on the business side.
I don't know if I'm doing something wrong but bug fixing with AI has always been a crapshoot. You can absolutely tell an agent to fix an issue, and it will fix that issue, but the vast majority of the time that solution will be by adding code to cover the specific symptom in the bug report. It rarely considers or recommends solutions that have tradeoffs that may change behavior but will eliminate an entire class of similar bugs.
This is mostly a problem of agents not having the autonomy of managing an entire project and having the ability to make tradeoffs to increase simplicity or maintainability, but when given that autonomy no AI model I've used uses that privilege to write simpler, more maintainable code.
I think this gets to something quite fundamental which is in the information loss when delegating a task in plain English.
It's very hard and actually a lot of work to draw an appropriate buffer around a problem in terms of how much agency you want to be exercised in considering other concerns when fixing it.
It takes a lot of skill on both sides to negotiate this even in an ongoing conversational loop.
There comes a point of course where essentially doing this communication and negotiation is equivalent to the work. At that point, you really aren't saving all that much time by not writing the code yourself in a lot of cases.
Maybe greenfield type work is the exception, where generating the code can actually save hours of typing, but certainly for bugfixes on mature systems where the right fix with all factors and competing priorities considered and negotiated only results in say 10 lines of new code, you start to wonder what actually is the benefit of delegation and automation here.
* It rarely considers or recommends solutions that have tradeoffs that may change behavior but will eliminate an entire class of similar bugs.*
Are you specifically asking it to do that? You can just point AI at a bug and say "fix" or you can ask it to review, plan, offer alternatives. You can get a lot more out of it if you actively engage.
Whenever I bug fix, I prompt it specifically to solve the problem not fix the symptom of the problem, and do so with the least amount of code changes possible to fix the problem. And I also tell it that if there's a more maintainable way to solve the problem(like prevent similar issues in other parts of the project, ie problems with a particular dependency or library being used) to suggest alternatives.
I think it's also important that the AI either has context to your entire project(like ability to read the source), or you put the context of what it's solving in the prompt you give it specifically.
It makes me wonder if it could be solved with better context management
A real developer builds up global context of the whole project over time, you always hold the overall architecture in your head
Whereas with agents there is a still a lot lost between sessions
Its similar in a sense that a new developer to a project does more hacky local solutions whereas the senior on the team knows when the architecture needs to change as part of a fix
I think it's rather a problem with the training pipeline incentivizing correctness and endlessly defensive code over simplicity and maintainability of the implementation.
The problem with context I see primarily in mixing up user intent and hard requirements with random half-correct notes some other agent left in the comments.
> It makes me wonder if it could be solved with better context management
There's a reason that repos are now bloated with AGENTS.md files that try to cram context in as pre-text. And that there's so many vibe-coded context/memory aggregation frameworks.
Performance, Defect rate, feature velocity (5x), cycle time on defects to prod are all at the best they've been on our 7 year LOB 1.5 million loc system. 3 devs/2 qa, 2 sme's. 200 paying customers, moves money so it has to work.
First time in 3 decades I don't have any tech debt that I NEED to tackle. Customers can't keep up with us. We are considering new products/adjacent markets. Strangely, our competitors (about 5 serious ones) don't seem to be matching us.
A hidden assumption in your question is that the ai tools themselves don't create bugs or new backlog items. This is false. Even with perfect AI tools, bugs would slip in because of imprecise specifications.
There are many universes where the number of bugs grows faster with better agents.
Maybe it's just me, but I've never worked at a company and on any substantial product that had an empty bug backlog. If a bug queue is empty, the product is either not used much (or not a priority), not tested well enough, or not monitored enough.
I don't know that a world where AI has mostly replaced programmers is a world with an empty bug queue.
While AI is reducing the _quantity_ of programmers needed in many cases, it is also an output multiplier for the remaining programmers, while often raising the bug-to-code ratio.
AI has led to more output, more projects, and more bugs.
Well, I think you just unintentionally made the same point the poster is making. If it's an output multiplier then it should also multiply the ability to fix bugs. Bugs should be just getting fixed in my sleep right?
AI only can improve velocity if it mostly works unsupervised. But most people don't trust their agents to run alone.
And to build that trust will take some time, people will need to learn how to use AI effectively.
I think the PS3 hardware could be a good analogy. It was really powerful, but nobody really knew how to use it. After a few years people got really good at it . And you can view this evolution by the improvement of graphical fidelity we got over the years.
I'm not sure how you can claim it takes time to build trust with AI as if AI isn't actively demonstrating every day why it can't be trusted. Just this morning I had it review clinical notes from a procedure I had done and it mathed things wrong (declaring an injection was 2.5ml when it was 2ml of one thing plus 1ml of another).
I ask it to fix one bug and it introduces three more. I ask it to add a feature and it adds four features.
It has very little common sense. I asked it to build a feature that required api calls to openai, and it decided the best approach was to hand roll the interface code with some bonkers defaults (explicitly zero retries, explicit 1 second timeouts to non steaming api) and wrote the most fragile code that didn't generally work. Even when directed to use the published library by openai it still tried to apply its insane invited requirements to make even that fragile.
I don't think it's nobody knows how to use it, its just not that reliable.
What model are you using/complaining about? Just seems so very removed from my work day where I am more usually put back by the amount of common sense displayed by the AI..
> can improve velocity if it mostly works unsupervised.
I would refute that a bit. I've found quite a nice workflow using a small dumb model (haiku mostly), and basically use it as editor tooling. Change function signature here, split out section into a function and update all similar usage locations there, add a new file and update the make system over there. Trivial tasks that just takes longer to write myself; things i would trust are mostly correct because it still compiles.
It's not a 10x workflow, but keeps me enjoying the coding without loosing track or stagnating my own learning. "commit this" is probably my most used prompt at this point. (with a few strict commit hooks to make it behave and keep brevity).
Mabie a 50% velocity bump to the coding itself on a good day.
Mine is. I've been telling my senior managements that it's cheaper to rewrite stuff from scratch with AI then fix OLD products. They ate that up and everything we have now never reach v1.0 status.
It is empty, for me and my team (I'm an engineering manager). We still have a support rotation but it's just so someone is aware they're supposed to respond to any alerts that come up.
Edit: product doesn't have a deep backlog either. Usually we can only plan somewhere between a couple days and a month in advance. Major recent product launch so we are longest-horizon we have been for a while.
I can confirm the same, backlog is sort of empty unless we try to attack a new domain, and that's usually a 2-3 month party, after that you need to almost imagine tasks. this really smells like end of backlog situation to me.
You can calm yourself and say we can attack and move upstream, e.g. instead of using crm lets write our own crm, etc. But if everyone else is doing so, what is the end game lol
Genuine question... How secure do you feel your job is, in that case? There's only so far that road goes, I imagine? I doubt your management are gonna buy you building a custom OS to run your custom CRM on...?
Standards are super low, so slop gets approved and no one is checking. We get graded on how many Jiras we close. A bug gets a Jira. So we are doing great!
Last year during the summer I had a bunch of things I wanted to do to improve on the trading system I was working on. A bunch of things that were mostly fiddly, which I had estimated would take me 6 months.
Someone told me to try out Claude Code during my holiday.
I learned the backstory of Sauron with a friend while getting the entire backlog cleaned up. Things worked, better and faster and more maintainable.
Did I run out of things to do? Well, the firm I was with didn't know what they wanted built, so it was a lot of not getting anything done while waiting for them. But the things they had previously wanted, yes, they were done.
I feel like you could only ask this question if you don't really understand software development. If you're building features, you're building bugs. Nobody ever said AI produced bug free code. But it does make it possible to evaluate code extensively to find bugs before you ship it. I believe the code I'm shipping now has far fewer bugs than ever before, but bug free? I don't even know if that makes sense. it all depends on what ends up counting as a bug, in the end.
All my solo project backlogs _are_ empty.
For my team projects, I still need my teammates to review code, so that bottlenecks how quickly we can burn down the backlog. If I didn't need code review I would have the backlog burned down in weeks, but I'd create a bunch of outages in the process (we do catch things in code review!)
Also, some of my solo projects are gamedev-related, and gamedev is also bottlenecked on actually playing your game to see if its fun :) but it is great to get out of a 2 hour playtest session with notes and have a new build in like 15 minutes with the feedback incorporated.
I was going to say that. Solo projects advance fast now, and bugs are very few.
Bugs on your solo project are not few. They are merely undiscovered because the bug detection is bottlenecked on you.
Feel free to share the product if there are really so few bugs
I am building Flightpad (https://flightpad.net) and absolutely wouldn't without AI tools.
I have several validation loops that run full test harnesses over live (and ever-changing) data. AI brings some subjective evaluation, since data output is all deterministic there are bugs that pop up.
My bug list is nearly empty always. It takes some focus to solve a lot of the data bugs, because I need to understand the root causes (or I feel I need to).
AI has been an incredibly useful addition to my solo work.
Here you go bro,
https://github.com/pullboard-dev/pullboard
green dot "live" what mean?
It pretty much is? Every easily-described bug has been fixed. The only things that remain are "this isn't wrong per se, it just violates customer expectations" and someone needs to figure out what the product should do in this case.
We burnt through our backlog of "obviously we should do this when we have time" stuff in the spring. It was pretty fun to kill pretty much all of my multi-year refactor and cleanup list.
Now we are pretty much bottlenecked on the business side.
I don't know if I'm doing something wrong but bug fixing with AI has always been a crapshoot. You can absolutely tell an agent to fix an issue, and it will fix that issue, but the vast majority of the time that solution will be by adding code to cover the specific symptom in the bug report. It rarely considers or recommends solutions that have tradeoffs that may change behavior but will eliminate an entire class of similar bugs.
This is mostly a problem of agents not having the autonomy of managing an entire project and having the ability to make tradeoffs to increase simplicity or maintainability, but when given that autonomy no AI model I've used uses that privilege to write simpler, more maintainable code.
I think this gets to something quite fundamental which is in the information loss when delegating a task in plain English.
It's very hard and actually a lot of work to draw an appropriate buffer around a problem in terms of how much agency you want to be exercised in considering other concerns when fixing it.
It takes a lot of skill on both sides to negotiate this even in an ongoing conversational loop.
There comes a point of course where essentially doing this communication and negotiation is equivalent to the work. At that point, you really aren't saving all that much time by not writing the code yourself in a lot of cases.
Maybe greenfield type work is the exception, where generating the code can actually save hours of typing, but certainly for bugfixes on mature systems where the right fix with all factors and competing priorities considered and negotiated only results in say 10 lines of new code, you start to wonder what actually is the benefit of delegation and automation here.
* It rarely considers or recommends solutions that have tradeoffs that may change behavior but will eliminate an entire class of similar bugs.*
Are you specifically asking it to do that? You can just point AI at a bug and say "fix" or you can ask it to review, plan, offer alternatives. You can get a lot more out of it if you actively engage.
I mean I also think it's the way you prompt it.
Whenever I bug fix, I prompt it specifically to solve the problem not fix the symptom of the problem, and do so with the least amount of code changes possible to fix the problem. And I also tell it that if there's a more maintainable way to solve the problem(like prevent similar issues in other parts of the project, ie problems with a particular dependency or library being used) to suggest alternatives.
I think it's also important that the AI either has context to your entire project(like ability to read the source), or you put the context of what it's solving in the prompt you give it specifically.
It makes me wonder if it could be solved with better context management
A real developer builds up global context of the whole project over time, you always hold the overall architecture in your head
Whereas with agents there is a still a lot lost between sessions
Its similar in a sense that a new developer to a project does more hacky local solutions whereas the senior on the team knows when the architecture needs to change as part of a fix
I think it's rather a problem with the training pipeline incentivizing correctness and endlessly defensive code over simplicity and maintainability of the implementation.
The problem with context I see primarily in mixing up user intent and hard requirements with random half-correct notes some other agent left in the comments.
> It makes me wonder if it could be solved with better context management
There's a reason that repos are now bloated with AGENTS.md files that try to cram context in as pre-text. And that there's so many vibe-coded context/memory aggregation frameworks.
It's an AI-hard problem.
[delayed]
Performance, Defect rate, feature velocity (5x), cycle time on defects to prod are all at the best they've been on our 7 year LOB 1.5 million loc system. 3 devs/2 qa, 2 sme's. 200 paying customers, moves money so it has to work.
First time in 3 decades I don't have any tech debt that I NEED to tackle. Customers can't keep up with us. We are considering new products/adjacent markets. Strangely, our competitors (about 5 serious ones) don't seem to be matching us.
A hidden assumption in your question is that the ai tools themselves don't create bugs or new backlog items. This is false. Even with perfect AI tools, bugs would slip in because of imprecise specifications.
There are many universes where the number of bugs grows faster with better agents.
The VisiData backlog is basically empty: https://github.com/saulpw/visidata/issues
Parkinson's Law:
"Work expands so as to fill the time available for its completion"
https://en.wikipedia.org/wiki/Parkinson%27s_Law
Bug tracker is empty but todo list is full with things to work on and new features to add
I haven't seen AI make a single mistake yet while coding my apps
> The bug backlog should be empty
I don't follow.
Maybe it's just me, but I've never worked at a company and on any substantial product that had an empty bug backlog. If a bug queue is empty, the product is either not used much (or not a priority), not tested well enough, or not monitored enough.
I don't know that a world where AI has mostly replaced programmers is a world with an empty bug queue.
While AI is reducing the _quantity_ of programmers needed in many cases, it is also an output multiplier for the remaining programmers, while often raising the bug-to-code ratio.
AI has led to more output, more projects, and more bugs.
Well, I think you just unintentionally made the same point the poster is making. If it's an output multiplier then it should also multiply the ability to fix bugs. Bugs should be just getting fixed in my sleep right?
My bug backlog has been empty [1] at each release for 5 years in a row. We did not need AI to get there.
[1] https://github.com/scala-js/scala-js/issues?q=is%3Aissue+sta...
AI only can improve velocity if it mostly works unsupervised. But most people don't trust their agents to run alone.
And to build that trust will take some time, people will need to learn how to use AI effectively.
I think the PS3 hardware could be a good analogy. It was really powerful, but nobody really knew how to use it. After a few years people got really good at it . And you can view this evolution by the improvement of graphical fidelity we got over the years.
I'm not sure how you can claim it takes time to build trust with AI as if AI isn't actively demonstrating every day why it can't be trusted. Just this morning I had it review clinical notes from a procedure I had done and it mathed things wrong (declaring an injection was 2.5ml when it was 2ml of one thing plus 1ml of another).
I ask it to fix one bug and it introduces three more. I ask it to add a feature and it adds four features.
It has very little common sense. I asked it to build a feature that required api calls to openai, and it decided the best approach was to hand roll the interface code with some bonkers defaults (explicitly zero retries, explicit 1 second timeouts to non steaming api) and wrote the most fragile code that didn't generally work. Even when directed to use the published library by openai it still tried to apply its insane invited requirements to make even that fragile.
I don't think it's nobody knows how to use it, its just not that reliable.
What model are you using/complaining about? Just seems so very removed from my work day where I am more usually put back by the amount of common sense displayed by the AI..
> can improve velocity if it mostly works unsupervised.
I would refute that a bit. I've found quite a nice workflow using a small dumb model (haiku mostly), and basically use it as editor tooling. Change function signature here, split out section into a function and update all similar usage locations there, add a new file and update the make system over there. Trivial tasks that just takes longer to write myself; things i would trust are mostly correct because it still compiles.
It's not a 10x workflow, but keeps me enjoying the coding without loosing track or stagnating my own learning. "commit this" is probably my most used prompt at this point. (with a few strict commit hooks to make it behave and keep brevity).
Mabie a 50% velocity bump to the coding itself on a good day.
Because just like without LLMs, focus is on delivering new products and features, not on paying down technical debt.
If you or your company has nothing to build it means they're out of ideas. And if they're out of ideas, they have too many people.
That's not a place you want to get to.
There's always post-mature optimization to work on.
Mine is. I've been telling my senior managements that it's cheaper to rewrite stuff from scratch with AI then fix OLD products. They ate that up and everything we have now never reach v1.0 status.
My bug backlog is empty. My task backlog is not.
It's empty for all the projects I'm working on. Is your team just not putting in the time to fix them?
It is empty, for me and my team (I'm an engineering manager). We still have a support rotation but it's just so someone is aware they're supposed to respond to any alerts that come up.
Edit: product doesn't have a deep backlog either. Usually we can only plan somewhere between a couple days and a month in advance. Major recent product launch so we are longest-horizon we have been for a while.
I can confirm the same, backlog is sort of empty unless we try to attack a new domain, and that's usually a 2-3 month party, after that you need to almost imagine tasks. this really smells like end of backlog situation to me.
You can calm yourself and say we can attack and move upstream, e.g. instead of using crm lets write our own crm, etc. But if everyone else is doing so, what is the end game lol
Genuine question... How secure do you feel your job is, in that case? There's only so far that road goes, I imagine? I doubt your management are gonna buy you building a custom OS to run your custom CRM on...?
"Competitor X is working on Y, we need to beat them to it!"
Thus Y is prioritized over bugs.
Even AI breaks things when trying to fix bugs.
Standards are super low, so slop gets approved and no one is checking. We get graded on how many Jiras we close. A bug gets a Jira. So we are doing great!
TIL Jira is fractal
Scope creep scales, that’s why.