I'm building solo and am extensionally using Claude Code for development and testing. The following scheme finally works great for me.
First of all, the agent writes code. After that, it creates tests and verifies that all of them work correctly by cracking the checks and rerunning the test suite.
Secondly, it gives me the ready to test code + set up environment (stage). I'm checking that the code actually works, and if not, we are making some fixes until I'm fully satisfied with the result.
Thirdly, the agent starts an external review using skill for code review and usually makes some additional fixes to codebase and corrects tests again.
After all, we are ready to commit and push the feature to the main branch. Before merging, we start CI/CD - which is important because it does a type check - and wait until the run finishes successfully.
That's how it happens in my case.
Quick note: this plan was built after many iterations of coding and testing, and it has finally proved that it works.
We still create a ton of end-to-end tests and run them a lot. What changed for us is the review process. While our whole team uses agents to do reviews, we also ask other team members to review the coding session itself rather than the diff. That team member can ask the coding session questions about the architecture decisions, scalability, edge cases, ..etc.
Disclosure: I'm building AQ (aq.dev), which is partly why I'm deep in this. We wrote up the session-review practice here: https://aq.dev/guides/how-to-review-an-ai-coding-session/. The practice works with any agents too, nothing tool-specific about it.
I have Claude implementing features from a spec, running tests, creating a build, and then publishing to TestFlight. All I have to do is verify that the build works once I get the TestFlight notification.
I have found a key is to use end-to-end tests and not unit tests.
This has been working well so far after around 100 pull requests for an app I recently decided to make. So far it’s been good.
More testing is better, so far! I'm using the same red-green-refactor TDD idea, and having Claude use Superpowers code review and running Coderabbit CLI on it all before pushing to CI/CD, which also runs a series of tests (mostly security ones).
Someone else posted that whenever something broke they would ask 'Why was it built that way?' to try to have the bot fix it's own coding patterns.. as well as adding a regression test to ensure it did not happen again.
I also have it do dry-runs and then do live on one item before batching. Catches a lot of things!
Oh, and insist that it RTFM's the API if you're using one.
Testing is not a formality. In fact, it's better and easier than ever. Tell your agents to develop your code-base with Red-Green-Refactor Test-driven development. Write failing tests first and run them, then write the code to make the tests pass, then refactor the code as needed. I used to hate writing tests before, but now that I can offload that (and the development, honestly) to AI, it's nothing to keep up with it. Your codebase quality will increase with TDD, because it will reduce the number of things that inadvertently break during updates. Things will break loudly and can be easily fixed (again, by AI) before shipping. If something does manage to get through, add more tests to cover it for next time.
This actually makes the LLMs better at coding as well. They can verify their own results and iterate with far less manual validation.
Yeah, that only works if they follow the plan (they don't), actually write the tests first (they don't) and don't silently defer anything which doesn't have a test written as "speculative without a use case".
Or just write tests to match the buggy code after you call them out for not writing tests.
I mean, the struggle is real...
No matter how weasel-proof you make the plans they are much better weasels and just do as little as possible and "the test not written is the test which never fails." There's a certain amount of zen to them.
I'm building solo and am extensionally using Claude Code for development and testing. The following scheme finally works great for me.
First of all, the agent writes code. After that, it creates tests and verifies that all of them work correctly by cracking the checks and rerunning the test suite.
Secondly, it gives me the ready to test code + set up environment (stage). I'm checking that the code actually works, and if not, we are making some fixes until I'm fully satisfied with the result.
Thirdly, the agent starts an external review using skill for code review and usually makes some additional fixes to codebase and corrects tests again.
After all, we are ready to commit and push the feature to the main branch. Before merging, we start CI/CD - which is important because it does a type check - and wait until the run finishes successfully.
That's how it happens in my case.
Quick note: this plan was built after many iterations of coding and testing, and it has finally proved that it works.
We still create a ton of end-to-end tests and run them a lot. What changed for us is the review process. While our whole team uses agents to do reviews, we also ask other team members to review the coding session itself rather than the diff. That team member can ask the coding session questions about the architecture decisions, scalability, edge cases, ..etc.
Disclosure: I'm building AQ (aq.dev), which is partly why I'm deep in this. We wrote up the session-review practice here: https://aq.dev/guides/how-to-review-an-ai-coding-session/. The practice works with any agents too, nothing tool-specific about it.
I have Claude implementing features from a spec, running tests, creating a build, and then publishing to TestFlight. All I have to do is verify that the build works once I get the TestFlight notification.
I have found a key is to use end-to-end tests and not unit tests.
This has been working well so far after around 100 pull requests for an app I recently decided to make. So far it’s been good.
It’s free, no subscription.
[1] https://trysojourn.app
I’ve done similar with backend projects utilizing GitHub actions to run tests and publish to staging for verification.
More testing is better, so far! I'm using the same red-green-refactor TDD idea, and having Claude use Superpowers code review and running Coderabbit CLI on it all before pushing to CI/CD, which also runs a series of tests (mostly security ones). Someone else posted that whenever something broke they would ask 'Why was it built that way?' to try to have the bot fix it's own coding patterns.. as well as adding a regression test to ensure it did not happen again. I also have it do dry-runs and then do live on one item before batching. Catches a lot of things! Oh, and insist that it RTFM's the API if you're using one.
Testing is not a formality. In fact, it's better and easier than ever. Tell your agents to develop your code-base with Red-Green-Refactor Test-driven development. Write failing tests first and run them, then write the code to make the tests pass, then refactor the code as needed. I used to hate writing tests before, but now that I can offload that (and the development, honestly) to AI, it's nothing to keep up with it. Your codebase quality will increase with TDD, because it will reduce the number of things that inadvertently break during updates. Things will break loudly and can be easily fixed (again, by AI) before shipping. If something does manage to get through, add more tests to cover it for next time.
This actually makes the LLMs better at coding as well. They can verify their own results and iterate with far less manual validation.
Yeah, that only works if they follow the plan (they don't), actually write the tests first (they don't) and don't silently defer anything which doesn't have a test written as "speculative without a use case".
Or just write tests to match the buggy code after you call them out for not writing tests.
I mean, the struggle is real...
No matter how weasel-proof you make the plans they are much better weasels and just do as little as possible and "the test not written is the test which never fails." There's a certain amount of zen to them.
super-dev.sh > super-test.sh > super-build.sh > super-e2e.sh > super-deploy.sh ? super-e2e2e.sh
Yes