I’m considering a set of skills that I can apply common lenses to PRs (metrics, error handling, testability + tests, architecture, security), and fetch the code and pr details via the GitHub CLI or an MCP. This seems like it might get absurdly expensive, but maybe w the right tuning it’ll be possible to run most of them on smaller models. Each would let me zoom in on its area and help me through the process of reviewing.
The problem is with full on misses that I’d be able to catch by going line by line. The more static analysis and style enforcement the better - I’m still not sure whether it will come out as a benefit, esp when you consider cost of tokens. AI often creates its own extra work alongside the benefits, and it’s a bit like nicotine in that it works for a minute but then you’ll need to keep applying it to even return to baseline.
I’m considering a set of skills that I can apply common lenses to PRs (metrics, error handling, testability + tests, architecture, security), and fetch the code and pr details via the GitHub CLI or an MCP. This seems like it might get absurdly expensive, but maybe w the right tuning it’ll be possible to run most of them on smaller models. Each would let me zoom in on its area and help me through the process of reviewing.
The problem is with full on misses that I’d be able to catch by going line by line. The more static analysis and style enforcement the better - I’m still not sure whether it will come out as a benefit, esp when you consider cost of tokens. AI often creates its own extra work alongside the benefits, and it’s a bit like nicotine in that it works for a minute but then you’ll need to keep applying it to even return to baseline.
That's what I do[0]. I run a gauntlet of judges against the PR, each with a strict lane to check.
0: https://github.com/jacquardlabs/gauntlet
Most of the time I just give Claude Code the PR link and then do the human side of the review based on the feedback it gives me.
Sometimes I'll prompt again to scrutinize further.
I use Gemini most of the time when using different AI tools because it's free.I occasionally use grok build.And I rarely do code reviews anymore.