Love some of the amazing feedbacks. I’d want to keep hearing more, with an intent and effort to build a product that actually serves the community instead of just being an abstract.
Keep pouring! Thanks
Interesting idea. I’ve mostly run into this kind of drift at the LLM output level rather than in the code itself even with similar setups, behavior can change quite a bit between runs.
Curious if you’ve explored that side as well.
That’s an interesting angle. VibeDrift currently focuses on the artifact i.e. the code that actually lands in your repo rather than the LLM output itself. The reasoning is that regardless of why the drift happened (different sessions, different prompts, different models), the codebase is what you ship and maintain. That said, tracking prompt-to-output consistency is a genuinely different problem and not something I’ve explored yet. Would be curious what patterns you’ve seen there. I’m always open to suggestions and feedbacks. There is always room for improvement
I think CodeRabbit/others do this already? They learn from your code base/previous PRs, then use that in the future. I built a mini PoC of this a few weeks ago as well, out of curiosity
CodeRabbit reviews one PR at a time using context from past PRs. VibeDrift scans the entire codebase at once and compares every file against its directory peers. Different question: not “does this PR look good” but “does this file follow the same patterns as the files sitting next to it.” Also runs fully locally, zero data sent. Curious what your PoC does though.
There is one potential issue with scanning the entire codebase IMHO - if the codebase is old enough, there can be old snippets of code that DON'T correspond to the current preferences anymore. So scanning the last 200 PRs and reading comments is a bit safer maybe.
This is what my vibe coded PoC did - it just used GitHub API to go through the last X merged PRs, see the diffs and all discussions and create a document based on that - the developer manifesto kind of, a set of preferences and rules for this dev team. Then, use this manifesto as context whenever reviewing new PRs
Cool project, but not really. From what I can see Allium is preventive, it gives the AI a spec to code against so intent doesn’t get lost. VibeDrift is diagnostic, it analyzes code that already exists and measures where patterns diverged. They’re actually complementary.
Claude Code memory helps the AI remember context within its own sessions. But it’s still one model’s view of what “should” be consistent. VibeDrift doesn’t rely on any AI’s memory or opinion. It looks at the code that actually exists in your repo and measures what the majority of files do vs which files break from that pattern. It’s also deterministic, meaning same codebase, same score every time, which matters if you want to track drift over time or gate PRs in CI.
Qodo reviews individual PRs. VibeDrift compares files against each other across your whole codebase. Qodo won’t tell you that the file you’re adding uses raw SQL while 7 sibling files use a repository pattern, it’s looking at the PR in isolation, not the project. If that makes sense.
Love some of the amazing feedbacks. I’d want to keep hearing more, with an intent and effort to build a product that actually serves the community instead of just being an abstract. Keep pouring! Thanks
Interesting idea. I’ve mostly run into this kind of drift at the LLM output level rather than in the code itself even with similar setups, behavior can change quite a bit between runs. Curious if you’ve explored that side as well.
That’s an interesting angle. VibeDrift currently focuses on the artifact i.e. the code that actually lands in your repo rather than the LLM output itself. The reasoning is that regardless of why the drift happened (different sessions, different prompts, different models), the codebase is what you ship and maintain. That said, tracking prompt-to-output consistency is a genuinely different problem and not something I’ve explored yet. Would be curious what patterns you’ve seen there. I’m always open to suggestions and feedbacks. There is always room for improvement
I think CodeRabbit/others do this already? They learn from your code base/previous PRs, then use that in the future. I built a mini PoC of this a few weeks ago as well, out of curiosity
CodeRabbit reviews one PR at a time using context from past PRs. VibeDrift scans the entire codebase at once and compares every file against its directory peers. Different question: not “does this PR look good” but “does this file follow the same patterns as the files sitting next to it.” Also runs fully locally, zero data sent. Curious what your PoC does though.
There is one potential issue with scanning the entire codebase IMHO - if the codebase is old enough, there can be old snippets of code that DON'T correspond to the current preferences anymore. So scanning the last 200 PRs and reading comments is a bit safer maybe.
This is what my vibe coded PoC did - it just used GitHub API to go through the last X merged PRs, see the diffs and all discussions and create a document based on that - the developer manifesto kind of, a set of preferences and rules for this dev team. Then, use this manifesto as context whenever reviewing new PRs
Never say never, you got it now - https://vibedrift.ai/releases
Similar sort of idea https://github.com/juxt/allium
Cool project, but not really. From what I can see Allium is preventive, it gives the AI a spec to code against so intent doesn’t get lost. VibeDrift is diagnostic, it analyzes code that already exists and measures where patterns diverged. They’re actually complementary.
How it is different then Claude code auto memory? I mean we can also store marker in claude code memory and ask drift anytime.
Claude Code memory helps the AI remember context within its own sessions. But it’s still one model’s view of what “should” be consistent. VibeDrift doesn’t rely on any AI’s memory or opinion. It looks at the code that actually exists in your repo and measures what the majority of files do vs which files break from that pattern. It’s also deterministic, meaning same codebase, same score every time, which matters if you want to track drift over time or gate PRs in CI.
What does this offer over a decent orchestration layer and a… prompt?
This is for multi file codebases written across multiple sessions/prompts.
Prompts help guide generation, but they don’t guarantee consistency over time.
VibeDrift checks the codebase itself and flags where files contradict each other.
Probably overkill for a weekend project, but shows up fast as things grow.
I’d suggest you give it a try and let me know your feedback.
How is this different from qodo?
Qodo reviews individual PRs. VibeDrift compares files against each other across your whole codebase. Qodo won’t tell you that the file you’re adding uses raw SQL while 7 sibling files use a repository pattern, it’s looking at the PR in isolation, not the project. If that makes sense.
Sounds cool. We'll take a look!