Anyone here using semantic diffing tools in their daily work? How good are they?
I use some for e.g. YAML [0] and JSON [1], and they're nice [2], but these are comparatively simple languages.
I'm particularly curious because just plain diffing ASTs is more on the "syntax-aware diffing" side rather than the "semantic diffing" side, yet most semantic tooling descriptions stop at saying they use ASTs.
ASTs are not necessarily in a minimal / optimized form by construction I believe, so I'm pretty sure you'll have situations where a "semantic" differ will report a difference, whereas a compiler would still compile the given translation unit to the same machine bytecode after all the optimization passes during later levels. Not even necessarily for target platform dependent reasons.
But maybe this doesn't matter much or would be more confusing than helpful?
[2] they allow me to ignore ordering differences within arrays (arrays are ordered in YAML and JSON as per the standard), which I found to be a surprisingly rare and useful capability; the programs that consume the YAMLs and JSONs I use these on are not sensitive to these ordering differences
I guess I don't understand the difference between semantic and syntax-aware, but I've been trying out difftastic which is a bit of an odd beast but does a great job at narrowing down diffs to the actual meaningful parts.
Was "sem" named "graft" last week and "got" a week before that? Everyone is vibing so hard it is difficult to keep track of things. Also, idea theft gets to entirely new levels. Bot swarms promote 100% vibed stolen projects... what a moment in time we all enjoy.
Still, my two cents: Beagle the AST-level version control system, experimental
It genuinely stores AST trees in (virtually any) key-value database (RocksDB at the moment). In fact, it is a versioned database for the code with very open format and complete freedom to build on top of it.
be is in fact, more of a format/protocol, like in the good old days (HTTP, SMTP, XML, JSON - remember those?)
I like this but would think "Semantic" is pushing it a bit as it relies on function names and changes therein having a direct mapping to meaning with standard text processing.
In fact, I fully expected a use of LLM to derive those meaningful descriptions before I checked the repo.
Anyway I definitely see this as a useful thing to try out as a potential addition to the armoury, but as we go further along the route to AI-coding I expect the diffs to be abstracted even further (derived using AI), for use by both agentic and human contributors.
cool! Line-level diffs only made sense when humans wrote every line. When AI agents are committing hundreds of changes a session, you need to know which functions changed, not which lines..
Line-level diffs didn't even make sense when humans wrote every line. It's just combining two different things—text-based files and formal languages—in ways that were never going to make sense. We've all wasted days of our lives fixing crappy diffs because nobody bothered to develop something that made sense for the files we were tracking outside of "everything is text who cares text is magic"
Still, some languages/heuristics handle this better than others.
Anyone here using semantic diffing tools in their daily work? How good are they?
I use some for e.g. YAML [0] and JSON [1], and they're nice [2], but these are comparatively simple languages.
I'm particularly curious because just plain diffing ASTs is more on the "syntax-aware diffing" side rather than the "semantic diffing" side, yet most semantic tooling descriptions stop at saying they use ASTs.
ASTs are not necessarily in a minimal / optimized form by construction I believe, so I'm pretty sure you'll have situations where a "semantic" differ will report a difference, whereas a compiler would still compile the given translation unit to the same machine bytecode after all the optimization passes during later levels. Not even necessarily for target platform dependent reasons.
But maybe this doesn't matter much or would be more confusing than helpful?
[0] dyff: https://github.com/homeport/dyff
[1] jd: https://github.com/josephburnett/jd
[2] they allow me to ignore ordering differences within arrays (arrays are ordered in YAML and JSON as per the standard), which I found to be a surprisingly rare and useful capability; the programs that consume the YAMLs and JSONs I use these on are not sensitive to these ordering differences
I guess I don't understand the difference between semantic and syntax-aware, but I've been trying out difftastic which is a bit of an odd beast but does a great job at narrowing down diffs to the actual meaningful parts.
Was "sem" named "graft" last week and "got" a week before that? Everyone is vibing so hard it is difficult to keep track of things. Also, idea theft gets to entirely new levels. Bot swarms promote 100% vibed stolen projects... what a moment in time we all enjoy.
Still, my two cents: Beagle the AST-level version control system, experimental
https://github.com/gritzko/librdx/tree/master/be#readme
It genuinely stores AST trees in (virtually any) key-value database (RocksDB at the moment). In fact, it is a versioned database for the code with very open format and complete freedom to build on top of it.
be is in fact, more of a format/protocol, like in the good old days (HTTP, SMTP, XML, JSON - remember those?)
I like this but would think "Semantic" is pushing it a bit as it relies on function names and changes therein having a direct mapping to meaning with standard text processing.
In fact, I fully expected a use of LLM to derive those meaningful descriptions before I checked the repo.
Anyway I definitely see this as a useful thing to try out as a potential addition to the armoury, but as we go further along the route to AI-coding I expect the diffs to be abstracted even further (derived using AI), for use by both agentic and human contributors.
> I like this but would think "Semantic" is pushing it a bit
It would be nice to get to the feature level, meaning across files/classes/functions etc.
Cool, knowledge graph matching on top of commits
Please could you add cargo binstall support?
I was just building a POC of something like this a couple of weeks ago. I'm glad someone else already implemented it with support for more languages.
cool! Line-level diffs only made sense when humans wrote every line. When AI agents are committing hundreds of changes a session, you need to know which functions changed, not which lines..
Line-level diffs didn't even make sense when humans wrote every line. It's just combining two different things—text-based files and formal languages—in ways that were never going to make sense. We've all wasted days of our lives fixing crappy diffs because nobody bothered to develop something that made sense for the files we were tracking outside of "everything is text who cares text is magic"
Still, some languages/heuristics handle this better than others.
Yeah, as long as the function gets called, it’ll work!
_sad panda face_