Looks pretty cool. After checking out spec.md, one thing I might suggest is using UUIDs for external codec ids. There's a lot of codecs/formats out there, and limiting to a u8 might lead to collisions.
It might also be nice to provide a mechanism to advertise the required codecs toward the beginning of the stream, in case the consumer does not have the necessary codecs and wishes to abort the transfer.
Good catch — you’re absolutely right to call that out.
The current u8 codec ID is mainly there to keep the block header very small and fast to parse, but it’s not meant to be the global identifier. The idea is to map that ID to something globally unique (most likely a UUID) through the plugin/manifest layer, so we can avoid collisions without bloating the on-disk format.
I also like the suggestion about advertising required codecs early in the stream. That would make it much nicer for a reader to fail fast if it doesn’t support something, especially for streaming use cases. We’re exploring adding a small capability section near the beginning for exactly that reason.
Since the format is still experimental, this kind of feedback is really helpful before we lock things down.
We did run internal benchmarks and experiments, but we focused on measuring our own performance rather than doing side-by-side comparisons with other tools. The goal was to validate stability, speed, and resource usage in real scenarios, not to create a public comparison that could be misleading or depend on many external factors. We’re planning to share our methodology and raw numbers so people can evaluate the results themselves and run their own comparisons if they want.
Looks pretty cool. After checking out spec.md, one thing I might suggest is using UUIDs for external codec ids. There's a lot of codecs/formats out there, and limiting to a u8 might lead to collisions.
It might also be nice to provide a mechanism to advertise the required codecs toward the beginning of the stream, in case the consumer does not have the necessary codecs and wishes to abort the transfer.
Good catch — you’re absolutely right to call that out.
The current u8 codec ID is mainly there to keep the block header very small and fast to parse, but it’s not meant to be the global identifier. The idea is to map that ID to something globally unique (most likely a UUID) through the plugin/manifest layer, so we can avoid collisions without bloating the on-disk format.
I also like the suggestion about advertising required codecs early in the stream. That would make it much nicer for a reader to fail fast if it doesn’t support something, especially for streaming use cases. We’re exploring adding a small capability section near the beginning for exactly that reason.
Since the format is still experimental, this kind of feedback is really helpful before we lock things down.
sounds great, may you please share some benchmarks/experiments?
We did run internal benchmarks and experiments, but we focused on measuring our own performance rather than doing side-by-side comparisons with other tools. The goal was to validate stability, speed, and resource usage in real scenarios, not to create a public comparison that could be misleading or depend on many external factors. We’re planning to share our methodology and raw numbers so people can evaluate the results themselves and run their own comparisons if they want.