> This write amplification is large enough that our efforts to tune indexing throughput have started to hit diminishing returns.
> don't key on the ANN address. That is precisely the change turbopuffer v3 makes. As you can imagine, it is not a trivial change.
This is a direct parallel to how Postgres and Mysql built indexes.
Your design choice went from a Postgres design pattern to a Mysql one. The difference is the reindexing cost vs the lookup cost - Postgres optimized for lookup and Mysql does for indexing on writes. Or more accurately, Postgres was better with good schema design using joins & mysql was optimized for a bad design with less normalization where many indexes exist for the same table.
Postgres always points an index to a row-id within postgres which is an arbitrary value which changes on each update.
Mysql, always assuming the storage engine is pluggable, points to the primary index entry and adds an extra indirection to the lookup.
This means that you point the mysql index to a stable id, so unless you go update the primary key for a row, you won't have to update the indexes for all the attribute lookups you might have made to data.
I don't do databases any more that much, but the design for NIMBLE file format has a lot of quirks which are relevant to this specific idea (wide tables).
But the old Uber post about switching from Postgres to Mysql to prevent index amplification[1] is a direct mirror to this post.
> Your design choice went from a Postgres design pattern to a Mysql one. The difference is the reindexing cost vs the lookup cost - Postgres optimized for lookup and Mysql does for indexing on writes. Or more accurately, Postgres was better with good schema design using joins & mysql was optimized for a bad design with less normalization where many indexes exist for the same table
You are right that MySQL does better when you have lots of indexes, but I don't think the tradeoff is that the overall Postgres architecture is better with good schema design.
Having secondary indexes point the primary key enables things like undo logging, which obviates the need for vacuums - vacuums being the most painful part of Postgres. On top of that your primary key index will be mostly cached so the cost of the indirection is much smaller than it may first appear
Many of the commenters on this site are also obviously LLMs. I'd imagine that quite a few of the entities quoting aren't necessarily people. Keep an eye on where they slide mentions of other products that a marketing team would like to promote.
Personally I haven't seen it too often; the other aspect is that HN is a forum for startups to pitch shit to each other, so this has been happening with or without marketers (f.e. "i'm working on a similar thing").
That LLMs are taking over the comments section is something that was already flagged, and Lobste.rs and others have started solving it by having gated registrations. HN should do this but it is unlikely to until it is too late.
Is it time to kill the database and replace it with a LLM optimized compiled version that simply implements the required API directly in (Rust) code, without any dynamic overhead? It probably will still be based of off a base design or a base file format.
Ultimately this system will encompass the whole OS, of course, but the DB might be the best place to start.
We've realized this a long time ago at TopK and built a flexible serverless search engine from scratch. Supports dense/sparse vectors, late interaction, lexical search, indexed regex, filtering, and custom scoring in one query.
I've really liked lancedb for similar use cases. Not just that it is OSS. But Lance treats ANN as a secondary index similar to what turbopuffer v3 does. Rows sit in fragments, and the vector index never moves them.
Vector databases were always more about retrieval than either vectors or data storage. But the term stuck all too well and companies held on to it a tad too long. Sorry :)
That's just a search engine, but then you're competing with traditional players like Elasticsearch and Vespa who all have built-in vector support by now, and you have to compete on attributes like price, performance, features, and who can mention 'AI' the most times on their web page.
Im not full read up on RAG pipelines, but has anyone ever tried to make the database a neural net itself? I.e get rid of any sort of traditional databases, and then you basically just have some sort of autoencoder?
I find very little reason to use a pure vector database for enterprise retrieval. We built an enterprise retrieval engine on top of a SQL database with native vector support, and the flexibility is something we cannot ignore. Vector similarity is just one query primitive alongside full text search, filters, joins, ordering and normal relational predicates. Tenant/app/collection isolation becomes part of the query itself. ACLs, document versions, categories, metadata constraints and temporal filters are ordinary predicates rather than something you have to bolt onto a vector store. SQL is already going to be part of almost any enterprise system. Adding a separate vector database introduces another moving part and syncing two system whenever you update your data is the most difficult thing to get right.
This sounds like the Postgres vs. InnoDB argument 10 years later. Postings pointed at physical location (the ANN slot), so every SPFresh rebalance rewrote every index touching that doc. InnoDB solved this by pointing secondary indexes at the PK and eating an extra lookup on read. Curious what that extra lookup costs you when it's an S3 GET instead of a B-tree hop.
"Updating one vector can move hundreds of attributes and their indexes" is basically Uber's 2016 Postgres write amplification post, but for search. Same fix too: stop pointing indexes at where the row lives.
So ANN becomes a secondary index that points at a doc ID, and vector search now needs a hop to complete. Do clusters keep their own copy of the vectors so the search itself stays local, and only result fetch pays the indirection? Otherwise cold p99 seems like it gets worse.
Yes, I get big money from the Internet Archive to promote their services. It's the new scheme that shills like me go for.
The reason is that I have no account on HN and rarely comment. I create a new account a few times a year because I don't remember or care about my previous account.
I could have made an account named john2026 and you would not think twice. Instead, I let people know upfront what type of account this is. Quite the opposite of what a true shill would do.
I got a Lighthouse score of 99 in Chrome. Believe it or not, I won't spend more of our time on this. (relevant XKCD: https://xkcd.com/386/ )
First Contentful Paint
0.7 s
Largest Contentful Paint
0.9 s
Speed Index
0.7 s
It makes a lot of requests, and some are stopped by my ad blocker, but most of them don't seem to make an difference. It is almost instant from my point of view. I disabled the ad blocker and didn't notice any visual difference.
Do you actually think that 10G makes pages load faster than 1G or even 100M? It doesn't. The blocker was most likely on the source server, not on your side.
> This write amplification is large enough that our efforts to tune indexing throughput have started to hit diminishing returns.
> don't key on the ANN address. That is precisely the change turbopuffer v3 makes. As you can imagine, it is not a trivial change.
This is a direct parallel to how Postgres and Mysql built indexes.
Your design choice went from a Postgres design pattern to a Mysql one. The difference is the reindexing cost vs the lookup cost - Postgres optimized for lookup and Mysql does for indexing on writes. Or more accurately, Postgres was better with good schema design using joins & mysql was optimized for a bad design with less normalization where many indexes exist for the same table.
Postgres always points an index to a row-id within postgres which is an arbitrary value which changes on each update.
Mysql, always assuming the storage engine is pluggable, points to the primary index entry and adds an extra indirection to the lookup.
This means that you point the mysql index to a stable id, so unless you go update the primary key for a row, you won't have to update the indexes for all the attribute lookups you might have made to data.
I don't do databases any more that much, but the design for NIMBLE file format has a lot of quirks which are relevant to this specific idea (wide tables).
But the old Uber post about switching from Postgres to Mysql to prevent index amplification[1] is a direct mirror to this post.
[1] - https://www.uber.com/us/en/blog/postgres-to-mysql-migration/
> Your design choice went from a Postgres design pattern to a Mysql one. The difference is the reindexing cost vs the lookup cost - Postgres optimized for lookup and Mysql does for indexing on writes. Or more accurately, Postgres was better with good schema design using joins & mysql was optimized for a bad design with less normalization where many indexes exist for the same table
You are right that MySQL does better when you have lots of indexes, but I don't think the tradeoff is that the overall Postgres architecture is better with good schema design.
Having secondary indexes point the primary key enables things like undo logging, which obviates the need for vacuums - vacuums being the most painful part of Postgres. On top of that your primary key index will be mostly cached so the cost of the indirection is much smaller than it may first appear
> mysql was optimized for a bad design
TIL I should have been using mysql the whole time
Richard Gabriel's "Worse is better" vibes.
I find it amusing people started quoting LLM output and are responding to it. Hopefully the original authors end up having the LLM respond back.
Many of the commenters on this site are also obviously LLMs. I'd imagine that quite a few of the entities quoting aren't necessarily people. Keep an eye on where they slide mentions of other products that a marketing team would like to promote.
Personally I haven't seen it too often; the other aspect is that HN is a forum for startups to pitch shit to each other, so this has been happening with or without marketers (f.e. "i'm working on a similar thing").
That LLMs are taking over the comments section is something that was already flagged, and Lobste.rs and others have started solving it by having gated registrations. HN should do this but it is unlikely to until it is too late.
Is it time to kill the database and replace it with a LLM optimized compiled version that simply implements the required API directly in (Rust) code, without any dynamic overhead? It probably will still be based of off a base design or a base file format.
Ultimately this system will encompass the whole OS, of course, but the DB might be the best place to start.
You mean get rid of Postgres and build bespoke database-esque systems for every use case?
If so, then no. It is not time for that.
> The problem with a vector primary index
We've realized this a long time ago at TopK and built a flexible serverless search engine from scratch. Supports dense/sparse vectors, late interaction, lexical search, indexed regex, filtering, and custom scoring in one query.
- https://www.topk.io/blog/vector-dbs-are-the-wrong-abstractio... - https://www.topk.io/blog/topk-embed-v1
I've really liked lancedb for similar use cases. Not just that it is OSS. But Lance treats ANN as a secondary index similar to what turbopuffer v3 does. Rows sit in fragments, and the vector index never moves them.
Vector databases were always more about retrieval than either vectors or data storage. But the term stuck all too well and companies held on to it a tad too long. Sorry :)
That's just a search engine, but then you're competing with traditional players like Elasticsearch and Vespa who all have built-in vector support by now, and you have to compete on attributes like price, performance, features, and who can mention 'AI' the most times on their web page.
Waitint for the CEO of Qdrant to step in
the multi-vector duplication thing makes sense, copying every attribute once per vector explodes quickly. what's the new primary index?
Im not full read up on RAG pipelines, but has anyone ever tried to make the database a neural net itself? I.e get rid of any sort of traditional databases, and then you basically just have some sort of autoencoder?
I find very little reason to use a pure vector database for enterprise retrieval. We built an enterprise retrieval engine on top of a SQL database with native vector support, and the flexibility is something we cannot ignore. Vector similarity is just one query primitive alongside full text search, filters, joins, ordering and normal relational predicates. Tenant/app/collection isolation becomes part of the query itself. ACLs, document versions, categories, metadata constraints and temporal filters are ordinary predicates rather than something you have to bolt onto a vector store. SQL is already going to be part of almost any enterprise system. Adding a separate vector database introduces another moving part and syncing two system whenever you update your data is the most difficult thing to get right.
This sounds like the Postgres vs. InnoDB argument 10 years later. Postings pointed at physical location (the ANN slot), so every SPFresh rebalance rewrote every index touching that doc. InnoDB solved this by pointing secondary indexes at the PK and eating an extra lookup on read. Curious what that extra lookup costs you when it's an S3 GET instead of a B-tree hop.
"Updating one vector can move hundreds of attributes and their indexes" is basically Uber's 2016 Postgres write amplification post, but for search. Same fix too: stop pointing indexes at where the row lives.
So ANN becomes a secondary index that points at a doc ID, and vector search now needs a hop to complete. Do clusters keep their own copy of the vectors so the search itself stays local, and only result fetch pays the indirection? Otherwise cold p99 seems like it gets worse.
It would be nice to have a page that actually loads. This one doesn't. RIP.
UPDATE: It loads now, but it didn't when it was first posted. Traffic load on the server does matter.
Loads really fast for me. (MacBook Air, average internet)
If you still have issues, try https://web.archive.org/web/20261001100105/https://turbopuff...
It has a pagespeed insights score of 55 and noticeably sluggish on my m3 max.
And what's with the throwaway account for this one comment? Is this becoming reddit with throwaway shills now?
fucking shills, making helpful comments and promoting seemingly nothing, what's this place coming to?
Yes, I get big money from the Internet Archive to promote their services. It's the new scheme that shills like me go for.
The reason is that I have no account on HN and rarely comment. I create a new account a few times a year because I don't remember or care about my previous account.
I could have made an account named john2026 and you would not think twice. Instead, I let people know upfront what type of account this is. Quite the opposite of what a true shill would do.
I got a Lighthouse score of 99 in Chrome. Believe it or not, I won't spend more of our time on this. (relevant XKCD: https://xkcd.com/386/ )
First Contentful Paint 0.7 s
Largest Contentful Paint 0.9 s
Speed Index 0.7 s
It makes a lot of requests, and some are stopped by my ad blocker, but most of them don't seem to make an difference. It is almost instant from my point of view. I disabled the ad blocker and didn't notice any visual difference.
loads just fine on my $10k laptop with 10g internet here in NYC
Also loads fine on my beater in the sticks :)
Takes 11 seconds to load on Firefox on Linux with 3G-level throttling enabled in Dev Tools.
Do you actually think that 10G makes pages load faster than 1G or even 100M? It doesn't. The blocker was most likely on the source server, not on your side.
You're the reason the /s tag has to exist.