This looks amazing!
I have a question. When we create a branch, are we cloning the entire production data? From what I understood was that neon runs a separate compute layer but the storage is the same.
Neon's engineer here: If you're asking about how branches work in general, they use Copy-on-Write (CoW), which means that no data is copied when they are created. However, when writes occur in a branch, only the modified specific Postgres pages are copied (working like deltas in your data), which is why they are created very quickly and don't increase storage usage when created, increasing only in the modified portion.
If you're asking about how the anonymized branches work, they currently use static masking, which means that all PII data selected for masking will be overridden in the anonymized branch, resulting in increased storage usage based on the amount of selected masked data. However, the team is also working on dynamic masking, which doesn't change the stored data but applies the masking rules when querying it.
This looks amazing! I have a question. When we create a branch, are we cloning the entire production data? From what I understood was that neon runs a separate compute layer but the storage is the same.
Neon's engineer here: If you're asking about how branches work in general, they use Copy-on-Write (CoW), which means that no data is copied when they are created. However, when writes occur in a branch, only the modified specific Postgres pages are copied (working like deltas in your data), which is why they are created very quickly and don't increase storage usage when created, increasing only in the modified portion.
If you're asking about how the anonymized branches work, they currently use static masking, which means that all PII data selected for masking will be overridden in the anonymized branch, resulting in increased storage usage based on the amount of selected masked data. However, the team is also working on dynamic masking, which doesn't change the stored data but applies the masking rules when querying it.
And to clarify - only writes on the _branch_ lead to a copy being created right? Writes on the original don’t get propagated to the branch?