Nice! Would recommend to decouple as much as possible and have tons of testing, every issue thats catched before having to go to the length of debugging client side issues is gonna save some sleep. Have been there, learned the hard way..
Yeah it was a deceptively complicated issue. I’m not sure if my implementation is as comprehensive, but it sounds similar: optimistic local writes, outbound queue, server as source of truth, each client has local SQLite.
The issue I found most difficult architecturally was how to deal with staying responsive on first sign in with a large DB. I ended up with a system I called “materialisation on demand” where it will search the server and download only those rows ahead of time.
EDIT: I also ended up using multiple local databases to ensure responsive writes during heavy synching activity.
This is cool. Reminds of Lotus Notes. In a strange way a concept that felt "outdated" is something that I would love to be the default behavior for many apps today.
It’s great to see apps being shipped with offline first (and flawless syncing as an optional feature) from ground up, sad to see many modern notes apps like notion or capacities treating offline as afterthought. But alas, it’s still a non trivial nut to crack.
Hope one day offline first will be a default and a solved problem.
Hi HN! I’ve been circling this problem for a long time.
In 2019 I built an offline-first datastore called debe. It used CRDTs and multi-master replication and never became something I could confidently ship. It did teach me that convergence is only a small part of sync. The difficult parts are durable writes, authorization changes, lost acknowledgements, schema upgrades, bounded bootstrap, and explaining what happened after something fails.
I eventually came back to the problem and built Syncular over the last year.
Every client has a real SQLite database: sqlite-wasm on OPFS in the browser and native SQLite elsewhere. Writes apply locally and enter a durable outbox in the same transaction. A server-authoritative commit log validates and orders them, using explicit scopes resolved inside the application backend.
The protocol is written down independently of the implementations. SSP2 is currently a draft, with canonical binary encoding and golden byte vectors. There are independent TypeScript and Rust client cores, both run against the same 95-scenario conformance catalog. The Rust core is exposed through Swift, Kotlin, Flutter, React Native, Tauri, Rust, and a small C API.
It also includes realtime WebSocket sync, resumable bootstrap segments, explicit conflict evidence, authorization revocation and local purge, windowed replicas, blobs, optional CRDT columns, per-column encryption, and typesafe generated queries for five languages.
Important boundaries: it is pre-1.0, self-hosted, and there is no managed service or peer-to-peer mode. Native packaging maturity still varies by ecosystem.
Let me know what you think, I'm happy about feedback!
There was a local-first conv just a few weeks ago in Berlin; if you're not yet part of that community I suggest reaching out. https://www.localfirst.fm/ for example, or https://lofi.so/
In which way this is different from PowerSync?
Interesting! I wrote something similar for my app recently, it was the first thing I coded using LLMs.
Nice! Would recommend to decouple as much as possible and have tons of testing, every issue thats catched before having to go to the length of debugging client side issues is gonna save some sleep. Have been there, learned the hard way..
Whats your app?
Yeah it was a deceptively complicated issue. I’m not sure if my implementation is as comprehensive, but it sounds similar: optimistic local writes, outbound queue, server as source of truth, each client has local SQLite.
The issue I found most difficult architecturally was how to deal with staying responsive on first sign in with a large DB. I ended up with a system I called “materialisation on demand” where it will search the server and download only those rows ahead of time.
EDIT: I also ended up using multiple local databases to ensure responsive writes during heavy synching activity.
The app is https://www.benko.app/ it’s available on iOS, android, Mac and windows.
This is cool. Reminds of Lotus Notes. In a strange way a concept that felt "outdated" is something that I would love to be the default behavior for many apps today.
It’s great to see apps being shipped with offline first (and flawless syncing as an optional feature) from ground up, sad to see many modern notes apps like notion or capacities treating offline as afterthought. But alas, it’s still a non trivial nut to crack.
Hope one day offline first will be a default and a solved problem.
Hi HN! I’ve been circling this problem for a long time.
In 2019 I built an offline-first datastore called debe. It used CRDTs and multi-master replication and never became something I could confidently ship. It did teach me that convergence is only a small part of sync. The difficult parts are durable writes, authorization changes, lost acknowledgements, schema upgrades, bounded bootstrap, and explaining what happened after something fails.
I eventually came back to the problem and built Syncular over the last year.
Every client has a real SQLite database: sqlite-wasm on OPFS in the browser and native SQLite elsewhere. Writes apply locally and enter a durable outbox in the same transaction. A server-authoritative commit log validates and orders them, using explicit scopes resolved inside the application backend.
The protocol is written down independently of the implementations. SSP2 is currently a draft, with canonical binary encoding and golden byte vectors. There are independent TypeScript and Rust client cores, both run against the same 95-scenario conformance catalog. The Rust core is exposed through Swift, Kotlin, Flutter, React Native, Tauri, Rust, and a small C API.
It also includes realtime WebSocket sync, resumable bootstrap segments, explicit conflict evidence, authorization revocation and local purge, windowed replicas, blobs, optional CRDT columns, per-column encryption, and typesafe generated queries for five languages. Important boundaries: it is pre-1.0, self-hosted, and there is no managed service or peer-to-peer mode. Native packaging maturity still varies by ecosystem.
Let me know what you think, I'm happy about feedback!
Demo: https://demo.syncular.dev Protocol: https://github.com/syncular/syncular/blob/main/docs/SPEC.md Background: https://syncular.dev/blog/offline-first-writes/ Docs: https://syncular.dev/
There was a local-first conv just a few weeks ago in Berlin; if you're not yet part of that community I suggest reaching out. https://www.localfirst.fm/ for example, or https://lofi.so/