Works for Linear because the tab stays open, and worse case if tab is closed you can recover later when the tab is opened again and deal with conflict resolution. Won't work if:
1. user clicks a button and closes the tab thinking transaction is done and it's important that transaction is done
2. conflict resolution is difficult or impossible in future client wake up
The user clicks the button, the mutation is stored in local storage. The user closes the tab, but it's not a problem.
A background worker picks up the mutation and sends it to the remote backend. It takes time, retries, etc.
Similarly, any errors reported by the background worker go to local store, and the next time the UI tab is loaded / activated, they are shown. A service worker can show a notification outright to let the user easily load the main UI. Normally this would be a rare occasion.
There's guaranteed persistence, but there's no guarantee that the host will be up anytime soon. E.g.: I might leave a final reply with all the details on an issue before going on vacation (or maybe I don't work the next day but my colleagues abroad do!). I see that it's properly posted and close the laptop.
The reply with be delayed by days or weeks, but the UI indicated that it had been properly saved.
I agree. To each their own, but the UI updating automatically doesn't really add much value to me. I would prefer that the view I am seeing is a snapshot in time of what the ground truth server was, not some mixed state that forces me to consider the possibility that seeing my request go through on the screen doesn't actually mean it went through and has been sent to the server.
Indeed. I have to say, I hate this. Suppose you are in a meeting, you update something and you see the result, but the rest of the team does not. Ok, a couple of hundred ms does not play into this but if the update does not make it through? And yes, it happens.
Changes go through and synced to everyone on your team in almost realtime. If there's a conflict on the server and your change cannot be applied (almost never happens), your change is rolled back on your client, again, almost in realtime. If servers cannot be reached, we will show you a syncing badge within 4 seconds to tell you that you have made changes that haven't been sent to others yet.
Strange that we can be so be polar opposites on this. You hate it, I would never write an app in any other way, ever again.
(curious) What if a user closes it before 4 seconds? Ctrl+enter, it optimistically locally updates within 1 second. I close ctrl+w. But my wifi goofed and it didn't reach the server.
It depends on where your clients are, and where your servers are. You may have a lot of customers in a country and not want to host servers in that country (tax, regulatory, maintenance cost, etc.)
RTT from Hyderabad to the East Coast USA is ~300ms.
Yeah you can easily get this speed down to like 100ms which is near enough to instant to not matter. You definitely don't need to go to client side rendering for most things to feel fast.
Also I couldn't see any explanation of what happens when a network request fails. That's a huge downside of local rendering.
I think for most sites the best option is still server-side rendering but use a fast backend (e.g. not Python) and lightweight frontend.
I love using linear and where applicable I try to use optimistic updates in the applications I build too and I think it's a good article that touches on the most important aspects of a performant web app of that caliber.
One thing I would like to point out though is that building a performant sync engine that behaves the way you would like in most cases is a non-trivial thing.
If two users are offline and add, edit, remove issues and come online again, you need to reason about what happens. Sometimes you can get away with Last Writer Wins but what happens if an issue is deleted and then edited. What happens if an item in a list gets re-ordered differently on different clients, what's the final ordering? In which cases can you merge what state and in which do you need to discard something. Do you show a conflict resolution UI? How do you deal with rollbacks. How do you deal with schema drift and updates on items that would be affected by schema drift? Business logic might change between being reconnects. FK constraints can shift. Can you set up your data and the sync engine so that it only syncs the minimum amount of changes and batches them correctly during longer offline sessions so you don't fire 5000 change requests after reconnecting?
I recently had to implement local first + remote sync on some fairly complex dynamic forms and where luckily there is usually only one writer and I can get away with last writer wins and reject if things are too old or if there is schema drift and can just display an error message and roll back. But what I am trying to say is: Whatever can go wrong in an online-first world, can also go wrong in an offline first world but you might get informed of that all at once at a later time - or not at all and your data is not what you think it should be. Some sync engines like zero from rocicorp has opted out of supporting offline writes entirely because of all these problems.
And just to be clear: I love offline first approaches. I yearn for fast performance. And in a lot of cases slapping a sync engine on your app can really be helpful for that if your use case allows it. But it's absolutely crucial to be aware of the pitfalls that come with it.
I'm glad to see more positive takes on client-side rendering. Unless SEO and crawlability are important for your site, server side rendering is such an overprescribed solution. If I put my tin-foil hat on, I'd say it gets a lot of attention because it's a lot easier to charge people for server time spent in SSR that you just don't have in CSR.
Bottom line is, if your app's content is behind a login screen, just use client side rendering. It is way lower complexity and a way better user experience.
I worry that optimistic updates is going to become trendy and applied to more software, but without any plan for the "sad path" - failed to sync, sync conflict, etc. Get ready for a whole new era of race conditions and frustration!
Don't you have the same problems with basic CRUD apps? Also you need to handle the sad path for every single request instead of having the sync engine do it all in one place.
Correct but the feedback is usually more immediate. Save a change to your issue and it fails - You will get an error toast and probably stay on the form.
In the local first world you might have navigated away already and created 3 more issues of which 2 more failed because of schema drift or other conflicts. And you might have edited one that was deleted. And now you need to figure out what exactly to tell the user - or what not to tell them.
Well the sync engine can figure out if there's an issue fast, say <500ms, if you build it that way. Then you can just make a toast telling the user there are issues and anything they do will be saved locally only for the time being.
Warn the user that if they leave the website their changes won't be saved remotely.
The issue that I foresee is that the point of error becomes decoupled from the UI and the UI doesn't handle a delayed error. Especially if retrofit into existing products.
You can locate an "intermediary backend" on the client, write outstanding mutations into local storage, have a background worker send it to the backend, with necessary retries, etc. At worst, the background worker would put out a message about a failed update which the UI tread would receive and show.
Thanks for the read. It is a bit more complicated than you think.
I completely rebuilt this sync engine + orm with relations, lazy loading etc, using Vue + Pinia in https://playcode.io. Google Linear’s videos, they explained in detail their architecture.
Yes, I spent a few months. But it worth it. Every new field, model I need to add, it is so straightforward. I do love frameworks and foundations. They make live easier later by a lot.
In gamedev, "optimistic updates" are called "client-side prediction," and are a standard part of multiplayer games. IMO it's somewhat risky to apply the technique to web-apps, since each network request typically corresponds to some important operation, and optimistically updating the UI is lying to the user about whether that operation completed successfully.
IMO a good approach is to update the UI immediately but still show some indication that the operation hasn't completed. So in a chat app, for example, add the message to the list of messages, but with contrast reduced slightly to indicate that other people can't see it yet.
Operations are on average applied within a few hundred milliseconds, and almost never fail. Because of this we treat the success path as default, and indicate that your changes haven't been applied only if we detect that you're offline, or if it takes more than 4 second to apply the changes.
This is bizarrely laudatory. The app is fast because it is not correct. The user has no way to know if their view is consistent with any other user's view. The user has no way to know if the app silently discarded one of their inputs because of a conflict. Linear developers seem to believe that silent data loss is an acceptable cost for maintaining the illusion of speed.
Honestly you can POST to an API to the other side of the world and receive a response in less than 300ms. That's literally the blink of an eye. There are very few use cases where you'd need even lower latency than that while at the same time sacrificing reliability (there's no guaranteed persistent storage in a browser).
They're doing some really cool stuff with Agents too.. automating software pm. it's a huge difference between this and their (unnamed) larger competitors.
I've never used linear, but just watching their example video [1] would worry me if I was a user and it's actually doing what they say it's doing.
He creates a task called "Create faster app launch", if we believe the article, it's processing that locally rather than going via the server, and then it's allocated an ID "BRO-5". That the ID is so low suggests it's just adding one to the previous issue ID, and so under heavy load, there are almost certainly going to be conflicts with other users creating tasks and getting identical IDs. Even if the system resolves this by changing one of those IDs, the system shouldn't be presenting the ID to the user until it is guaranteed unique. What if they've already pasted it into a document when the system notices the collision and renumbers it?
EDIT: WTH, there are some seriously bad karma people in this thread - just because I dared to have an opinion that the approach taken by this software might not be the best, my post was downvoted in less than a minute after posting! I'm sure whoever did that carefully considered my argument. If I'm wrong, explain why, don't just downvote my comment. If I'm not wrong, shame on you.
TL;DR: Background syncing instead of synchronous updates to the cloud.
This is basically a thick client, and comes with according trade-offs. It's interesting and there are some best practices, but I can't help but feeling that either the author is a huge fan or the post is an ad (or "sponsored").
Linear is the best web app I have ever seen, period. It is also the best bug-tracker I've ever used. I use it, and pay for it (gladly). It's worth it.
I am genuinely impressed with their engineering and design — I aspire to attain these levels, though I lack not only the skills, but also several zeroes in my bank account, I think. Still, it's worth looking at what they do and try to get there!
Big props and kudos to the Linear team. It's an impressive app.
Interestingly, as a professional developer I had the opposite first experience. It's a tracker. It looks like every other tracker. Everything is really small and "clean" which means they just hide everything, for that modern look.
I was thoroughly unimpressed. "This is it?" - "It's a tracker"
I feel comfortable saying this because in the weeks after, actually using it is the aha experience. Linear nailed the UX of working where people already work. Which again is really funny because the best part of Linear is how well it works outside of Linear.
(disclaimer: I actually now use their UI a lot. It's a helpful dashboard. But it suffers from every other hard-problem of information dense task-based dashboards.)
Last year a guy reverse engineered Linear's sync engine and published it on GitHub with a cool explainer.
https://github.com/wzhudev/reverse-linear-sync-engine/blob/m...
Whole blog post is basically: Make a mutation in the clientside, assume it worked, and save in the background.
Works for Linear because the tab stays open, and worse case if tab is closed you can recover later when the tab is opened again and deal with conflict resolution. Won't work if:
1. user clicks a button and closes the tab thinking transaction is done and it's important that transaction is done
2. conflict resolution is difficult or impossible in future client wake up
The user clicks the button, the mutation is stored in local storage. The user closes the tab, but it's not a problem.
A background worker picks up the mutation and sends it to the remote backend. It takes time, retries, etc.
Similarly, any errors reported by the background worker go to local store, and the next time the UI tab is loaded / activated, they are shown. A service worker can show a notification outright to let the user easily load the main UI. Normally this would be a rare occasion.
For native apps this is less of an issue since they have access to persistent storage but with browsers there's no guaranteed persistence.
There's guaranteed persistence, but there's no guarantee that the host will be up anytime soon. E.g.: I might leave a final reply with all the details on an issue before going on vacation (or maybe I don't work the next day but my colleagues abroad do!). I see that it's properly posted and close the laptop.
The reply with be delayed by days or weeks, but the UI indicated that it had been properly saved.
> There's guaranteed persistence
There's not. Browsers can delete "persistent" storage at any time.
https://developer.mozilla.org/en-US/docs/Web/API/Storage_API...
It depends. From the link:
> If, for any reason, developers need persistent storage [...] they can do so by using the navigator.storage.persist() method of the Storage API.
This makes a request for guaranteed permanent storage ... which can be approved (or denied) by the user or by browser defaults.
I agree. To each their own, but the UI updating automatically doesn't really add much value to me. I would prefer that the view I am seeing is a snapshot in time of what the ground truth server was, not some mixed state that forces me to consider the possibility that seeing my request go through on the screen doesn't actually mean it went through and has been sent to the server.
Indeed. I have to say, I hate this. Suppose you are in a meeting, you update something and you see the result, but the rest of the team does not. Ok, a couple of hundred ms does not play into this but if the update does not make it through? And yes, it happens.
Changes go through and synced to everyone on your team in almost realtime. If there's a conflict on the server and your change cannot be applied (almost never happens), your change is rolled back on your client, again, almost in realtime. If servers cannot be reached, we will show you a syncing badge within 4 seconds to tell you that you have made changes that haven't been sent to others yet.
Strange that we can be so be polar opposites on this. You hate it, I would never write an app in any other way, ever again.
(curious) What if a user closes it before 4 seconds? Ctrl+enter, it optimistically locally updates within 1 second. I close ctrl+w. But my wifi goofed and it didn't reach the server.
> A traditional CRUD app doing the same thing takes about 300ms.
300ms seems like a lot. Even if this includes the whole http request+response it still seems unbelievably long.
It depends on where your clients are, and where your servers are. You may have a lot of customers in a country and not want to host servers in that country (tax, regulatory, maintenance cost, etc.)
RTT from Hyderabad to the East Coast USA is ~300ms.
Then you have execution and database retrieval.
https://wondernetwork.com/pings
this is kinda missing the point. yes, the raw http calls __should__ be enough for 300ms.
but the benchmark is similar software like JIRA, which takes agonizingly long to do anything reasonable.
Yeah you can easily get this speed down to like 100ms which is near enough to instant to not matter. You definitely don't need to go to client side rendering for most things to feel fast.
Also I couldn't see any explanation of what happens when a network request fails. That's a huge downside of local rendering.
I think for most sites the best option is still server-side rendering but use a fast backend (e.g. not Python) and lightweight frontend.
I love using linear and where applicable I try to use optimistic updates in the applications I build too and I think it's a good article that touches on the most important aspects of a performant web app of that caliber.
One thing I would like to point out though is that building a performant sync engine that behaves the way you would like in most cases is a non-trivial thing.
If two users are offline and add, edit, remove issues and come online again, you need to reason about what happens. Sometimes you can get away with Last Writer Wins but what happens if an issue is deleted and then edited. What happens if an item in a list gets re-ordered differently on different clients, what's the final ordering? In which cases can you merge what state and in which do you need to discard something. Do you show a conflict resolution UI? How do you deal with rollbacks. How do you deal with schema drift and updates on items that would be affected by schema drift? Business logic might change between being reconnects. FK constraints can shift. Can you set up your data and the sync engine so that it only syncs the minimum amount of changes and batches them correctly during longer offline sessions so you don't fire 5000 change requests after reconnecting?
I recently had to implement local first + remote sync on some fairly complex dynamic forms and where luckily there is usually only one writer and I can get away with last writer wins and reject if things are too old or if there is schema drift and can just display an error message and roll back. But what I am trying to say is: Whatever can go wrong in an online-first world, can also go wrong in an offline first world but you might get informed of that all at once at a later time - or not at all and your data is not what you think it should be. Some sync engines like zero from rocicorp has opted out of supporting offline writes entirely because of all these problems.
And just to be clear: I love offline first approaches. I yearn for fast performance. And in a lot of cases slapping a sync engine on your app can really be helpful for that if your use case allows it. But it's absolutely crucial to be aware of the pitfalls that come with it.
I'm glad to see more positive takes on client-side rendering. Unless SEO and crawlability are important for your site, server side rendering is such an overprescribed solution. If I put my tin-foil hat on, I'd say it gets a lot of attention because it's a lot easier to charge people for server time spent in SSR that you just don't have in CSR.
Bottom line is, if your app's content is behind a login screen, just use client side rendering. It is way lower complexity and a way better user experience.
Am I the only that doesn't "get" Linear? Even if speed is the killer feature, isn't an issue tracker a relatively low-frequency application for a dev?
Whenever I use it, I don't feel like I'm doing anything new when compared to all the other issue trackers and Kanban boards I've used before.
I worry that optimistic updates is going to become trendy and applied to more software, but without any plan for the "sad path" - failed to sync, sync conflict, etc. Get ready for a whole new era of race conditions and frustration!
Don't you have the same problems with basic CRUD apps? Also you need to handle the sad path for every single request instead of having the sync engine do it all in one place.
Correct but the feedback is usually more immediate. Save a change to your issue and it fails - You will get an error toast and probably stay on the form.
In the local first world you might have navigated away already and created 3 more issues of which 2 more failed because of schema drift or other conflicts. And you might have edited one that was deleted. And now you need to figure out what exactly to tell the user - or what not to tell them.
Well the sync engine can figure out if there's an issue fast, say <500ms, if you build it that way. Then you can just make a toast telling the user there are issues and anything they do will be saved locally only for the time being.
Warn the user that if they leave the website their changes won't be saved remotely.
In reality conflicts almost never happen.
The issue that I foresee is that the point of error becomes decoupled from the UI and the UI doesn't handle a delayed error. Especially if retrofit into existing products.
These kinds of local-first syncing web apps are really interesting and can be really useful, but I think the premise is somewhat wrong.
"A few milliseconds is all it takes to update an issue in Linear. A traditional CRUD app doing the same thing takes about 300ms."
"Any data sent between the client and server costs hundreds of milliseconds."
There’s no solving the problem of a large RTT between an HTTP client and server when it’s due to the speed of light.
But what you can do is locate the backend near users and make sure it’s fast.
For example, it’s very possible to run a web app backend within ~10ms RTT of most users and have the backend render responses within ~10ms too.
In other words, you can absolutely create a traditional CRUD app where doing the same thing takes more like 30ms, not 300ms.
You can locate an "intermediary backend" on the client, write outstanding mutations into local storage, have a background worker send it to the backend, with necessary retries, etc. At worst, the background worker would put out a message about a failed update which the UI tread would receive and show.
But the happy path stays lightning-fast.
Thanks for the read. It is a bit more complicated than you think. I completely rebuilt this sync engine + orm with relations, lazy loading etc, using Vue + Pinia in https://playcode.io. Google Linear’s videos, they explained in detail their architecture.
Yes, I spent a few months. But it worth it. Every new field, model I need to add, it is so straightforward. I do love frameworks and foundations. They make live easier later by a lot.
Feel free to ask questions.
what google linear video?
I think google is an imperative verb here. You're supposed to google (as in search for) Linears Videos.
https://m.youtube.com/watch?v=Wo2m3jaJixU&ra=m
https://m.youtube.com/watch?v=WxK11RsLqp4&t=2175s&pp=2AH_EJA...
These are original video. Very clear and a way better than the blog post.
I’ve never thought of Linear as particularly fast.
What web app would you put forward instead?
In gamedev, "optimistic updates" are called "client-side prediction," and are a standard part of multiplayer games. IMO it's somewhat risky to apply the technique to web-apps, since each network request typically corresponds to some important operation, and optimistically updating the UI is lying to the user about whether that operation completed successfully.
IMO a good approach is to update the UI immediately but still show some indication that the operation hasn't completed. So in a chat app, for example, add the message to the list of messages, but with contrast reduced slightly to indicate that other people can't see it yet.
Operations are on average applied within a few hundred milliseconds, and almost never fail. Because of this we treat the success path as default, and indicate that your changes haven't been applied only if we detect that you're offline, or if it takes more than 4 second to apply the changes.
I was thinking exactly the same thing, this is CSP/SR. One of my favorite topics! https://gabrielgambetta.com/client-server-game-architecture....
Any good resources or wrappers for indexedDB people would recommend? The API seems kinda unapproachable compared to other data stores.
https://github.com/tinyplex/tinybase seems kinda good maybe?
This is bizarrely laudatory. The app is fast because it is not correct. The user has no way to know if their view is consistent with any other user's view. The user has no way to know if the app silently discarded one of their inputs because of a conflict. Linear developers seem to believe that silent data loss is an acceptable cost for maintaining the illusion of speed.
Honestly you can POST to an API to the other side of the world and receive a response in less than 300ms. That's literally the blink of an eye. There are very few use cases where you'd need even lower latency than that while at the same time sacrificing reliability (there's no guaranteed persistent storage in a browser).
I’ve actually found Linear to be quite slow? There was a week where it would also spin at 100% CPU if I left a tab with it open for a while :(
It appears to use a lot of memory too, at least in Firefox. I can't have more than a few Linear tabs open at a time.
They're doing some really cool stuff with Agents too.. automating software pm. it's a huge difference between this and their (unnamed) larger competitors.
I've never used linear, but just watching their example video [1] would worry me if I was a user and it's actually doing what they say it's doing.
He creates a task called "Create faster app launch", if we believe the article, it's processing that locally rather than going via the server, and then it's allocated an ID "BRO-5". That the ID is so low suggests it's just adding one to the previous issue ID, and so under heavy load, there are almost certainly going to be conflicts with other users creating tasks and getting identical IDs. Even if the system resolves this by changing one of those IDs, the system shouldn't be presenting the ID to the user until it is guaranteed unique. What if they've already pasted it into a document when the system notices the collision and renumbers it?
[1] https://media.performance.dev/posts/p_gAMR6Z7y49Fp/NZrXs70M_...
EDIT: WTH, there are some seriously bad karma people in this thread - just because I dared to have an opinion that the approach taken by this software might not be the best, my post was downvoted in less than a minute after posting! I'm sure whoever did that carefully considered my argument. If I'm wrong, explain why, don't just downvote my comment. If I'm not wrong, shame on you.
funny how things come back.
I loved meteorjs and its still my fav framework to this day.
Seeing optimistic ui and browser based databases being used this way makes me happy.
I would like to understand how they do the chunking and only updating part of the graph instead all of it.
I've always wondered how they made the UX so slick - thanks for sharing.
TL;DR: Background syncing instead of synchronous updates to the cloud.
This is basically a thick client, and comes with according trade-offs. It's interesting and there are some best practices, but I can't help but feeling that either the author is a huge fan or the post is an ad (or "sponsored").
> Literally the first lines of code that I wrote was the sync engine, which is very uncommon to what you usually do when you're a startup
He chose the path to a better product rather than the path to a quick buck. That is definitely odd for a silicon valley startup
I was expecting a post on how hardware accelerate linear algebra lol
Linear is the best web app I have ever seen, period. It is also the best bug-tracker I've ever used. I use it, and pay for it (gladly). It's worth it.
I am genuinely impressed with their engineering and design — I aspire to attain these levels, though I lack not only the skills, but also several zeroes in my bank account, I think. Still, it's worth looking at what they do and try to get there!
Big props and kudos to the Linear team. It's an impressive app.
Interestingly, as a professional developer I had the opposite first experience. It's a tracker. It looks like every other tracker. Everything is really small and "clean" which means they just hide everything, for that modern look.
I was thoroughly unimpressed. "This is it?" - "It's a tracker"
I feel comfortable saying this because in the weeks after, actually using it is the aha experience. Linear nailed the UX of working where people already work. Which again is really funny because the best part of Linear is how well it works outside of Linear.
(disclaimer: I actually now use their UI a lot. It's a helpful dashboard. But it suffers from every other hard-problem of information dense task-based dashboards.)