Native controls have dozens to hundreds of subtle UI interactions. Consider a text input element. On macos there are text shortcuts - cmd+A to select all. cmd+left / right to go to the next or previous word. Page up / page down. Home / end. Spell check. A right click menu with about 10 more options, including OS based text services. The same control on ios or android will pull up a keyboard, configured with the user's smart keyboard & language. The UI changes in small subtle ways with each major OS version.
You simply can't reimplement this stuff in the browser. Browser APIs don't let you intercept a lot of the keyboard shortcuts you need. You don't have access to macos text services, or the native iOS keyboard UI.
Some people try to reimplement this stuff in the browser, but it always feels horrible to use. Nearly native, but laggy and nothing looks quite right. You can't select text on the page properly. Fonts render slightly wrong. I have muscle memory for keyboard shortcuts that don't work. I hate it.
The dev tools in the browser become much more useless when you draw everything in a canvas. It's gonna be sad when everyone starts using frameworks that draw on canvases. Arguably more sad than when Webassembly came. One could probably write new dev tools for those frameworks though.
I also imagine this will be a big setback for web accessibility.
I’m always curious how Google, Microsoft and Co. build their web apps - given that they need to work well on any computer, from a top-shelf speed machine to a potato with wires.
In my experience, Microsoft webapps (which now includes many of their desktop apps) are rather sluggish. Even the Copilot Chat website. How hard could it be to make a chatbox and a session transcript not be sluggish?
I think some of the Youtube thumbnail stuff is lazy-loaded, so depending on the speed of your browser and connection it can just hang there doing nothing for a while.
Microsoft web apps are sluggish on my i7 with 64gb of RAM. But it's not performance issues, I think. It's networking, they have some crazy interconnected backend spread across about 20 systems and everything you do is going to need to wait for all of those to talk with each other. Switching HTML to canvas isn't gonna fix it.
The text entry of the Google AI is a catastrophe. Not scalable, sometimes text lines are half hidden, the cursor disappears on empty lines, it eats line breaks after sending the text. I reported it via mail, Google never answered.
The scrollbar is blocked on the bottom by the chat window.
Keyboard navigation page up, page down, arrow up, arrow down only work if you are inside inside the answer window otherwise you get a totally different scroll behavior.
On Edge you can’t change anything of that per extension because MS block any change
Not a word in here about accessibility, of course. If you’re blind you’re blind to the canvas as well without a lot of work, it’s much more straightforward to make the DOM accessible
> For most web apps, the DOM remains the better choice. It gives you accessibility, responsive layouts, text selection, input handling, and countless other features for free
I was thinking you could probably get a fair bit of accessibility by using <div>s for your text rendering- or by maintaining a separate, hidden outline of DOM elements mirroring what’s on screen.
I call this the bluebox problem. Back when I worked for SAP there used to be this control called the bluebox control that embedded a browser (shdocvw) into a C++/MFC app. Almost every single annoying UI bug came from that control. I see the same now in the other direction. Every 5 years we come up with the bright idea to use browser as a distribution engine for a UI surface that is rendered by a box. This has historically never worked. I just wanted to recount applets, activex controls, flash, flex (also flash), silverlight, flutter ... and now this. The issue is developer tooling, screen readers, browser extensions and all the other small things that are difficult to account for.
And of course, if you're going to go with Canvas, might as well go all the way and write the app in a cross-platform way in one of the many languages that supports Web Assembly and then compile to Web (which then gets embedded into Canvas). That way you could target more hardware and give users more options. There is some overhead but in my opinion browsers should be a backup UI for when you're away from a work station.
> So don’t choose Canvas simply because it sounds fast. Choose it when your interface no longer behaves like a document and starts behaving more like a scene.
Unity can literally deploy scenes to web canvases with its webgl target.
If you are going to use canvas, why not use the most extreme form of it with the most complete tooling available? You can put the universal render pipeline in anyone's browser in 5 minutes. These deployed web assets are surprisingly small if we are responsible with art.
Accessibility is even an option because we are leveraging a gigantic corporate engineering team and actually have time to try and solve this rather than reinventing basic layout and rendering concerns.
> Google Docs isn’t pure-canvas: they only switched their document area to use it for layout. Text rendering is still browser (necessary to keep performance even close), and all the rest of the UI is still DOM. Having thought extensively about it, I cannot come up with any advantage to the approach they’ve taken—neither the performance¹ nor the consistency² angles make any sense, and the product is actively worse because of it³. I’ve written more about it on HN at times, skim https://hn.algolia.com/?query=chrismorgan%20google%20docs&ty... for more. Seriously, having thought about it very carefully and reviewed the matter several times over the years, I honestly believe that they lied in their justifications.
I haven’t ever examined Google Sheets closely, but it looks to use a similar approach, but worse in scrolling (it’s very obnoxious compared with native).
Excel I can’t comment on.
Docs should definitely have stayed HTML.
Sheets and Excel should probably be HTML, or possibly SVG.
People who work on web browsers have ranted that HTML/CSS isn't a very natural interface into the engine, so it's inefficient and has nasty edge cases. I know nothing about browser engines but do find CSS awkward as a user when I'm not making a plaintext website, so have been relying heavily on React for side projects.
Been thinking about doing a toy project where I try to build my own alternative to HTML/CSS that runs on top of Canvas, drawing inspiration from how Wayland is actually less abstracted than X11.
I've been tempted to do something similar for browser canvas, perhaps with some slightly different choices than he made. His approach uses a very elegant way of looking at things though.
If you've decided to make your next webapp's interface in Canvas, you are jeopardizing what makes the web flourish in the first place: openness. I personally agree with DHH that View Source is one of the browser's most liberating feature. Others in this thread have also pointed issues in accessibility.
It's a shame that this is what the web is trending towards with more and more enterprise interest in the internet.
Canvas is a last resort if you absolutely cannot build the thing any other way.
When you use canvas you're giving up on not just accessibility but also optimizations that the browser does for you transparently when you use the native render tree. There's no guarantee that your app will be faster in canvas, it could easily be slower. Do you really think you can beat the browser's C++ by writing code in a language that doesn't even have a native integer type? Perhaps you can, but your base assumption should be that you can't.
Please don't do this.
Native controls have dozens to hundreds of subtle UI interactions. Consider a text input element. On macos there are text shortcuts - cmd+A to select all. cmd+left / right to go to the next or previous word. Page up / page down. Home / end. Spell check. A right click menu with about 10 more options, including OS based text services. The same control on ios or android will pull up a keyboard, configured with the user's smart keyboard & language. The UI changes in small subtle ways with each major OS version.
You simply can't reimplement this stuff in the browser. Browser APIs don't let you intercept a lot of the keyboard shortcuts you need. You don't have access to macos text services, or the native iOS keyboard UI.
Some people try to reimplement this stuff in the browser, but it always feels horrible to use. Nearly native, but laggy and nothing looks quite right. You can't select text on the page properly. Fonts render slightly wrong. I have muscle memory for keyboard shortcuts that don't work. I hate it.
The dev tools in the browser become much more useless when you draw everything in a canvas. It's gonna be sad when everyone starts using frameworks that draw on canvases. Arguably more sad than when Webassembly came. One could probably write new dev tools for those frameworks though.
I also imagine this will be a big setback for web accessibility.
I’m always curious how Google, Microsoft and Co. build their web apps - given that they need to work well on any computer, from a top-shelf speed machine to a potato with wires.
In my experience, Microsoft webapps (which now includes many of their desktop apps) are rather sluggish. Even the Copilot Chat website. How hard could it be to make a chatbox and a session transcript not be sluggish?
YouTube too. And I'm not even talking about the video player. Displaying a 4x4 grid of thumbnails seems to be some kind of herculean task.
I think some of the Youtube thumbnail stuff is lazy-loaded, so depending on the speed of your browser and connection it can just hang there doing nothing for a while.
Microsoft web apps are sluggish on my i7 with 64gb of RAM. But it's not performance issues, I think. It's networking, they have some crazy interconnected backend spread across about 20 systems and everything you do is going to need to wait for all of those to talk with each other. Switching HTML to canvas isn't gonna fix it.
Most Microsoft web apps are sluggish and horribly designed.
I was using Entra and Azure the other day and boy are they slow and clunky.
The text entry of the Google AI is a catastrophe. Not scalable, sometimes text lines are half hidden, the cursor disappears on empty lines, it eats line breaks after sending the text. I reported it via mail, Google never answered.
Chatgpt makes teams feel like an optimized app. The azure website makes everything else feel like it's been optimized.
Especially in the era of AI, performance has been thrown out of the window
YouTube is a half working disaster.
The Copilot Char got even worse.
The scrollbar is blocked on the bottom by the chat window. Keyboard navigation page up, page down, arrow up, arrow down only work if you are inside inside the answer window otherwise you get a totally different scroll behavior.
On Edge you can’t change anything of that per extension because MS block any change
All of their webapps are garbage.
Not a word in here about accessibility, of course. If you’re blind you’re blind to the canvas as well without a lot of work, it’s much more straightforward to make the DOM accessible
It says:
> For most web apps, the DOM remains the better choice. It gives you accessibility, responsive layouts, text selection, input handling, and countless other features for free
Ah, so literally a word.
I was thinking you could probably get a fair bit of accessibility by using <div>s for your text rendering- or by maintaining a separate, hidden outline of DOM elements mirroring what’s on screen.
Yep. Accessibility isn't just a nice to have. Its a legal requirement for a lot of websites under the ADA (Americans with Disabilities Act).
I call this the bluebox problem. Back when I worked for SAP there used to be this control called the bluebox control that embedded a browser (shdocvw) into a C++/MFC app. Almost every single annoying UI bug came from that control. I see the same now in the other direction. Every 5 years we come up with the bright idea to use browser as a distribution engine for a UI surface that is rendered by a box. This has historically never worked. I just wanted to recount applets, activex controls, flash, flex (also flash), silverlight, flutter ... and now this. The issue is developer tooling, screen readers, browser extensions and all the other small things that are difficult to account for.
I’m pretty sure Google Sheets transitioned to this type of thing and it has been successful
And of course, if you're going to go with Canvas, might as well go all the way and write the app in a cross-platform way in one of the many languages that supports Web Assembly and then compile to Web (which then gets embedded into Canvas). That way you could target more hardware and give users more options. There is some overhead but in my opinion browsers should be a backup UI for when you're away from a work station.
> So don’t choose Canvas simply because it sounds fast. Choose it when your interface no longer behaves like a document and starts behaving more like a scene.
Unity can literally deploy scenes to web canvases with its webgl target.
If you are going to use canvas, why not use the most extreme form of it with the most complete tooling available? You can put the universal render pipeline in anyone's browser in 5 minutes. These deployed web assets are surprisingly small if we are responsible with art.
Accessibility is even an option because we are leveraging a gigantic corporate engineering team and actually have time to try and solve this rather than reinventing basic layout and rendering concerns.
https://github.com/mikrima/UnityAccessibilityPlugin
> The document in Google Docs is a Canvas.
This is only mostly true, and no justification at that. I’ll just quote myself from a couple of years ago <https://news.ycombinator.com/item?id=42253177>:
> Google Docs isn’t pure-canvas: they only switched their document area to use it for layout. Text rendering is still browser (necessary to keep performance even close), and all the rest of the UI is still DOM. Having thought extensively about it, I cannot come up with any advantage to the approach they’ve taken—neither the performance¹ nor the consistency² angles make any sense, and the product is actively worse because of it³. I’ve written more about it on HN at times, skim https://hn.algolia.com/?query=chrismorgan%20google%20docs&ty... for more. Seriously, having thought about it very carefully and reviewed the matter several times over the years, I honestly believe that they lied in their justifications.
I haven’t ever examined Google Sheets closely, but it looks to use a similar approach, but worse in scrolling (it’s very obnoxious compared with native).
Excel I can’t comment on.
Docs should definitely have stayed HTML.
Sheets and Excel should probably be HTML, or possibly SVG.
(I’ll fill out more later, got to go.)
I block all canvas requests. When I can no longer do so, I will just try to use the web less.
People who work on web browsers have ranted that HTML/CSS isn't a very natural interface into the engine, so it's inefficient and has nasty edge cases. I know nothing about browser engines but do find CSS awkward as a user when I'm not making a plaintext website, so have been relying heavily on React for side projects.
Been thinking about doing a toy project where I try to build my own alternative to HTML/CSS that runs on top of Canvas, drawing inspiration from how Wayland is actually less abstracted than X11.
You might find this video useful or interesting: https://youtu.be/by9lQvpvMIc
I've been tempted to do something similar for browser canvas, perhaps with some slightly different choices than he made. His approach uses a very elegant way of looking at things though.
Canvas will be increasingly popular in the arms race against ad blockers.
If you've decided to make your next webapp's interface in Canvas, you are jeopardizing what makes the web flourish in the first place: openness. I personally agree with DHH that View Source is one of the browser's most liberating feature. Others in this thread have also pointed issues in accessibility.
It's a shame that this is what the web is trending towards with more and more enterprise interest in the internet.
Compose Multiplatform is an interesting framework that can render to a web canvas, including full accessibility support.
Canvas is a last resort if you absolutely cannot build the thing any other way.
When you use canvas you're giving up on not just accessibility but also optimizations that the browser does for you transparently when you use the native render tree. There's no guarantee that your app will be faster in canvas, it could easily be slower. Do you really think you can beat the browser's C++ by writing code in a language that doesn't even have a native integer type? Perhaps you can, but your base assumption should be that you can't.
You’re not “giving up” on accessibility. You’re just not getting it “for free” — though it was never free to begin with.
When you’re dealing with things like spreadsheet viewers, disengaging from the DOM for sighted users can actually make accessibility simpler.
> disengaging from the DOM for sighted users can actually make accessibility simpler.
Do you have any examples? In my experience, a lot of software written like this just doesn't work with screen readers at all.
Can WASM write to Canvas though?
You need some JS in the middle but yes, you write your image to a buffer and share it with JS, then in the JS you put it in the canvas.