Paper Computing (great name!) is something I've been thinking about a lot to help my kids benefit from tech without exposing them to the brain melting addiction of screens. I sacrificed a few crazy nights of sleep to try to build a Paper Computer Agent prototype for a recent Gemini hackathon (only to disappointingly have submission issues right before the actual deadline) which my kids loved and keep asking me to set up permanently for them.
It's essentially a poor man's hacked up DynamicLand - projector, camera, live agent. There are so many things you could do if you had a strong working baseline for this. My kids used it to create stories, learn how to draw various things, and watching safe videos they could hold in their hand.
There's something weirdly compelling and delightfully physical about holding a piece of paper that shows a live rocket launch, with the flames streaming down the page. It could also project targeted pieces of text, such as inline homework advice, or graphs next to data. It doesn't take long to imagine any other number of fun use cases, and it feels a lot more freeing and inspiring than keeping everything bound to a screen.
Receive email, render page with the email and a reply section and a unique ID, print it out physically
Human picks up all the sheets out of the printer, writes out replies with pen
Human puts the stack of answered email sheets in a multi-page scanner
Scanner physically scans them, agent transcribes them and matches them back to the incoming emails via the unique ID on each sheet, sends replies
You could adjust this flow for anything where human input is just one part of a larger sequence: just add print -> write -> scan into your flow where you'd normally have a human type. It's kind of a rebirth of faxing
I will say scanners are somewhat unergonomic, but if you had a high enough definition camera, you could photograph the document in its "natural environment". Granted, it's harder to get an evenly lit picture that way, but I think it's a nicer interface.
The idea of writing a draft on paper, or cutting out squares to prototype layouts on a table, sounds like a nightmare to me. But I never did like pen and paper much and have lived and breathed computers since I was young. My ideal method of writing is a full screen monospaced terminal
That said, I do much prefer reading on paper, or at least on e-ink, for many of the same reasons outlined in the post. Computers and phones are just too distracting, and too dynamic.
And I'd love some way to write down shopping lists or appointments, and have them available wherever, without having to pull out the phone. Our current method is a whiteboard + a photo whenever we need it, which doesn't quite cut it.
Just the other day, I noticed my thinking was so hijacked by distractions while building something (with AI help) that I started writing in a notebook to stay on track. The last time I'd written in the notebook was 3 years ago; in this case writing stuff down in it really helped to get me unstuck.
I'm excited to imagine workflows that could make computing a more physical activity. Thanks for writing and sharing this.
This was my gut reaction as well as an eInk enthusiast, but I think the author is looking for something quite different. As much as the rM is a calmer, slower-paced device by design, it's still a device with a screen that doesn't have the same physical affordances and spatial flexibility as pieces of paper.
> they have the problem that they make it difficult to just use your calendar, todo list, or map—or even just respond to a friend's message—without encountering something else along the way, like a social network, short-form video, Slack, the news, or some other notification.
I see this seemingly everywhere. People are looking for these extreme solutions to solve the problem of getting distracted by an app like Instagram or TikTok on their phone. Wouldn’t uninstalling the app, and going a step further, deleting the account, be the more pragmatic solution here? We control what is installed on our devices, what accounts we have, and which notifications we receive. If someone has enough agency to move to a pen and paper, surely they can uninstall some apps?
While I like the idea of having a magic paper notebook that would somehow interact with computer systems, that idea seems like mostly science fiction without having significant levels of technology all around you (cameras, projectors, etc) which would kind of defeat the purpose imo.
I watched the first video on Dynamic Land and I think I’d feel very uncomfortable in a room like that. Look the wrong way and catch a projector’s light in the eye, and once big tech gets into the game, who knows what happens with all the data from the cameras. I’ve grown rather paranoid.
A phone with just utilities installed, no social media, or going a step further to something like an e-ink tablet (something like Remarkable), seems like it would get most of the way there and actually work today. The biggest concern then becomes the web browser, but the big tech companies do most of the work for us by making sites insufferable to use while logged out and without an app.
Something might be able to get rigged up with RocketBook as well, for an actual pen on paper experience, but having to take a picture of the pages is kind of a pain. I have one and the novelty wore off very quickly; it has sat in a drawer for years now.
I’ve struggled with this idea a bit myself, as I sometimes romanticize the idea of using analog tools, but when they exist alone on an island, that seems to come with some considerable downsides in the modern world.
Apple Notes can be good for some of this too. Instead of using ChatGPT, Apple Notes can use the phone camera to do live OCR on text and add it into a note. I’ve used it a couple times and it’s pretty handy, when I remember it.
Yeah, and it's really worth checking out https://dynamicland.org/, because Bret Victor is actually doing this -- slash pointing the way to what such a world could look like. It just seems like now might be a good time for specific smaller parts of that vision to be carved off and developed further. I say that largely because of the advances in multimodal AI, which maybe haven't been fully applied yet in this area.
And a shout-out to https://folk.computer/ as well! They're not as far along in terms of feature parity, but they are open source, and exploring the space in other directions.
Emacs, and technologies built on it, such as org-mode, come somewhat close to ideas expressed here by having plain text in a buffer be the unifying data format. You can organize stuff by just moving snippets of text around.
I think it's difficult in practice to design data manipulation interfaces based on real-world objects because atoms are heavy and bits are not. Data is just much more malleable and transformable than real world objects, at least at the pre-Diamond Age tech level we're at. But maybe ML will help make this easier by allowing computers to track and scan the objects more easily.
Paper Computing (great name!) is something I've been thinking about a lot to help my kids benefit from tech without exposing them to the brain melting addiction of screens. I sacrificed a few crazy nights of sleep to try to build a Paper Computer Agent prototype for a recent Gemini hackathon (only to disappointingly have submission issues right before the actual deadline) which my kids loved and keep asking me to set up permanently for them.
It's essentially a poor man's hacked up DynamicLand - projector, camera, live agent. There are so many things you could do if you had a strong working baseline for this. My kids used it to create stories, learn how to draw various things, and watching safe videos they could hold in their hand.
There's something weirdly compelling and delightfully physical about holding a piece of paper that shows a live rocket launch, with the flames streaming down the page. It could also project targeted pieces of text, such as inline homework advice, or graphs next to data. It doesn't take long to imagine any other number of fun use cases, and it feels a lot more freeing and inspiring than keeping everything bound to a screen.
Github - https://github.com/Pugio/Orly (hacky minimal prototype that did the thing)
Video Pitch - https://youtu.be/-9l1x7GnmxU (filmed an hour before the deadline on an old phone with no sleep)
Receive email, render page with the email and a reply section and a unique ID, print it out physically
Human picks up all the sheets out of the printer, writes out replies with pen
Human puts the stack of answered email sheets in a multi-page scanner
Scanner physically scans them, agent transcribes them and matches them back to the incoming emails via the unique ID on each sheet, sends replies
You could adjust this flow for anything where human input is just one part of a larger sequence: just add print -> write -> scan into your flow where you'd normally have a human type. It's kind of a rebirth of faxing
I will say scanners are somewhat unergonomic, but if you had a high enough definition camera, you could photograph the document in its "natural environment". Granted, it's harder to get an evenly lit picture that way, but I think it's a nicer interface.
The idea of writing a draft on paper, or cutting out squares to prototype layouts on a table, sounds like a nightmare to me. But I never did like pen and paper much and have lived and breathed computers since I was young. My ideal method of writing is a full screen monospaced terminal
That said, I do much prefer reading on paper, or at least on e-ink, for many of the same reasons outlined in the post. Computers and phones are just too distracting, and too dynamic.
And I'd love some way to write down shopping lists or appointments, and have them available wherever, without having to pull out the phone. Our current method is a whiteboard + a photo whenever we need it, which doesn't quite cut it.
Reminds me of Paper Website from the Tiny Projects series, discussed back in 2021.
https://daily.tinyprojects.dev/paper_website
https://news.ycombinator.com/item?id=29550812
I love the idea.
Just the other day, I noticed my thinking was so hijacked by distractions while building something (with AI help) that I started writing in a notebook to stay on track. The last time I'd written in the notebook was 3 years ago; in this case writing stuff down in it really helped to get me unstuck.
I'm excited to imagine workflows that could make computing a more physical activity. Thanks for writing and sharing this.
How about hacking a remarkable e-ink tablet as an easy prototype? The remarkable is basically a "better paper" already.
This was my gut reaction as well as an eInk enthusiast, but I think the author is looking for something quite different. As much as the rM is a calmer, slower-paced device by design, it's still a device with a screen that doesn't have the same physical affordances and spatial flexibility as pieces of paper.
> they have the problem that they make it difficult to just use your calendar, todo list, or map—or even just respond to a friend's message—without encountering something else along the way, like a social network, short-form video, Slack, the news, or some other notification.
I see this seemingly everywhere. People are looking for these extreme solutions to solve the problem of getting distracted by an app like Instagram or TikTok on their phone. Wouldn’t uninstalling the app, and going a step further, deleting the account, be the more pragmatic solution here? We control what is installed on our devices, what accounts we have, and which notifications we receive. If someone has enough agency to move to a pen and paper, surely they can uninstall some apps?
While I like the idea of having a magic paper notebook that would somehow interact with computer systems, that idea seems like mostly science fiction without having significant levels of technology all around you (cameras, projectors, etc) which would kind of defeat the purpose imo.
I watched the first video on Dynamic Land and I think I’d feel very uncomfortable in a room like that. Look the wrong way and catch a projector’s light in the eye, and once big tech gets into the game, who knows what happens with all the data from the cameras. I’ve grown rather paranoid.
A phone with just utilities installed, no social media, or going a step further to something like an e-ink tablet (something like Remarkable), seems like it would get most of the way there and actually work today. The biggest concern then becomes the web browser, but the big tech companies do most of the work for us by making sites insufferable to use while logged out and without an app.
Something might be able to get rigged up with RocketBook as well, for an actual pen on paper experience, but having to take a picture of the pages is kind of a pain. I have one and the novelty wore off very quickly; it has sat in a drawer for years now.
I’ve struggled with this idea a bit myself, as I sometimes romanticize the idea of using analog tools, but when they exist alone on an island, that seems to come with some considerable downsides in the modern world.
Apple Notes can be good for some of this too. Instead of using ChatGPT, Apple Notes can use the phone camera to do live OCR on text and add it into a note. I’ve used it a couple times and it’s pretty handy, when I remember it.
If I understand this correctly, you're talking about using paper as a computing interface? That's such a neat idea!
Yeah, and it's really worth checking out https://dynamicland.org/, because Bret Victor is actually doing this -- slash pointing the way to what such a world could look like. It just seems like now might be a good time for specific smaller parts of that vision to be carved off and developed further. I say that largely because of the advances in multimodal AI, which maybe haven't been fully applied yet in this area.
And a shout-out to https://folk.computer/ as well! They're not as far along in terms of feature parity, but they are open source, and exploring the space in other directions.
Thought this was gonna be about CARDIAC, lol.
Emacs, and technologies built on it, such as org-mode, come somewhat close to ideas expressed here by having plain text in a buffer be the unifying data format. You can organize stuff by just moving snippets of text around.
I think it's difficult in practice to design data manipulation interfaces based on real-world objects because atoms are heavy and bits are not. Data is just much more malleable and transformable than real world objects, at least at the pre-Diamond Age tech level we're at. But maybe ML will help make this easier by allowing computers to track and scan the objects more easily.