"If we want to order food in a country where we don't know the language at all, we're forced to go into the kitchen and use a see-and-point interface. With a little understanding of the language, we can point at menus to select our dinner from the dining room. But language allows us to discuss exactly what we would like to eat with the waiter or chef."
Ironically, Japanese menus almost universally have pictures of the food, and often (amazingly detailed) plastic models* of the dish in the window.
I frequently wish this was adopted by western restaurants, as being surprised by what actually arrives on my plate after I order is a regular occurrence.
It may partly a legal thing: Japanese law is that depictions of what you're going to get are pretty important promises that should be kept.
As opposed to, say, an artistic free-expression of a shared dream of a platonic product, or a bunch of metaphorical things which evoke the joy of having the product, etc.
I'm not sure how well it could be adopted and adapted for American law, but I wish someone'd try.
I got to use a real Magic Cap, one of the examples of alternative metaphors, in the article, a black and white view of a room with a desk full of old office oddities. It was the worst user interface that may have ever been designed, like an Alice In Wonderland nightmare. Click an envelope on a desk or a clock, and it starts some other metaphor like an image of a spreadsheet in a dialog, or something, which might appeal to some kind of “grand adventure” logic, but in today’s context…I’ll avoid ending with a negative comment.
I own a Sony Magic Link and you are 100% correct. The UI is the worst sort of point-n-click adventure game memes. Not only do you have to guess what visual elements actually do something you need to figure out what functionality they represent. The spacial metaphor is insipid because it takes a lot of taps to get from one "room" to another.
None of this is helped by how slow the Magic Link is. Supposedly the DataRover 840 was much faster but I've never owned one to tell for sure.
The UI of the Newton MessagePad (I own several) is far from perfect but makes much more sense than MagicCap. It also requires fewer taps to reach different functions.
Every once in a while I'll pull out my Magic Link but the insanity of the UI just inspires me to put it back in a box.
LLMs, for better or worse, bring some of that capability to users who aren't fluent in scripting. It allows language centric control, but a couple more layers of abstraction up from direct coding, especially with the rise of agents.
Though, I do find that breaking down instructions into concrete specific steps and validating the LLM output is its own skill that is not too dissimilar to the mindset needed for coding.
I think what it describes is about right too: computers programs should have a REPL and we should have agents that can input them for us if we don't want to get into the weeds and wish to automate tasks, in some ways anticipating the browser too.
> Most computer programmers gave up complete control some time ago when they stopped writing in machine language and let assemblers, compilers, and interpreters worry about all the little details.
Ah, there it is. The slippery slope that has stubbornly refused to be slippery for many decades now. Perhaps the author is completely misunderstanding these "metaphors".
What do you mean? With so many average users hopping onboard the LLM train to do what they could basically already do but with less effort (and less control), it seems like the slope's been slippery as predicted.
(And even setting AI aside, I think many people would agree that e.g. Windows 11 gives them less "control" than versions of Windows from decades ago, with the advantage of being harder to break in some ways. Same on the Mac side, and even in the GNU/Linux ecosystem in some ways.)
"If we want to order food in a country where we don't know the language at all, we're forced to go into the kitchen and use a see-and-point interface. With a little understanding of the language, we can point at menus to select our dinner from the dining room. But language allows us to discuss exactly what we would like to eat with the waiter or chef."
Ironically, Japanese menus almost universally have pictures of the food, and often (amazingly detailed) plastic models* of the dish in the window.
I frequently wish this was adopted by western restaurants, as being surprised by what actually arrives on my plate after I order is a regular occurrence.
I'm fully onboard with see-and-point.
* https://en.wikipedia.org/wiki/Food_model
> pictures of the food
It may partly a legal thing: Japanese law is that depictions of what you're going to get are pretty important promises that should be kept.
As opposed to, say, an artistic free-expression of a shared dream of a platonic product, or a bunch of metaphorical things which evoke the joy of having the product, etc.
I'm not sure how well it could be adopted and adapted for American law, but I wish someone'd try.
I got to use a real Magic Cap, one of the examples of alternative metaphors, in the article, a black and white view of a room with a desk full of old office oddities. It was the worst user interface that may have ever been designed, like an Alice In Wonderland nightmare. Click an envelope on a desk or a clock, and it starts some other metaphor like an image of a spreadsheet in a dialog, or something, which might appeal to some kind of “grand adventure” logic, but in today’s context…I’ll avoid ending with a negative comment.
I own a Sony Magic Link and you are 100% correct. The UI is the worst sort of point-n-click adventure game memes. Not only do you have to guess what visual elements actually do something you need to figure out what functionality they represent. The spacial metaphor is insipid because it takes a lot of taps to get from one "room" to another.
None of this is helped by how slow the Magic Link is. Supposedly the DataRover 840 was much faster but I've never owned one to tell for sure.
The UI of the Newton MessagePad (I own several) is far from perfect but makes much more sense than MagicCap. It also requires fewer taps to reach different functions.
Every once in a while I'll pull out my Magic Link but the insanity of the UI just inspires me to put it back in a box.
This was a thought experiment about UIs if the windows and mouse metaphor hadn’t taken off.
As developers we have the best of both worlds: direct visual manipulation, but also a language-centric control of richer objects in the terminal.
Being able to flip between these has always felt like a superpower.
LLMs, for better or worse, bring some of that capability to users who aren't fluent in scripting. It allows language centric control, but a couple more layers of abstraction up from direct coding, especially with the rise of agents.
Though, I do find that breaking down instructions into concrete specific steps and validating the LLM output is its own skill that is not too dissimilar to the mindset needed for coding.
[delayed]
The description of interacting with a computer through language seems oddly prescient of LLMs.
I was thinking the same thing.
I think what it describes is about right too: computers programs should have a REPL and we should have agents that can input them for us if we don't want to get into the weeds and wish to automate tasks, in some ways anticipating the browser too.
> Most computer programmers gave up complete control some time ago when they stopped writing in machine language and let assemblers, compilers, and interpreters worry about all the little details.
Ah, there it is. The slippery slope that has stubbornly refused to be slippery for many decades now. Perhaps the author is completely misunderstanding these "metaphors".
What do you mean? With so many average users hopping onboard the LLM train to do what they could basically already do but with less effort (and less control), it seems like the slope's been slippery as predicted.
(And even setting AI aside, I think many people would agree that e.g. Windows 11 gives them less "control" than versions of Windows from decades ago, with the advantage of being harder to break in some ways. Same on the Mac side, and even in the GNU/Linux ecosystem in some ways.)