I feel like this is a feature which improves the perceived confidence of the LLM but doesn't do much for correctness of other outputs, i.e. an exacerbation of the "confidently incorrect" criticism.
It's a mismatch with our intuition about how much effort things take.
If there's humans involved, "I took this data and made a really fancy interactive chart" means that you put a lot more work into it, and you can probably somewhat assume that this means some more effort was also put into the accuracy of the data.
But with the LLM it's not really very much more work to get the fancy chart. So the thing that was a signifier of effort is now misleading us into trusting data that got no extra effort.
(Humans have been exploiting this tendency to trust fancy graphics forever, of course.)
It's a usability / quality of life feature to me. Nothing to do with increasing perceived confidence. I guess it depends on how much you already (dis)trust LLMs.
I'm finding more and more often the limiting factor isn't the LLM, it's my intuition. This goes a way towards helping with that.
A similar thing happened when Google started really pushing generating flowcharts as a use-case with Nano Banana. A slick presentation can distract people from the only thing that really matters - the accuracy of the underlying data.
As a slightly different tack, I’ve been using Copilot to generate flowcharts from some of the fiendishly complex (and badly written) standard operating procedures we have at work.
People find them quite easy to check - easier than the raw document. My angle with teams is use these to check your processes. If the flow is wrong it’s either because the LLM has screwed up, or because the policy is wrong/badly written. It’s usually the latter. It’s a good way to fix SOPs
This was my first thought as well, all this does is further remove the user from seeing the chat output and instead makes it appear as if the information is concretely reliable.
I mean is it really that shocking that you can have an LLM generate structured data and shove that into a visualizer? The concern is if is reliable, which we know it isnt.
Its' a reasonable concern. Often it can be mitigated by prompting in a manner that invokes research and verification instead of defaulting to a corpus.
When I ask chatgpt to create a mermaid diagram for me it regularly will add new lines to certain labels that will break the parse. If you then feed the parse error back to it the second version is always correct And it seems to exactly know the problem. There are some other examples where it will almost always get it wrong the first time but right if nudged to correct itself. I wonder what the underlying cause is
It responds with the statistically most probable text based on its training data, which happens to be different with the errors vs without. I suspect high-fidelity diagramming requires a different attention architecture from the common ones used in sentence-optimized models.
Observed from 5.2, on chatgpt.com. earlier versions did worse.. as in, they might take a few prompts to generate a parseable syntax. Newer versions just usually deliver one unparseable version then get it right second try. Likely I could prompt engineer to one shot but I think I would always need the specific warning about newlines.
Mermaid is really bad about cutting off text after spaces, so you have to insert <br>s everywhere. I’m guessing this is getting rendered instead of escaped by your interface. Or just lost in translation at the tokenizer.
Today I asked Claude to create me a squidward looking out the window meme and it started generating HTML & CSS to draw squidward in a style best described as "4 year old preschooler". Not quite it yet.
I tried the periodic table in their examples using sonnet 4.6 on the $20/mo plan. After a few minutes Claude told me it reached the max message length and bailed. I pressed continue and eventually it generated the table, but it wasn't inline, it was a jsx artifact, and I've now hit my daily usage limit.
I’m intermittently getting artifacts vs the new visuals api, depending on which version of the Claude app I use. iOS/iPadOS apps are not yet supporting the visualization API, and I don’t see an app-store update yet.
I find it absolutely mindblowing to witness the rate at which Anthropic can ship new features. Only a year ago I couldn't wait to see some sort of Github integration and then it appeared only a week later. Seriously impressive stuff.
I think to the extent they are making a speed v quality tradeoff, I think they are making the right call. 10x speed over quality any day for me. Reminds me of:
"If brute force doesn't work, you aren't using enough of it." - Isaac Arthur
Everyone is
making this tradeoff now. Surely nothing bad could come from it.
In the meantime I can’t even continue a Claude Code session I started on desktop on my phone. What’s the point of shipping a billion features of they are all half baked?
They also pay... insane salaries, like double industry average. That coupled with an IPO on the horizon means they probably have their pick of engineers.
Meanwhile, you still can't Sign in with Apple on the website.
But you can Sign in with Google.
If you signed up with your Apple on the iOS Claude app, to access your account on the computer, you have to open the passwords app and copy your random email address and paste it into the Claude website login.
Also if you try to copy-paste a prompt from Notes etc into the Claude chat, it gets added as an attachment, so you can't edit the prompt. If you do the four-finger shortcut to paste it as text, it mangles newlines etc.
Why are they so dumb about such basic UX for so long?
> you still can't Sign in with Apple on the website.
Apple forces developers to offer Sign in with Apple on iOS devices if any other sign in service is used. Apple can't force them to do it on non-Apple platforms.
> If you signed up with your Apple on the iOS Claude app, to access your account on the computer, you have to open the passwords app and copy your random email address and paste it into the Claude website login.
Isn't this basically Apple's fault? When you signed up, Apple provided a fake email address in leu of your real one. This is great for privacy but means the service has the wrong email.
I'm sure they didn't want to provide an Apple sign in option at all, but it's required by App Store rules.
They could also just implement sign in with apple on their website, they have the ability to sign in with google so not supporting Apple is still a weird choice they are making.
Apple should not have had to require developers to have options other than Google for authentication, but clearly some companies have to be dragged kicking and screaming.
So clearly they support it, and there is no reason it should not work on the web also.
There are a lot of websites that only support third party login, so that is not always an option.
They don't have to bend for another, but they made a choice to put an app on iOS. They added support for apple signin, and then for some reason did not put it on their website.
You can criticize Apple for requiring that all you want, but they clearly have support for it and are choosing to not put it on their website which is causing a worse user experience.
IF apple did not support website loggin than sure, but they do. So the ability to fix this is on Anthropic (and many other websites).
If you are already going to support third party login you should not limit it to only Google accounts and there is no reason to support Apple on iOS and not the web.
Also for the record, Apple only requires sign in with apple if you already support third party authentication. So if you are already going to support that, giving the user more choice (and making it so we are all a bit less dependent on google) is a good thing.
No criticism from me towards apple or Anthropic. Both parties made their choice. Apple was late to the identity business and the other ships had already sailed.
Third party logins are an extension and a massive risk to any website that doesn't include email hosting.
We have see identity providers dissapear, and people may change their mind.
Easiest way is to register you rown domain and use it with an identity provider of your choice and be able to move it anywhere.
Otherwise we are a faceless citizen of a corporation that can handle access to our identity and everything attached to it without recourse or access to anyone.
Not really. It's the user's fault. Apple provides an option to hide your email, it's not required. It's an option that shows up when you're prompted to create an account.
My original thinking was that Apple makes it too easy for a general audience to hide their email without considering the implications (the service won't know your email). But of course there's a tension here, since you also want the option to be easy and accessible.
The party I do not consider at fault in this case is Anthropic.
> I'm sure they didn't want to provide an Apple sign in option at all
But they wanted to provide a Google Sign In? wth?
> This is great for privacy but means the service has the wrong email.
So harm the users to benefit the service? wtf?
I don't want to give my real email or anything to random services, specially not one like Claude where they don't even let you remove your payment info.
> I don't want to give my real email or anything to random services, specially not one like Claude where they don't even let you remove your payment info.
The original complaint was:
>> If you signed up with your Apple on the iOS Claude app, to access your account on the computer, you have to open the passwords app and copy your random email address and paste it into the Claude website login.
Either you use your original email or you use a per-service email. Apple helps you do the latter, but this does come with UX tradeoffs.
Using a per-service email, then complaining that the service does not have your real email, strikes me as misguided.
Add to the list backtick handling. If you start a backtick block on the claude web chat, you cannot leave it with the keyboard. You are now stuck between the backticks. It is as if they wanted to reproduce Slack misery.
They clearly vibe code a lot (most? all?) of their stuff, and it shows. Elementary features are broken regularly and while I appreciate them trying new features, I'd appreciate it more if existing ones were reliable and promptly fixed if broken.
Trouble is, vibe coding refinements and bug fixes works well but probably isn't a good track to promotions at Anthropic (or virtually any other company.)
I asked it to do some portfolio analysis for me and it created BEAUTIFUL, tabbed, interactive charts UNPROMPTED. This is kind of magical. The charts were not just beautiful, but actually super useful in understanding the data faster. I honestly could not have produced those in a week if you asked me to.
Wow, I asked it to build me a simple diagram explaining agile development and it did an amazing job. Wow it felt magical to watch that diagram slowly animating to life.
Like a much prettier version of Mermaid.
Kudos, Anthropic. Geez, this is so nice.
Now I'm going to ask it to draw a diagram of a pelican riding a bicycle, why not?
The artifact output model is more useful than it looks at first. We use Claude in a multi-agent pipeline and discovered that structured artifact outputs reduce parse errors significantly compared to freeform text responses -- the model seems to reason differently when it knows the output will be rendered. Curious whether Anthropic sees similar quality improvements in tool-use tasks when the output has a concrete format constraint.
This is pretty neat and I am experimenting with it now, but hasn't ChatGPT had capability to create graphs and interact with data for a while? "ChatGPT advanced data analysis" for example. I'm asking in good faith as maybe some of you have been using that and can compare the two and give an informed opinion.
I usually use a lot of other tools for data analysis or write code with Claude code or another LLM to do data analysis and visualization.
> but hasn't ChatGPT had capability to create graphs and interact with data for a while?
It's pretty bad (for me). I have to use extremely prescriptive language to tell ChatGPT what to create. Even down to the colours in the chart, because otherwise it puts black font on black background (for example). Then I have to specifically tell it to put it in a canvas, and make it interactive, and make it executable in the canvas. Then if I'm lucky I have to hit a "preview" button in the top right and hope it works (it doesn't). I could write several paragraphs telling it to do something like what Claude just demo'd and it wouldn't come close. I'm trying Claude now for financial insights and it's effortless with beautiful UX.
For posterity, Gemini is pretty good with these interactive canvases. Not nearly as good, but FAR better than ChatGPT.
It was inevitable until the point all apps will disappear and AI will be the entry point for all work. You can see how anything required appear based on a single request. After which world models and other forms of interaction that are more dynamic will make sense and we'll need something that's not a screen.
Its a large leap from “we made a config driven diagram tool and trained an llm on that config” to “all apps will disappear”. If you’re predicting such grand claims please be more precise than “AI” which is a term we cant define.
Yeah an app doesn't "disappear" because you put an AI interface in front of it and then use a bunch of old school programming to parse LLM output and feed that into your old app. 99% of the work is still building the old app.
Interesting. So if I'm reading this correctly, this is distinctly different than the artifacts that Claude creates? If that's the case, why create it inline as opposed to an artifact? Any time I get a visual, I tend to find them so useful that I _want_ them to be an artifact that I can export and share.
Unable to reproduce the recipe image in the iOS app. It first gave a normal text answer. Then when referencing this blog post it produced a wonky HTML artifact.
Reliability has been the real bottleneck for multi-agent setups in production. The hard part isnt getting one agent to do something clever once - its making repeated runs observable and bounded when tools fail halfway through. Idempotency checks, explicit handoff state, and human review gates have mattered more for us than adding another model or another agent role.
I feel like this is a feature which improves the perceived confidence of the LLM but doesn't do much for correctness of other outputs, i.e. an exacerbation of the "confidently incorrect" criticism.
I suspect chain of thought while building the chart will improve the overall correctness of the answer
It's a mismatch with our intuition about how much effort things take.
If there's humans involved, "I took this data and made a really fancy interactive chart" means that you put a lot more work into it, and you can probably somewhat assume that this means some more effort was also put into the accuracy of the data.
But with the LLM it's not really very much more work to get the fancy chart. So the thing that was a signifier of effort is now misleading us into trusting data that got no extra effort.
(Humans have been exploiting this tendency to trust fancy graphics forever, of course.)
Already happened. :)
https://www.reddit.com/r/dataisugly/comments/1mk5wdb/this_ch...
It's a usability / quality of life feature to me. Nothing to do with increasing perceived confidence. I guess it depends on how much you already (dis)trust LLMs.
I'm finding more and more often the limiting factor isn't the LLM, it's my intuition. This goes a way towards helping with that.
A recent LinkedIn post that I came across as an example of people trusting (or learning to trust) AI too much while not realizing that it can make up numbers too: https://www.linkedin.com/posts/mariamartin1728_claude-wrote-...
P.S. Credit to the poster, she posted a correction note when someone caught the issue: https://www.linkedin.com/posts/mariamartin1728_correction-on...
A similar thing happened when Google started really pushing generating flowcharts as a use-case with Nano Banana. A slick presentation can distract people from the only thing that really matters - the accuracy of the underlying data.
As a slightly different tack, I’ve been using Copilot to generate flowcharts from some of the fiendishly complex (and badly written) standard operating procedures we have at work.
People find them quite easy to check - easier than the raw document. My angle with teams is use these to check your processes. If the flow is wrong it’s either because the LLM has screwed up, or because the policy is wrong/badly written. It’s usually the latter. It’s a good way to fix SOPs
This was my first thought as well, all this does is further remove the user from seeing the chat output and instead makes it appear as if the information is concretely reliable.
I mean is it really that shocking that you can have an LLM generate structured data and shove that into a visualizer? The concern is if is reliable, which we know it isnt.
The further they can get people from the reality of `This just spits out whatever it thinks the next token will be` the more they can push the agenda.
Its' a reasonable concern. Often it can be mitigated by prompting in a manner that invokes research and verification instead of defaulting to a corpus.
Passive questions generate passive responses.
I agree. Maybe next they'll add emotionally evocative music, with swelling orchestral bits when you reach the exciting climate of the slop.
When I ask chatgpt to create a mermaid diagram for me it regularly will add new lines to certain labels that will break the parse. If you then feed the parse error back to it the second version is always correct And it seems to exactly know the problem. There are some other examples where it will almost always get it wrong the first time but right if nudged to correct itself. I wonder what the underlying cause is
> I wonder what the underlying cause is
It responds with the statistically most probable text based on its training data, which happens to be different with the errors vs without. I suspect high-fidelity diagramming requires a different attention architecture from the common ones used in sentence-optimized models.
"Prompt Repetition Improves Non-Reasoning LLMs " - https://arxiv.org/pdf/2512.14982
What instance of ChatGPT are you doing that with? (Reasoning?)
Observed from 5.2, on chatgpt.com. earlier versions did worse.. as in, they might take a few prompts to generate a parseable syntax. Newer versions just usually deliver one unparseable version then get it right second try. Likely I could prompt engineer to one shot but I think I would always need the specific warning about newlines.
Mermaid is really bad about cutting off text after spaces, so you have to insert <br>s everywhere. I’m guessing this is getting rendered instead of escaped by your interface. Or just lost in translation at the tokenizer.
Today I asked Claude to create me a squidward looking out the window meme and it started generating HTML & CSS to draw squidward in a style best described as "4 year old preschooler". Not quite it yet.
I tried the periodic table in their examples using sonnet 4.6 on the $20/mo plan. After a few minutes Claude told me it reached the max message length and bailed. I pressed continue and eventually it generated the table, but it wasn't inline, it was a jsx artifact, and I've now hit my daily usage limit.
I’m intermittently getting artifacts vs the new visuals api, depending on which version of the Claude app I use. iOS/iPadOS apps are not yet supporting the visualization API, and I don’t see an app-store update yet.
I find it absolutely mindblowing to witness the rate at which Anthropic can ship new features. Only a year ago I couldn't wait to see some sort of Github integration and then it appeared only a week later. Seriously impressive stuff.
Have you actually used their products? They are janky, full of bugs and barely work half the time.
They write 100% of their code with Claude. Some of their engineers apparently burn over 100k worth of tokens per month.
It’s not surprising they ship fast at all when the product is actually falling apart at the seams and they just vibe code everything.
I think to the extent they are making a speed v quality tradeoff, I think they are making the right call. 10x speed over quality any day for me. Reminds me of:
"If brute force doesn't work, you aren't using enough of it." - Isaac Arthur
Everyone is making this tradeoff now. Surely nothing bad could come from it.
In the meantime I can’t even continue a Claude Code session I started on desktop on my phone. What’s the point of shipping a billion features of they are all half baked?
They also pay... insane salaries, like double industry average. That coupled with an IPO on the horizon means they probably have their pick of engineers.
You have 23 year olds earning $2mn/year, at least it isn't in HFT though!
Their interviews are actually very much focused on how fast you can code something that works.
Meanwhile, you still can't Sign in with Apple on the website.
But you can Sign in with Google.
If you signed up with your Apple on the iOS Claude app, to access your account on the computer, you have to open the passwords app and copy your random email address and paste it into the Claude website login.
Also if you try to copy-paste a prompt from Notes etc into the Claude chat, it gets added as an attachment, so you can't edit the prompt. If you do the four-finger shortcut to paste it as text, it mangles newlines etc.
Why are they so dumb about such basic UX for so long?
> you still can't Sign in with Apple on the website.
Apple forces developers to offer Sign in with Apple on iOS devices if any other sign in service is used. Apple can't force them to do it on non-Apple platforms.
> If you signed up with your Apple on the iOS Claude app, to access your account on the computer, you have to open the passwords app and copy your random email address and paste it into the Claude website login.
Isn't this basically Apple's fault? When you signed up, Apple provided a fake email address in leu of your real one. This is great for privacy but means the service has the wrong email.
I'm sure they didn't want to provide an Apple sign in option at all, but it's required by App Store rules.
They could also just implement sign in with apple on their website, they have the ability to sign in with google so not supporting Apple is still a weird choice they are making.
Apple should not have had to require developers to have options other than Google for authentication, but clearly some companies have to be dragged kicking and screaming.
So clearly they support it, and there is no reason it should not work on the web also.
A vendor doesn't have to bend for another.
Always best to sign in with your own email address.
> Always best to sign in with your own email address.
Using a randomly generated email per service is a huge improvement over always using the same email.
There are a lot of websites that only support third party login, so that is not always an option.
They don't have to bend for another, but they made a choice to put an app on iOS. They added support for apple signin, and then for some reason did not put it on their website.
You can criticize Apple for requiring that all you want, but they clearly have support for it and are choosing to not put it on their website which is causing a worse user experience.
IF apple did not support website loggin than sure, but they do. So the ability to fix this is on Anthropic (and many other websites).
If you are already going to support third party login you should not limit it to only Google accounts and there is no reason to support Apple on iOS and not the web.
Also for the record, Apple only requires sign in with apple if you already support third party authentication. So if you are already going to support that, giving the user more choice (and making it so we are all a bit less dependent on google) is a good thing.
No criticism from me towards apple or Anthropic. Both parties made their choice. Apple was late to the identity business and the other ships had already sailed.
Third party logins are an extension and a massive risk to any website that doesn't include email hosting.
We have see identity providers dissapear, and people may change their mind.
Easiest way is to register you rown domain and use it with an identity provider of your choice and be able to move it anywhere.
Otherwise we are a faceless citizen of a corporation that can handle access to our identity and everything attached to it without recourse or access to anyone.
Not really. It's the user's fault. Apple provides an option to hide your email, it's not required. It's an option that shows up when you're prompted to create an account.
Oh, I agree with this.
My original thinking was that Apple makes it too easy for a general audience to hide their email without considering the implications (the service won't know your email). But of course there's a tension here, since you also want the option to be easy and accessible.
The party I do not consider at fault in this case is Anthropic.
> I'm sure they didn't want to provide an Apple sign in option at all
But they wanted to provide a Google Sign In? wth?
> This is great for privacy but means the service has the wrong email.
So harm the users to benefit the service? wtf?
I don't want to give my real email or anything to random services, specially not one like Claude where they don't even let you remove your payment info.
> I don't want to give my real email or anything to random services, specially not one like Claude where they don't even let you remove your payment info.
The original complaint was:
>> If you signed up with your Apple on the iOS Claude app, to access your account on the computer, you have to open the passwords app and copy your random email address and paste it into the Claude website login.
Either you use your original email or you use a per-service email. Apple helps you do the latter, but this does come with UX tradeoffs.
Using a per-service email, then complaining that the service does not have your real email, strikes me as misguided.
Add to the list backtick handling. If you start a backtick block on the claude web chat, you cannot leave it with the keyboard. You are now stuck between the backticks. It is as if they wanted to reproduce Slack misery.
Pressing the down arrow while inside a block exits it for me.
They clearly vibe code a lot (most? all?) of their stuff, and it shows. Elementary features are broken regularly and while I appreciate them trying new features, I'd appreciate it more if existing ones were reliable and promptly fixed if broken.
Trouble is, vibe coding refinements and bug fixes works well but probably isn't a good track to promotions at Anthropic (or virtually any other company.)
I asked it to do some portfolio analysis for me and it created BEAUTIFUL, tabbed, interactive charts UNPROMPTED. This is kind of magical. The charts were not just beautiful, but actually super useful in understanding the data faster. I honestly could not have produced those in a week if you asked me to.
Wow, I asked it to build me a simple diagram explaining agile development and it did an amazing job. Wow it felt magical to watch that diagram slowly animating to life.
Like a much prettier version of Mermaid.
Kudos, Anthropic. Geez, this is so nice.
Now I'm going to ask it to draw a diagram of a pelican riding a bicycle, why not?
The artifact output model is more useful than it looks at first. We use Claude in a multi-agent pipeline and discovered that structured artifact outputs reduce parse errors significantly compared to freeform text responses -- the model seems to reason differently when it knows the output will be rendered. Curious whether Anthropic sees similar quality improvements in tool-use tasks when the output has a concrete format constraint.
This is pretty neat and I am experimenting with it now, but hasn't ChatGPT had capability to create graphs and interact with data for a while? "ChatGPT advanced data analysis" for example. I'm asking in good faith as maybe some of you have been using that and can compare the two and give an informed opinion.
I usually use a lot of other tools for data analysis or write code with Claude code or another LLM to do data analysis and visualization.
article about the ChatGPT charts and graphs https://www.zdnet.com/article/how-to-use-chatgpt-to-make-cha...
> but hasn't ChatGPT had capability to create graphs and interact with data for a while?
It's pretty bad (for me). I have to use extremely prescriptive language to tell ChatGPT what to create. Even down to the colours in the chart, because otherwise it puts black font on black background (for example). Then I have to specifically tell it to put it in a canvas, and make it interactive, and make it executable in the canvas. Then if I'm lucky I have to hit a "preview" button in the top right and hope it works (it doesn't). I could write several paragraphs telling it to do something like what Claude just demo'd and it wouldn't come close. I'm trying Claude now for financial insights and it's effortless with beautiful UX.
For posterity, Gemini is pretty good with these interactive canvases. Not nearly as good, but FAR better than ChatGPT.
When using Claude Code, we often prompt it to draft diagrams in MermaidJS syntax.
Great for summarizing a multi-step process and quick to render with simple tools.
It was inevitable until the point all apps will disappear and AI will be the entry point for all work. You can see how anything required appear based on a single request. After which world models and other forms of interaction that are more dynamic will make sense and we'll need something that's not a screen.
Its a large leap from “we made a config driven diagram tool and trained an llm on that config” to “all apps will disappear”. If you’re predicting such grand claims please be more precise than “AI” which is a term we cant define.
Yeah an app doesn't "disappear" because you put an AI interface in front of it and then use a bunch of old school programming to parse LLM output and feed that into your old app. 99% of the work is still building the old app.
You're harshing the vibe, man.
For sure, leave the hype profiteers alone!
And the people that still get excited about life!
I would love to know how they built this. Did they use json-render [0], openui [1], or rolled their own?
[0]: https://github.com/vercel-labs/json-render
[1]: https://github.com/thesysdev/openui
Right-click the page and inspect the source code?
Interesting. So if I'm reading this correctly, this is distinctly different than the artifacts that Claude creates? If that's the case, why create it inline as opposed to an artifact? Any time I get a visual, I tend to find them so useful that I _want_ them to be an artifact that I can export and share.
love to see it, my auto researcher is getting more capable with less effort every release
Claude is broken for me since this was released, prompts are just timing out and stopping after 10 attempts
Does anyone know which library they use? Or something developed internally?
Ok, asked it myself: Chart.js
Unable to reproduce the recipe image in the iOS app. It first gave a normal text answer. Then when referencing this blog post it produced a wonky HTML artifact.
Reliability has been the real bottleneck for multi-agent setups in production. The hard part isnt getting one agent to do something clever once - its making repeated runs observable and bounded when tools fail halfway through. Idempotency checks, explicit handoff state, and human review gates have mattered more for us than adding another model or another agent role.
New account spamming LM replies, nice
Anyone else able to use Claude with Excel? I've tried adding it to our (very small) Office365 org and it just fails. Been failing for months.
All the office js integrations are still pretty shitty
Isn't this mainly a skill injected in the context? Rather a model/platform specific feature?
Aaand all the way at the bottom, there it is. The first glimpse of what will be an ad carousel.
(Literally nobody needs an image of a cake when asking for a cake recipe)
What if you've never seen a cake before?
That's the target group for the feature. You're right. You got me.
Interactive slop is still slop.