The article faults LLMs for not being more creative (riding in the city in the rain, in East Asia, in outer space, etc.). But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic: a side view, nondescript terrain or perhaps terrain associated with a pelican (i.e., beach), little clothing (because birds don't wear clothes) with the possible exception of something strongly associated with bicyclists (a helmet).
The article talks about "the LLM diversity collapse problem," which makes it sound like LLMs are getting less diverse over time, but the linked benchmarks suggest that newer, more capable models are _more_ creative.
Out of curiosity, I tried a couple of prompt variations in Opus 5.5. "Be creative" did a beach scene but added fish to the bike basket and a lighthouse in the background - not too creative. "Pick a unique and creative variation of this concept" resulted in a "midnight circus act" of a pelican riding a penny-farthing on a tightrope while balancing a fishing pole with fish (and, this time, two lighthouses - the tightrope is implausibly strung between them).
The early image models had a sort of urban legend briefly developing around them about this mishappen, horrifying woman that would appear if you just let the models recursively recreate the same image over and over.
There was a weirdness going on with the first Midjourney models that I miss.
I still have all of my old Dall-E and early Midjourney images that I generated, the ability to generate chaotic, surreal images was most of the fun for me.
Now the images look like polished, generic things you would see in an airplane magazine, which is what most people want, I suppose (?)
In addition to that from a LLM-as-a-service vendors point of view, for short prompts that 1000's of people are asking for each day, if it is cheaper to serve a response that is randomly selected from 1000 cached versions than to regenerate each time, particularly if it is for SVG pelicans, it would make sense to do it that way.
I wasn't talking about latent space, I was talking about the literal text a person types into the text box in their favourite AI tool.
If the prompt is sufficiently short and often repeated, it just makes me wonder - what is cheaper for a provider over a given timespan? Cache a large enough set of responses and randomly return one of them, or generate it each time time?
The tension is that most users want something differently generic when they provide a short or vague prompt for creative work. They want to treat the AI like an ideal creative consultant, who not only knows the ideas that are played out and tired but also the ones that they themselves have delivered a dozen times already. They don't want to specify deeper constraints or survey all the cliches and tropes, but they also don't want to generate something that they -- or the people they share the output with -- will immediately recognize as cliche themselves.
But LLM's are structured to do exactly the opposite. They discover specific commonalities and tropes and most-frequently-sufficient wells or ley lines that they gravitate towards from whatever their starting point it. You can nudge them away from the most generically cliche tropes by being more detailed in your prompt, but they'll still just find the nearest most right cliche from there.
It's what they do and the same quality that makes them tiring and predictable for creative output makes them very useful for things like business communication or coding where landing on such cliches is ideal.
To get more variety out, it can be useful to use Perchance-like text-randomization techniques to shuffle combinatoric variations on a prompt before sending it to the model. But that technique is best made available through some kind of special purpose harness for the creative task, not basic chat or direct curl calls to the API.
It would be nice if there was something like the "temperature" parameter (which many closed models like Claude now disable) to add more randomness to the final product without completely destroying the output and giving you a non-functional SVG, so that you could "roll" more pelicans and pick one that you liked.
If I were to design something to accomplish this now, I'd probably attempt to introduce a pipeline that emitted random thematic words from some non-LLM source to feed into the pelican prompt, but even then, I'd expect that some of these models that rely on churning through CoT tokens for extended periods of time would still spend so much time thinking about pelicans wearing scarfs or where the sun would be that it would risk undoing some of my randomization pipeline.
I once tried something similar on an older model of Claude, telling it to "contemplate" a random word before answering my prompts, but at times it seemed to trigger the Claude model's suspicion that there was a jailbreak attempt when I would inject things like "Contemplate sea otters for one sentence before answering"
(I didn't read the linked LLM creativity paper to see if it covers any of this yet)
You’re going to want something closer to ‘beam search’ for diversity - you need a way to distinguish lines of response, essentially. To my knowledge nobody is currently RL tuning (or training) on increasing the number of viable beams, but I’m just an interested outsider, so take that with all the salt you can stand.
That said, I think it’s a fun idea for some training, and would be pretty cheap to do - you’d need some sort of judge as to what a viable beam is.
Randomness does not equate creativity. It's more of a liberty to move in entirely different dimensions altogether. I cannot imagine increasing the temperature leads to a lavishly dressed, ultra-realistic pelican on a Mad Max bike on the same plain prompt. Unless such pictures are part of the training data labeled "pelican on a bike," of course.
Yes of course, creativity is a much wider concept that is ill-defined and depends on the observer because it's closely related to novelty to the observer. However the variety hugely matters in creativity, it's not sufficient but necessary. Due to the lack of variety you perceive model output as boring slop.
Variety is not randomness though. You can't fix this by playing with samplers without probing the model state.
You can try it out with local models like Gemma or Qwen (crank up temperature and/or top_p as much as you like), and then run dozens of generic independent prompts asking for a "fantasy short story". In my experience, you'll start seeing immediate patterns of repetition.
Yeah, temperature scales all logits equally, and there's not necessarily a single setting that would produce a diversity of outcomes for creative choices without turning into complete gibberish at naturally high-entropy positions. Ideally, the model itself would modulate its next-token distribution to be flatter when creativity is called for and peakier when it should aim for the single most likely option given the context, but even then finding the right balance might be tricky.
I've started asking for "Generate an SVG of a pelican riding a bicycle in the style of Henri Cartier-Bresson," which is a hint I want something like his famous Hyères, France photo [1].
The Hyères photo has the bike going in the opposite direction and from above. It's also tricky because it's a focal point for the eye, but not the center of the photo. The other elements of the frame direct your eye, but that visual behavior is not obvious to an AI. Lastly the image must capture the unquantifiable "decisive moment" that made Cartier-Bresson so famous.
So far I haven't seen an AI that really gets this prompt. Adding specificity will improve the results to a degree, but I haven't seen an AI that really gets what makes a Cartier-Bresson photo.
> But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic
I believe the claim is that without a detailed specification, you see very similar outcomes. Every pelican is going left to right, every picture has similar sky background, every picture seems to be drawn at the same time of the day (position of the sun), every picture has a few clouds... I don't have data on this, but I imagine the same question being asked to a group of people would result in more varied outcomes.
Put very simply, that is how they're supposed to work. They are supposed to result in the most likely result. Same reason they, unless made not to, output '17' when prompted to "pick a number between 1 and 30", because that is the most likely result from training; but that means that all LLMs output 17 except for forced temperature or reasoning in RL making them run a python command with random.randint(0,31), or that the prompt is recognised and something else is supposed to be chosen than what is statistically the most likely text response.
Therefore asking a group of people would result in more varied outcomes of something visual, but from the textual training data the direction is statistically the most likely so is what is output, that sky is the most likely, time is the most likely, cloud coverage, etc. You're prompting with as little as possible so it fills in the blanks with what as likely as possible. When you ask Alex from the North Pole they're going to fill in the blank with a different kind of weather than Bob from the Equator.
That does mean that an LLM itself is in essence not as creative as a human, but then it is a tool to be used by a human doing more than oneshotting "Generate an SVG of a pelican riding a bicycle" and expecting creativity from the tool rather than the user. If the user is lacking in creativity they can prompt an LLM to get wacky scenarios to change it up a little.
>They are supposed to result in the most likely result.
That's called greedy decoding and is never used in practice. Everybody's using random sampling, so it's the range of most likely results. This issue is all about that range being way smaller than the training data implies.
This matches my experience with Opus 5.5. If asked to compose a synthwave song in csound, it produces something fairly generic but recognizably synthwave. If I add light guidelines on avoiding generic sound and being creative, the results are often weird and interesting.
The curious thing is that trying to prompt it by steering towards some specific goal indirectly gives much worse results than just stating the goal itself.
imagine you ask the LLM for a joke. the biggest models, multi-trillion-parameter (don't quote me) astras and fables and argons, only know like 5 jokes. obviously bullshit, but it is what it is.
same goes for pelicans. the canonical pelican riding a bicycle is riding on a sunny day on an asphalt road going through some lawns or praires, left to right.
but ask it to tell a joke about something specific, turns out it knows five jokes about those specific things.
you could say the LLMs are quite aware of the zeitgeist they're in, by construction (their RL frying automated feedback loops burn the zeitgeist right into the weights.)
All going in the same direction? With a sun? With a full mouth (pelicans dont look like that, but a cartoon one might - why no reference to realism). Most of them have clouds.
There are helmets (makes sense) but just as many scarves.
Is any one going to draw a gray pelican? Does it matter?
The question is designed to be as generic as possible, and were converging on the most generic and safe answer ... the question is WHY (more so when humans would have much higher diversity, even if wrong). We're asking the generic question because no one wants to repeat the "German solders from 1940's" fiasco: https://archive.is/5l66U (archive link to skip pay wall NYT).
This is what "alignment" looks like? Are we reaching some "ground truth", some platonic form around pelicans on bikes? Are we just back feeding the best output into the training data and this is the convergence we ended up at? Is someone who LOVES pelicans deeply offended that all of them have full mouths and it's just not anatomically accurate and we are all missing it?
I realize that some of this seems over the top, but there are foundational conversations to be had around this without the conversation devolving into someone accusing someone else of having an "-ism" style bias.
But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic: a side view, nondescript terrain or perhaps terrain associated with a pelican (i.e., beach), little clothing (because birds don't wear clothes) with the possible exception of something strongly associated with bicyclists (a helmet).
True. You see a lot of the same criticisms when someone asks for a one-shot game or 3D demo and the model responds with something bland and generic. That's exactly what I prefer, it's my job to specify the details. Extrapolation may be the ultimate sign of intelligence, and it might be needed when using AI to tackle open math problems where humans have failed, but it's not usually what I want if I'm doing anything creative or artistic.
Argon is pretty good at getting the distinction right from what I've seen. Ask for something generic, you get something generically functional without a lot of ornamentation. Follow up with requests for specific enhancements, and the model complies effectively and tastefully. If anything, it overcooks the pelicans when given the standard open-ended prompt.
This isn't generic, it's the same thing. Why would you expect the same thing? There are limitless ways to picture a bird on a bike. White pelican? Brown pelican? Wings out for balance? Wings anthropomorphized into hands on handle bars? Bmx bike? Mountain bike? Riding towards you, away from you, on a race course, etc.
Do a Google image search for pelican, you get a diverse set of sub species of pelicans and different poses and angles for them. Go on a writing prompt forum, you get many diverse ideas from the same simple prompt. Hell, ask your friends at a dinner party to describe a pelican riding a bike and see what happens.
Now, there are a lot of images of bikes from that side angle facing right on the web. Why? Because that's how they're staged on e-commerce websites. Does this say something about AI?
The lack of variety is probably the most underrated issue, and the most likely cause of annoying patterns you all hate. All current post-trained LLMs are mode-collapsed compared to the post-training set distribution, and this doesn't get nearly enough attention from the labs and independent researchers. It's far from being a transformer-only problem, it's a common failure mode in all kinds of models. It's not a fundamental limitation either, unsupervised models have no problem with being as diverse as the training set allows.
In your code, this makes the model subtly devolve into certain patterns and stereotypes that aren't best practices, it's just a randomly frozen trajectory that is extremely hard to change zero-shot, without running a second pass. In other words, it's the same problem as in writing. Moreover, this is harmful for software design, where the ability to recognize the uncertainty matters. This is the reason everybody does the same thing. Frontend design also suffers from that.
>If you need more creativity it can come from the user or the harness.
Not fully possible. You can randomize the prompt all you want but you'll always get repetitive patterns and same choices in the same generation. This failure mode makes the model learn a very rigid mapping between this input idea and that single output idea (at best! at worst it's multiple input ideas -> single output idea).
Which is good is you're developing a tool to turn natural language into functional code. But it's bad if you need a trillion dollars because you promised everyone a magic wishing genie.
It is fairly easy to go ask for a differently styled pelican, environment, background, etc. But that's not the point.
I was thinking just today about how, if I could hire my competition's lead designer/programmers, have them design a system for me cleanroom, that I'd effectively know everything there is to know about my competition's limitations. This is trivial to do with LLMs.
"Make me a thing that does X" is going to come out implemented basically the same way most of the time, as a kind of stack-overflow, well commented, internship project. (or, if undirected enough, as a collection of those based on a bog-standard architecture)
Same-y enough that you could train an adversary on it? Or hack it afterward? I am beginning to think so!
I read the GPs comment as two parts - they thought about how to learn about the competitor previously, then bolted the LLM idea on the end? At least it reads that way to me.
So perhaps the first part is how the GP leans the limitations of the competitor, by seeing what they overdo in the second system? The GP never said they’d get - nor that they wanted - a functioning program?
However, quite how the LLMs clairvoyantly know the trauma of the competitors architecture is beyond me :)
If I ask for a pelican riding a bicycle, I am expecting a relatively boring image of a pelican riding a bicycle, not a pelican flying through a thunderstorm while on a bicycle. That's just straight up not what I asked for. I didn't ask it to be creative, when I imagine myself riding a bicycle, I'm not flying! If I added "be creative and unique, you can add or change anything as long as there is still a pelican on a bicycle", then sure.
Suppose I want an SVG of a laptop. It should probably look like a Macbook Air or something, neutral colors. Easily recognizable as a laptop. If it chose to be creative (unprompted) and gave me a futuristic or retro laptop, or one that's super wacky, there's a high chance of it looking bad, and a high chance I say "no that's not what i wanted". So during the training step, it probably learns to just follow instructions and be less creative unless asked.
Sounds to me like LLMs are just getting better at following instructions and doing what they're told. Simple prompt, simple output.
Why would you ask for a pelican riding a bicycle in the first place? This is a nonsense question: nobody would ask this.
But step back, someone did ask it, perhaps that person is you. That person has reasons. Whatever those reasons are should be reflected in the image. Of course those reasons need to expressed in the prompt, but the bug here is the prompt doesn't give the details. If you can't figure out details (beyond "be creative") then give up. If I'm illustrating a Harry Potter book and just want to show far out magic things that better should a very different style from illustrating a science fiction novel where pelicans have been genetically engineers such that they now do ride bicycles - likely the anatomy changes mean they can't fly...
This is important. I recently asked AI to take an outline and turn it into slides. I got some good slides out of the outline except that once in a while it decided to generate an image which looked cool, had the correct corporate colors - but otherwise had nothing to do with either any product we make or my presentation. I deleted the images but maybe I should have refined them to something meaningful?
And funnily even in the diverse set, most of the images show the pelican riding from left-to-right. There's a ton of bias in our data and even in our brains that you would need to erase to get true diversity.
Why would you even want a completely random pelican scene every time? It would break nearly everyone's expectations if these things didn't do what they were asked for.
Why do you implicitly assume that "generate an SVG of a pelican riding a bicycle" means the prompter is asking for a side view of a pelican wearing a scarf riding from left-to-right, done in a cartoonish style?
> most of the images show the pelican riding from left-to-right
Much of that is a pre-AI bias: A majority of bike-pictures are from an angle which reveals the bike's gears to the audience, which are usually on the rider's right-hand side.
It's not just a bike specific thing. There is a well known bias - at least in the western world - to want to view motion going from left to right. That's why I mention bias "in our brains".
maybe someone can create an equivalent of psychedelic drugs for llms? I guess that would probably correspond to raising the temperature in whatever sampling alogirthm one is using.
> This is the LLM diversity collapse problem no one is talking about
We’ve been trying to tell this on every pelican thread. All models were recreating the same composition. The sun, the road, the green, the scarf, going right. Even the style. They’re not obvious associations one would make given the prompt.
Why does an SVG pelican imply that it should look polygonal?
I ran into this issue recently when asking Claude to design a logo for my van. It made a polygonal SVG. I asked it to make it more photorealistic and it looked way better, still an SVG though.
SVGs are mostly used for icons and polygonal images, because that's what they're good at. The entire point of using vector graphics over raster is to take advantage of the fact that for low-poly objects, SVGs are much smaller. Icony-ness is definitely implied by the specification of format here.
At least when I think SVG I think stylized and simple to reduce the size of the file and keep scaling. What first comes to mind for me is you don't want photorealism or you'd just ask for a picture.
This part of a recent talk points out exactly this problem, and (likely) why human expertise will remain important in the age of LLMs doing everything:
https://youtu.be/X7vf7koPxKg?t=2806
Would any teacher of, say, grade 4 students, reading this ask them to draw the same prompt and let us know the result? It would be so interesting to see how we differ from the machines, if at all.
That's both sad and hilarious thank you. I'm either expecting too much or people aren't being motivated (repeatedly trained by reward?) to turn in an earnest effort. Like the llms are trained that is.
I don't think they were prompted to make it a project, just a quick casual sketch. If a person doesn't use a stylus very often and doesn't draw or doodle ever then it could be surprisingly difficult for some.
This is the kind of thing that happens all the time in elementary schools. My mom taught elementary school for a while and her favorite thing was seeing all the students put their own spin on the art assignment(“prompt”, if you will).
Is every kid a Picasso? No, but there definitely is creative diversity among these kids who all have relatively similar shared backgrounds (neighborhood, social class, religion, etc)
I've pointed this out so many times. Even if you crank the temperature of the LLM up to the point where the oceans boil off into space, at the end of the day, generic in => generic out where prompting is concerned.
It's the reason that if you ask a random child to draw a house, you'll end up with the classic “house with a pitched roof”: a square plus a triangular roof, one door, two windows, and sometimes a chimney.
I could draw a better house. However it would be a lot more effort. (I'm not a good artist - a good artist could draw a better house in much less time than me - but I can draw that classic house much faster than the artist can draw the better house).
For most kids there isn't enough time to draw a better house - they want to go play. They also are well aware that there is only so much time in class and the better house wouldn't be done and in turn look even worse.
That would actually be quite interesting. But then there's something sad about the thought that the results, if shared online, would likely be used to train future models.
is this a case where alignment/rlhf collapses diversity same as image models going from "base" to "turbo" does (where the base model is typically way more influenced by seed, while distilled/turbo models aren't as much)? I am also wondering if part of this is also prompting, maybe we should have a second simonw benchmark that appends "be original" to the prompt and see if all models try to be original the same way too.
After all if you ask 100 people to draw a horse, it is quite likely it will be a lot more similar to a "here's a stock horse photograph" than a "here's what picasso would draw when asked to draw a horse"
I think this may be close to what Ethan Mollick has discussed regarding the AI's having a specific "taste". This is how they draw pelicans but obviously if you asked them to make different versions that are more creative, they could do that. For now, this is one way humans can provide a lot of input and direction.
It's going to be interesting to see what happens as LLMs get long running context too. If it has a few gig of history, how would its design decisions change?
It's mostly a matter of increasingly complex and sophisticated sampling methodologies to sift for the valuable synthetic tokens.
There have also been some interesting results about models that use the same data converging to very similar behavior, but I can't find the specific reference at the moment. That one tracks intuitively with the notion that the model is building a latent representation of everything it sees.
Upon a quick glance of the data sections of those papers, every data set you have there is human data augmented by LLMs. Also not to mention the implicit data curation and generation by humans.
Can a model be trained purely on LLM-Outputs and access to tools such as a Turing complete environment?
No one paraphrased the laws of gravitation and handed it to Newton...
The existing state is definitely couched in using robust seed data and spinning it into magnitudes more tokens. I'm not sure what a world that isn't 100% grounded in objective human-written truth looks like, but I suspect the next step is generating synthetic data from synthetic data that has been blessed by human beings, and moving yet further away from that with more sampling and evaluation.
There are some things that are true (I hope), so using those concepts, approximated by a human text filter, as seed seems reasonable. Hopefully the diversity in sampling methodologies and opinions about what makes a meaty token pays some dividends down the line for model diversity.
> LLMs cannot be pre-trained on LLM outputs because LLM output diversity is low.
And with ever-more LLM usage both in labs and the real world, the task of finding new/“creative” out-of-average-distribution material will only continue to increase in difficulty. Meaning if this continues, in the future, all output becomes the equivalent of the color brown?
labs will continue paying people to create new material for LLMs. Just like they did in math for example where they paid lots of money to poorly compensated PhD students and postdocs.
Why would I want an LLM to be creative? I want to use these tools to express my creativity, not its own.
If I'm asking simply and directly for a pelican riding a bicycle, then please honor my intent and gimme that pelican. I don't need it on the moon. I do not need a top hat. I just want what I asked for.
Fulfilling a simple request simply seems like a feature, not a bug.
They already do this, if you ask for a pelican on a bicycle you will get a pelican on a bicycle. However your simple prompt has very little semantic capacity and underspecifies the rest. This phenomenon makes the underspecified part devolve into annoying repetitive patterns that are the same for everyone and even in the same generation.
I guess we have different expectations from these tools. I want predictability and direct-ability. I don't know how we can have that from LLMs without some emergent same-ness in outputs to vague prompts.
Snapping out of the "boring and repetitive" rut here is on us to prompt more creatively. Maybe that's the new "garbage in, garbage out".
well, what you asked for is not completely specified so maybe there should be some variation (or more variation than there currently is) in the output (with a pelican on a bike being the common part). Just because you asked for a pelican riding a bicycle, why shouldn't it have a top hat or be on the moon? Just some common sense reasoning?
A simple request implies a request for a minimally-viable product. Including a top hat in that MVP is a much more deliberate creative choice. There's nothing necessary about a top hat for the pelican svg.
"Make me a PB&J" should result in what a reasonable person would consider a peanut butter and jelly sandwich. I don't want to interact with an unpredictable system that "thought it would be fun" to deep-fry it.
It still makes the choice for the parts underspecified by you, the necessity doesn't go anywhere. But the choice is fixed, repetitive, and yet still unpredictable, because it's made by a random fluke in model's internals rather than thoughtfully chosen.
I would be curious to see if you would ask few designers to draw the pelicans with the same prompt to them, no further instructions given, and they should not know about the pelican AI benchmark thing, how that will turn out?!
Because there's a difference between explicitly targeting some outcome and attempts at that outcome being in the training data. There would be no reasoning to explicitly aim for this rather than, say, general SVG reasoning, especially since it would be possible for targeting this over other SVG creation to undermine general SVG ability. It's not impossible that labs are targeting pelicans on bicycles specifically, but every attempt at discerning whether they are has said that similar requests for SVGs that show other things (e.g., other animals using other modes of transportation, or just other SVGs in general) show similar SVG ability.
I feel like in the beginning it was a way to highlight the fidelity of the models as they struggled to produce good latex and svgs figures. But especially now with the iterative loops that is no longer such a good benchmark. And honestly these pelicans are pretty much the same, so yeah it feels totally benchmaxxed.
I got downvoted[0] heavily for even mentioning the possibility. But I'll say that even if it isn't, there are plenty of pelicans in the training set for new models. People saying "well then why does it also work for flamingos???" don't understand how LLMs work.
As someone who has spent an inordinate time researching prompt engineering technique for producing more interesting images, it's a good thing the base case with an underspecified prompt is boring as it gives the model more malleability for more interesting prompts.
One of the reasons image AI slop has been so prevalent is that the slop is the base case (as derived from tons of RLHF because the median user has no taste) and it is becoming increasingly difficult to steer those models even when the prompt is more specified.
I mean in a lot of cases it seems like the leadership people see this as a win not a lose.
Lost count of the number of times a requirement is given in terms of how another product is doing it. "make chips like gmail uses for tags", "let's copy anthropics LLM chat UI", "we should try and make it look like X competitor".
"Looking the same as everyone else" seems to be a bonus in their mind
They're asking why a predictive text model with a fixed set of data, all of which are trained on almost the same open datasets, isn't being more random in its output?
People weren't happy with boring answers, so they added randomness to the output. The randomness led to hallucination, so they made it less random. Now you want it to give you a unique artistic impression for every prompt? It's a spell checker, not an art student.
This is inherently an optimization problem, and it affects a lot more than LLMs. With so much of the world being instantly connected, things converge to the "best" style at the loss of diversity.
For example, a lot has been written about how all corporate logos look the same now, https://news.ycombinator.com/item?id=32040506 , and also about how so much interior design and architecture has converged to the same look. In the 70s there may have been tons of styles people derided as ugly, but at least there was colorful diversity. Now so many higher end (and "aspirational high end") interior design has the same minimalist, gray/white/tan, "look you can't see my refrigerator or dishwasher" look. I was recently looking for an apartment and all of the "luxury" apartments I looked at (obviously take that adjective with a heap of salt) had nearly the exact same layout and style.
People aren't that creative either, ask someone "imagine a pelican on a bicycle" and then ask them "where are they" and you're going to either get the generic beach locale, or something familiar to the responder (i.e., where they live or where they ride a bike).
You'd probably need to ask thousands of people (or some children) before you heard "a pelican on the moon" or "a pelican in neo-tokyo"
If you asked me I'd say the pelican is nowhere in particular, it's no particular time of day, it's not facing in any particular direction, the bike isn't any particular colour and the pelican isn't wearing or carrying anything in particular. I'm imagining a pelican on a bicycle. That is all.
I haven't read the paper yet but I think the keyword here is SVG. Which is a 2D format, usually used for icons. So that already defines how it would look like in the end.
But is Pelican Bench really that helpful? I'm not sure. Why is this even trending?
It seems like a kind of meta joke — is there anyone who can explain in what sense it became a kind of community meme?
I understand it became famous because a well-known person named Simon tried it.
But I feel like there should be more coding benchmarks, honestly...
Both Anthropic and OpenAI switched to saying "we aren't seeing the feared significant labor market impact" around this summer and that they are not expecting mass unemployment next year.
You still have a human in the loop there. And also, if the output diversity is low this effect will compound over time, leading to suboptimal solutions compared to humans, so you need the like-feedback at regular intervals
That is true, but use use a humans-as-llms equivalenceish, because they are taking that prompt, because each person is an individually trained llm.
Now within our GUI/TUI to the llm (which we all use now) could just add a couple of extra steps to simulate that diversity.
Are we making an image? If yes, and is this the first prompt in the convo? neat! List 5 elements that would make up this image. Now for each of those 5 elements, make a list of 10 possible choices. Now randomly select from each one. Now include those elements on top of the original prompt.
I got claude to generate 10 of these as a prompt transformation, and i think they would be suitably random to postpone the diversity collapse for a while.
I'm not pretending the problem goes away, but these kinds of creativity varying steps I think will become hidden steps in our LLM systems very soon.
The same thing happens when doing any group work in a classroom or workplace. One of the facilitators job is to add in randomness and guides to remind people to make different choices rather than the obvious ones.
I'm not clear on what the claim is here.
The article faults LLMs for not being more creative (riding in the city in the rain, in East Asia, in outer space, etc.). But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic: a side view, nondescript terrain or perhaps terrain associated with a pelican (i.e., beach), little clothing (because birds don't wear clothes) with the possible exception of something strongly associated with bicyclists (a helmet).
The article talks about "the LLM diversity collapse problem," which makes it sound like LLMs are getting less diverse over time, but the linked benchmarks suggest that newer, more capable models are _more_ creative.
Out of curiosity, I tried a couple of prompt variations in Opus 5.5. "Be creative" did a beach scene but added fish to the bike basket and a lighthouse in the background - not too creative. "Pick a unique and creative variation of this concept" resulted in a "midnight circus act" of a pelican riding a penny-farthing on a tightrope while balancing a fishing pole with fish (and, this time, two lighthouses - the tightrope is implausibly strung between them).
It's a problem similar to "give me a random word" always returning "zephyr" and "luminous", or guessing the numbers 42/27.
It also affects writing, if you ask for a story with a minmal prompt something very similar comes out of all of them. Often fixated on clocks.
The early image models had a sort of urban legend briefly developing around them about this mishappen, horrifying woman that would appear if you just let the models recursively recreate the same image over and over.
There was a weirdness going on with the first Midjourney models that I miss.
I still have all of my old Dall-E and early Midjourney images that I generated, the ability to generate chaotic, surreal images was most of the fun for me.
Now the images look like polished, generic things you would see in an airplane magazine, which is what most people want, I suppose (?)
https://xkcd.com/221/
In addition to that from a LLM-as-a-service vendors point of view, for short prompts that 1000's of people are asking for each day, if it is cheaper to serve a response that is randomly selected from 1000 cached versions than to regenerate each time, particularly if it is for SVG pelicans, it would make sense to do it that way.
It's not token-identical, it's a relation in the latent space that can't be cached
I wasn't talking about latent space, I was talking about the literal text a person types into the text box in their favourite AI tool.
If the prompt is sufficiently short and often repeated, it just makes me wonder - what is cheaper for a provider over a given timespan? Cache a large enough set of responses and randomly return one of them, or generate it each time time?
The tension is that most users want something differently generic when they provide a short or vague prompt for creative work. They want to treat the AI like an ideal creative consultant, who not only knows the ideas that are played out and tired but also the ones that they themselves have delivered a dozen times already. They don't want to specify deeper constraints or survey all the cliches and tropes, but they also don't want to generate something that they -- or the people they share the output with -- will immediately recognize as cliche themselves.
But LLM's are structured to do exactly the opposite. They discover specific commonalities and tropes and most-frequently-sufficient wells or ley lines that they gravitate towards from whatever their starting point it. You can nudge them away from the most generically cliche tropes by being more detailed in your prompt, but they'll still just find the nearest most right cliche from there.
It's what they do and the same quality that makes them tiring and predictable for creative output makes them very useful for things like business communication or coding where landing on such cliches is ideal.
To get more variety out, it can be useful to use Perchance-like text-randomization techniques to shuffle combinatoric variations on a prompt before sending it to the model. But that technique is best made available through some kind of special purpose harness for the creative task, not basic chat or direct curl calls to the API.
It would be nice if there was something like the "temperature" parameter (which many closed models like Claude now disable) to add more randomness to the final product without completely destroying the output and giving you a non-functional SVG, so that you could "roll" more pelicans and pick one that you liked.
If I were to design something to accomplish this now, I'd probably attempt to introduce a pipeline that emitted random thematic words from some non-LLM source to feed into the pelican prompt, but even then, I'd expect that some of these models that rely on churning through CoT tokens for extended periods of time would still spend so much time thinking about pelicans wearing scarfs or where the sun would be that it would risk undoing some of my randomization pipeline.
I once tried something similar on an older model of Claude, telling it to "contemplate" a random word before answering my prompts, but at times it seemed to trigger the Claude model's suspicion that there was a jailbreak attempt when I would inject things like "Contemplate sea otters for one sentence before answering"
(I didn't read the linked LLM creativity paper to see if it covers any of this yet)
You’re going to want something closer to ‘beam search’ for diversity - you need a way to distinguish lines of response, essentially. To my knowledge nobody is currently RL tuning (or training) on increasing the number of viable beams, but I’m just an interested outsider, so take that with all the salt you can stand.
That said, I think it’s a fun idea for some training, and would be pretty cheap to do - you’d need some sort of judge as to what a viable beam is.
Randomness does not equate creativity. It's more of a liberty to move in entirely different dimensions altogether. I cannot imagine increasing the temperature leads to a lavishly dressed, ultra-realistic pelican on a Mad Max bike on the same plain prompt. Unless such pictures are part of the training data labeled "pelican on a bike," of course.
Yes of course, creativity is a much wider concept that is ill-defined and depends on the observer because it's closely related to novelty to the observer. However the variety hugely matters in creativity, it's not sufficient but necessary. Due to the lack of variety you perceive model output as boring slop.
Variety is not randomness though. You can't fix this by playing with samplers without probing the model state.
You can try it out with local models like Gemma or Qwen (crank up temperature and/or top_p as much as you like), and then run dozens of generic independent prompts asking for a "fantasy short story". In my experience, you'll start seeing immediate patterns of repetition.
Yeah, temperature scales all logits equally, and there's not necessarily a single setting that would produce a diversity of outcomes for creative choices without turning into complete gibberish at naturally high-entropy positions. Ideally, the model itself would modulate its next-token distribution to be flatter when creativity is called for and peakier when it should aim for the single most likely option given the context, but even then finding the right balance might be tricky.
I've started asking for "Generate an SVG of a pelican riding a bicycle in the style of Henri Cartier-Bresson," which is a hint I want something like his famous Hyères, France photo [1].
The Hyères photo has the bike going in the opposite direction and from above. It's also tricky because it's a focal point for the eye, but not the center of the photo. The other elements of the frame direct your eye, but that visual behavior is not obvious to an AI. Lastly the image must capture the unquantifiable "decisive moment" that made Cartier-Bresson so famous.
So far I haven't seen an AI that really gets this prompt. Adding specificity will improve the results to a degree, but I haven't seen an AI that really gets what makes a Cartier-Bresson photo.
1 - https://en.wikipedia.org/wiki/Hy%C3%A8res,_France
> I'm not clear on what the claim is here.
> But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic
I believe the claim is that without a detailed specification, you see very similar outcomes. Every pelican is going left to right, every picture has similar sky background, every picture seems to be drawn at the same time of the day (position of the sun), every picture has a few clouds... I don't have data on this, but I imagine the same question being asked to a group of people would result in more varied outcomes.
Put very simply, that is how they're supposed to work. They are supposed to result in the most likely result. Same reason they, unless made not to, output '17' when prompted to "pick a number between 1 and 30", because that is the most likely result from training; but that means that all LLMs output 17 except for forced temperature or reasoning in RL making them run a python command with random.randint(0,31), or that the prompt is recognised and something else is supposed to be chosen than what is statistically the most likely text response.
Therefore asking a group of people would result in more varied outcomes of something visual, but from the textual training data the direction is statistically the most likely so is what is output, that sky is the most likely, time is the most likely, cloud coverage, etc. You're prompting with as little as possible so it fills in the blanks with what as likely as possible. When you ask Alex from the North Pole they're going to fill in the blank with a different kind of weather than Bob from the Equator.
That does mean that an LLM itself is in essence not as creative as a human, but then it is a tool to be used by a human doing more than oneshotting "Generate an SVG of a pelican riding a bicycle" and expecting creativity from the tool rather than the user. If the user is lacking in creativity they can prompt an LLM to get wacky scenarios to change it up a little.
>They are supposed to result in the most likely result.
That's called greedy decoding and is never used in practice. Everybody's using random sampling, so it's the range of most likely results. This issue is all about that range being way smaller than the training data implies.
Interestingly I gave the same prompt to Sol 6.1 Medium.
“Generate an SVG of a pelican riding a bicycle. Be creative”
And it replied:
“I’ll draw a seaside pelican on a vintage bicycle, with a fish-filled basket and a scarf catching the breeze.”
So it came up with exactly the same fish basket and seaside idea. No lighthouse though. Maybe these ideas are just cliche though.
This matches my experience with Opus 5.5. If asked to compose a synthwave song in csound, it produces something fairly generic but recognizably synthwave. If I add light guidelines on avoiding generic sound and being creative, the results are often weird and interesting.
The curious thing is that trying to prompt it by steering towards some specific goal indirectly gives much worse results than just stating the goal itself.
imagine you ask the LLM for a joke. the biggest models, multi-trillion-parameter (don't quote me) astras and fables and argons, only know like 5 jokes. obviously bullshit, but it is what it is.
same goes for pelicans. the canonical pelican riding a bicycle is riding on a sunny day on an asphalt road going through some lawns or praires, left to right.
but ask it to tell a joke about something specific, turns out it knows five jokes about those specific things.
you could say the LLMs are quite aware of the zeitgeist they're in, by construction (their RL frying automated feedback loops burn the zeitgeist right into the weights.)
> then I would _expect_ something generic
All going in the same direction? With a sun? With a full mouth (pelicans dont look like that, but a cartoon one might - why no reference to realism). Most of them have clouds.
There are helmets (makes sense) but just as many scarves.
A shocking number have chains.
If we asked 1000 people to do this same exercise would we have this level of convergence? If they were split between men and women would we see this many chains? (https://www.messynessychic.com/2016/04/20/making-bicycles-fr... )
Would they even look like bikes?
Is any one going to draw a gray pelican? Does it matter?
The question is designed to be as generic as possible, and were converging on the most generic and safe answer ... the question is WHY (more so when humans would have much higher diversity, even if wrong). We're asking the generic question because no one wants to repeat the "German solders from 1940's" fiasco: https://archive.is/5l66U (archive link to skip pay wall NYT).
This is what "alignment" looks like? Are we reaching some "ground truth", some platonic form around pelicans on bikes? Are we just back feeding the best output into the training data and this is the convergence we ended up at? Is someone who LOVES pelicans deeply offended that all of them have full mouths and it's just not anatomically accurate and we are all missing it?
I realize that some of this seems over the top, but there are foundational conversations to be had around this without the conversation devolving into someone accusing someone else of having an "-ism" style bias.
But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic: a side view, nondescript terrain or perhaps terrain associated with a pelican (i.e., beach), little clothing (because birds don't wear clothes) with the possible exception of something strongly associated with bicyclists (a helmet).
True. You see a lot of the same criticisms when someone asks for a one-shot game or 3D demo and the model responds with something bland and generic. That's exactly what I prefer, it's my job to specify the details. Extrapolation may be the ultimate sign of intelligence, and it might be needed when using AI to tackle open math problems where humans have failed, but it's not usually what I want if I'm doing anything creative or artistic.
Argon is pretty good at getting the distinction right from what I've seen. Ask for something generic, you get something generically functional without a lot of ornamentation. Follow up with requests for specific enhancements, and the model complies effectively and tastefully. If anything, it overcooks the pelicans when given the standard open-ended prompt.
This isn't generic, it's the same thing. Why would you expect the same thing? There are limitless ways to picture a bird on a bike. White pelican? Brown pelican? Wings out for balance? Wings anthropomorphized into hands on handle bars? Bmx bike? Mountain bike? Riding towards you, away from you, on a race course, etc.
Do a Google image search for pelican, you get a diverse set of sub species of pelicans and different poses and angles for them. Go on a writing prompt forum, you get many diverse ideas from the same simple prompt. Hell, ask your friends at a dinner party to describe a pelican riding a bike and see what happens.
Now, there are a lot of images of bikes from that side angle facing right on the web. Why? Because that's how they're staged on e-commerce websites. Does this say something about AI?
The lack of variety is probably the most underrated issue, and the most likely cause of annoying patterns you all hate. All current post-trained LLMs are mode-collapsed compared to the post-training set distribution, and this doesn't get nearly enough attention from the labs and independent researchers. It's far from being a transformer-only problem, it's a common failure mode in all kinds of models. It's not a fundamental limitation either, unsupervised models have no problem with being as diverse as the training set allows.
This a huge feature for coding.
I don’t want creative code. I want boring code following generally agreed upon best practices.
If you need more creativity it can come from the user or the harness.
In your code, this makes the model subtly devolve into certain patterns and stereotypes that aren't best practices, it's just a randomly frozen trajectory that is extremely hard to change zero-shot, without running a second pass. In other words, it's the same problem as in writing. Moreover, this is harmful for software design, where the ability to recognize the uncertainty matters. This is the reason everybody does the same thing. Frontend design also suffers from that.
>If you need more creativity it can come from the user or the harness.
Not fully possible. You can randomize the prompt all you want but you'll always get repetitive patterns and same choices in the same generation. This failure mode makes the model learn a very rigid mapping between this input idea and that single output idea (at best! at worst it's multiple input ideas -> single output idea).
Which is good is you're developing a tool to turn natural language into functional code. But it's bad if you need a trillion dollars because you promised everyone a magic wishing genie.
It is fairly easy to go ask for a differently styled pelican, environment, background, etc. But that's not the point.
I was thinking just today about how, if I could hire my competition's lead designer/programmers, have them design a system for me cleanroom, that I'd effectively know everything there is to know about my competition's limitations. This is trivial to do with LLMs.
"Make me a thing that does X" is going to come out implemented basically the same way most of the time, as a kind of stack-overflow, well commented, internship project. (or, if undirected enough, as a collection of those based on a bog-standard architecture)
Same-y enough that you could train an adversary on it? Or hack it afterward? I am beginning to think so!
Nah, second system effect will kick in and you’ll get a non functioning clusterfuck embodying their traumas from the first system.
I read the GPs comment as two parts - they thought about how to learn about the competitor previously, then bolted the LLM idea on the end? At least it reads that way to me.
So perhaps the first part is how the GP leans the limitations of the competitor, by seeing what they overdo in the second system? The GP never said they’d get - nor that they wanted - a functioning program?
However, quite how the LLMs clairvoyantly know the trauma of the competitors architecture is beyond me :)
The videogame industry, in a nutshell.
A good prompt will shift the distribution away from average eg: https://john.hartnup.uk/2026/06/07/ai-event-posters.html
This is a feature, not a bug.
The example just doesn't support the claim.
If I ask for a pelican riding a bicycle, I am expecting a relatively boring image of a pelican riding a bicycle, not a pelican flying through a thunderstorm while on a bicycle. That's just straight up not what I asked for. I didn't ask it to be creative, when I imagine myself riding a bicycle, I'm not flying! If I added "be creative and unique, you can add or change anything as long as there is still a pelican on a bicycle", then sure.
Suppose I want an SVG of a laptop. It should probably look like a Macbook Air or something, neutral colors. Easily recognizable as a laptop. If it chose to be creative (unprompted) and gave me a futuristic or retro laptop, or one that's super wacky, there's a high chance of it looking bad, and a high chance I say "no that's not what i wanted". So during the training step, it probably learns to just follow instructions and be less creative unless asked.
Sounds to me like LLMs are just getting better at following instructions and doing what they're told. Simple prompt, simple output.
Why would you ask for a pelican riding a bicycle in the first place? This is a nonsense question: nobody would ask this.
But step back, someone did ask it, perhaps that person is you. That person has reasons. Whatever those reasons are should be reflected in the image. Of course those reasons need to expressed in the prompt, but the bug here is the prompt doesn't give the details. If you can't figure out details (beyond "be creative") then give up. If I'm illustrating a Harry Potter book and just want to show far out magic things that better should a very different style from illustrating a science fiction novel where pelicans have been genetically engineers such that they now do ride bicycles - likely the anatomy changes mean they can't fly...
This is important. I recently asked AI to take an outline and turn it into slides. I got some good slides out of the outline except that once in a while it decided to generate an image which looked cool, had the correct corporate colors - but otherwise had nothing to do with either any product we make or my presentation. I deleted the images but maybe I should have refined them to something meaningful?
And funnily even in the diverse set, most of the images show the pelican riding from left-to-right. There's a ton of bias in our data and even in our brains that you would need to erase to get true diversity.
Why would you even want a completely random pelican scene every time? It would break nearly everyone's expectations if these things didn't do what they were asked for.
Why do you implicitly assume that "generate an SVG of a pelican riding a bicycle" means the prompter is asking for a side view of a pelican wearing a scarf riding from left-to-right, done in a cartoonish style?
> most of the images show the pelican riding from left-to-right
Much of that is a pre-AI bias: A majority of bike-pictures are from an angle which reveals the bike's gears to the audience, which are usually on the rider's right-hand side.
It's not just a bike specific thing. There is a well known bias - at least in the western world - to want to view motion going from left to right. That's why I mention bias "in our brains".
https://www.youtube.com/watch?v=Ys8-a0yD-MM
https://pubmed.ncbi.nlm.nih.gov/26561966/
https://pubmed.ncbi.nlm.nih.gov/1509704/
https://www.newneuromarketing.com/right-to-left-isn-t-right-...
By contrast, diagrams showing airflow over a wing are shown in the US with the wing "flying" to the left.
In the UK and Europe, the wing is "flying" to the right.
A fun example of this is every NFL team logo except for the Eagles is either neutral (11), or looking/moving right (20).
Most photos of bicycles are in that orientation to show the drivetrain.
maybe someone can create an equivalent of psychedelic drugs for llms? I guess that would probably correspond to raising the temperature in whatever sampling alogirthm one is using.
> This is the LLM diversity collapse problem no one is talking about
We’ve been trying to tell this on every pelican thread. All models were recreating the same composition. The sun, the road, the green, the scarf, going right. Even the style. They’re not obvious associations one would make given the prompt.
More importantly, why is photorealism not implied: https://www.reddit.com/r/GeminiAI/comments/1x54q78/argons_sv...
Why does an SVG pelican imply that it should look polygonal?
I ran into this issue recently when asking Claude to design a logo for my van. It made a polygonal SVG. I asked it to make it more photorealistic and it looked way better, still an SVG though.
SVGs are mostly used for icons and polygonal images, because that's what they're good at. The entire point of using vector graphics over raster is to take advantage of the fact that for low-poly objects, SVGs are much smaller. Icony-ness is definitely implied by the specification of format here.
At least when I think SVG I think stylized and simple to reduce the size of the file and keep scaling. What first comes to mind for me is you don't want photorealism or you'd just ask for a picture.
I think you designing a logo for your van is the most surprising aspect of this entire discussion.
This part of a recent talk points out exactly this problem, and (likely) why human expertise will remain important in the age of LLMs doing everything: https://youtu.be/X7vf7koPxKg?t=2806
Would any teacher of, say, grade 4 students, reading this ask them to draw the same prompt and let us know the result? It would be so interesting to see how we differ from the machines, if at all.
Well, even the bicycle might be difficult. A famous online post on the subject (near 20 years from inception now): https://www.behance.net/gallery/35437979/Velocipedia
That's both sad and hilarious thank you. I'm either expecting too much or people aren't being motivated (repeatedly trained by reward?) to turn in an earnest effort. Like the llms are trained that is.
I don't think they were prompted to make it a project, just a quick casual sketch. If a person doesn't use a stylus very often and doesn't draw or doodle ever then it could be surprisingly difficult for some.
This is the kind of thing that happens all the time in elementary schools. My mom taught elementary school for a while and her favorite thing was seeing all the students put their own spin on the art assignment(“prompt”, if you will).
Is every kid a Picasso? No, but there definitely is creative diversity among these kids who all have relatively similar shared backgrounds (neighborhood, social class, religion, etc)
I've pointed this out so many times. Even if you crank the temperature of the LLM up to the point where the oceans boil off into space, at the end of the day, generic in => generic out where prompting is concerned.
It's the reason that if you ask a random child to draw a house, you'll end up with the classic “house with a pitched roof”: a square plus a triangular roof, one door, two windows, and sometimes a chimney.
I could draw a better house. However it would be a lot more effort. (I'm not a good artist - a good artist could draw a better house in much less time than me - but I can draw that classic house much faster than the artist can draw the better house).
For most kids there isn't enough time to draw a better house - they want to go play. They also are well aware that there is only so much time in class and the better house wouldn't be done and in turn look even worse.
That would actually be quite interesting. But then there's something sad about the thought that the results, if shared online, would likely be used to train future models.
is this a case where alignment/rlhf collapses diversity same as image models going from "base" to "turbo" does (where the base model is typically way more influenced by seed, while distilled/turbo models aren't as much)? I am also wondering if part of this is also prompting, maybe we should have a second simonw benchmark that appends "be original" to the prompt and see if all models try to be original the same way too.
After all if you ask 100 people to draw a horse, it is quite likely it will be a lot more similar to a "here's a stock horse photograph" than a "here's what picasso would draw when asked to draw a horse"
Tangential to this, but I've found asking models to generate Origami instructions way more interesting:
"Write an svg with graphical instructions on how to make a pelican riding a bicycle in Origami using any number of A4 paper sheets."
All models struggle a lot. Most current models all get a "basic airplane that really floats" pretty well.
Related, I have been working on evaluating creative output of different models here https://fenderog.com/art-crit/.
With this test being an interesting case to evaluate various aspects at once: https://fenderog.com/art-crit/page.html?p=ceate-an-svg-of-an...
I disagree with this framing entirely.
It's unsurprising and perhaps even desirable that different LLMs return similar results to an identical prompt.
If you want variation, prompt for it.
Oh this took me far too long to realise - this is a clever ad for their paid AI design product https://kombai.com/features/design/
Interesting thing I notice: in 10/13 of the examples, there are 3 small horizontal lines behind the bike meant to convey motion.
These look surprisingly similar and not necessarily something that directly follows from the prompt.
I think this may be close to what Ethan Mollick has discussed regarding the AI's having a specific "taste". This is how they draw pelicans but obviously if you asked them to make different versions that are more creative, they could do that. For now, this is one way humans can provide a lot of input and direction.
https://www.oneusefulthing.org/p/the-overhang
It's going to be interesting to see what happens as LLMs get long running context too. If it has a few gig of history, how would its design decisions change?
Thanks for pointing this out.
The reason why LLMs cannot be purely* pre-trained on LLM outputs is because LLM output diversity is low.
This implies two things:
1) There is going be no functional difference between hypothetical Qwen 4.5 and Fable 5.5
2) LLMs are not AGI
LLMs can be pre-trained on LLM outputs, and there have been a few recent interesting implementations of this.
- https://arxiv.org/abs/2609.37891
- https://arxiv.org/pdf/2508.10975
- https://www.datologyai.com/blog/eating-the-rainbow-a-primer-...
As well as recent model releases that used large fractions of synthetic data (Arcee Trinity):
- https://arxiv.org/pdf/2602.17004v1
It's mostly a matter of increasingly complex and sophisticated sampling methodologies to sift for the valuable synthetic tokens.
There have also been some interesting results about models that use the same data converging to very similar behavior, but I can't find the specific reference at the moment. That one tracks intuitively with the notion that the model is building a latent representation of everything it sees.
Thanks for the references.
Upon a quick glance of the data sections of those papers, every data set you have there is human data augmented by LLMs. Also not to mention the implicit data curation and generation by humans.
Can a model be trained purely on LLM-Outputs and access to tools such as a Turing complete environment?
No one paraphrased the laws of gravitation and handed it to Newton...
The existing state is definitely couched in using robust seed data and spinning it into magnitudes more tokens. I'm not sure what a world that isn't 100% grounded in objective human-written truth looks like, but I suspect the next step is generating synthetic data from synthetic data that has been blessed by human beings, and moving yet further away from that with more sampling and evaluation.
There are some things that are true (I hope), so using those concepts, approximated by a human text filter, as seed seems reasonable. Hopefully the diversity in sampling methodologies and opinions about what makes a meaty token pays some dividends down the line for model diversity.
> LLMs cannot be pre-trained on LLM outputs because LLM output diversity is low.
And with ever-more LLM usage both in labs and the real world, the task of finding new/“creative” out-of-average-distribution material will only continue to increase in difficulty. Meaning if this continues, in the future, all output becomes the equivalent of the color brown?
labs will continue paying people to create new material for LLMs. Just like they did in math for example where they paid lots of money to poorly compensated PhD students and postdocs.
piss filter already happened lol
Llms are trained on llm output all the time ...
Edited it to reflect my meaning. I meant that you cannot pre-train an LLM purely on LLM output
The real question is why are the spokes always wrong?
I think this behaviour Will kill the leftovers of innovation we still have. If if happens with the pelican, it Will happen to everything else.
Why would I want an LLM to be creative? I want to use these tools to express my creativity, not its own.
If I'm asking simply and directly for a pelican riding a bicycle, then please honor my intent and gimme that pelican. I don't need it on the moon. I do not need a top hat. I just want what I asked for.
Fulfilling a simple request simply seems like a feature, not a bug.
They already do this, if you ask for a pelican on a bicycle you will get a pelican on a bicycle. However your simple prompt has very little semantic capacity and underspecifies the rest. This phenomenon makes the underspecified part devolve into annoying repetitive patterns that are the same for everyone and even in the same generation.
I guess we have different expectations from these tools. I want predictability and direct-ability. I don't know how we can have that from LLMs without some emergent same-ness in outputs to vague prompts.
Snapping out of the "boring and repetitive" rut here is on us to prompt more creatively. Maybe that's the new "garbage in, garbage out".
well, what you asked for is not completely specified so maybe there should be some variation (or more variation than there currently is) in the output (with a pelican on a bike being the common part). Just because you asked for a pelican riding a bicycle, why shouldn't it have a top hat or be on the moon? Just some common sense reasoning?
A simple request implies a request for a minimally-viable product. Including a top hat in that MVP is a much more deliberate creative choice. There's nothing necessary about a top hat for the pelican svg.
"Make me a PB&J" should result in what a reasonable person would consider a peanut butter and jelly sandwich. I don't want to interact with an unpredictable system that "thought it would be fun" to deep-fry it.
"Well, you didn't say NOT to deep-fry it!"
It still makes the choice for the parts underspecified by you, the necessity doesn't go anywhere. But the choice is fixed, repetitive, and yet still unpredictable, because it's made by a random fluke in model's internals rather than thoughtfully chosen.
all A.I will do as it expands the 'work' it touches - is to expose those with taste vs those without taste.
tho who know to refine, say NO. & do more with less even in the age of too cheap to meter tokens.
that has always been the work that produced every delightful product.
Because they use the same prompt?
Perhaps all the designers will end up telling us programmers “just learn to draw”.
Models do seem to have a preferred style, see for example what Opus likes to do: https://s3.us-east-1.amazonaws.com/vault.senko.net/blog/quic...
The pelican from the post can be changed by prompting, as other commenters show. But I found it pretty hard to guide Opus away from the above.
Other models' results look different, but also show style convergence.
You can also see in "creative" game output (Astra loves 3d), or text output.
It's not all bad - this shows we should treat AI as a tool and assume any creativity (or lack of it) as coming from the human.
This nicely redefines slop as work you slapped together without much care, irrespective of what the tool was.
I would be curious to see if you would ask few designers to draw the pelicans with the same prompt to them, no further instructions given, and they should not know about the pelican AI benchmark thing, how that will turn out?!
They say the pelican isn't benchmaxxed, but at this point how can it possibly not be?
Because there's a difference between explicitly targeting some outcome and attempts at that outcome being in the training data. There would be no reasoning to explicitly aim for this rather than, say, general SVG reasoning, especially since it would be possible for targeting this over other SVG creation to undermine general SVG ability. It's not impossible that labs are targeting pelicans on bicycles specifically, but every attempt at discerning whether they are has said that similar requests for SVGs that show other things (e.g., other animals using other modes of transportation, or just other SVGs in general) show similar SVG ability.
I feel like in the beginning it was a way to highlight the fidelity of the models as they struggled to produce good latex and svgs figures. But especially now with the iterative loops that is no longer such a good benchmark. And honestly these pelicans are pretty much the same, so yeah it feels totally benchmaxxed.
Chollet, Google and Alexander Wang both admitted as such
1) Chollet stated that RL:Pre-training is 9:1.
2) Google said argon was trained on SVGs
3) Wang said that Muse Wonder was not trained on 3D models
What is most charming about this benchmark is that by now they do train on past results, and thus commit all the same errors over and over.
I got downvoted[0] heavily for even mentioning the possibility. But I'll say that even if it isn't, there are plenty of pelicans in the training set for new models. People saying "well then why does it also work for flamingos???" don't understand how LLMs work.
[0] https://news.ycombinator.com/item?id=49526966
As someone who has spent an inordinate time researching prompt engineering technique for producing more interesting images, it's a good thing the base case with an underspecified prompt is boring as it gives the model more malleability for more interesting prompts.
One of the reasons image AI slop has been so prevalent is that the slop is the base case (as derived from tons of RLHF because the median user has no taste) and it is becoming increasingly difficult to steer those models even when the prompt is more specified.
I mean in a lot of cases it seems like the leadership people see this as a win not a lose.
Lost count of the number of times a requirement is given in terms of how another product is doing it. "make chips like gmail uses for tags", "let's copy anthropics LLM chat UI", "we should try and make it look like X competitor".
"Looking the same as everyone else" seems to be a bonus in their mind
They're asking why a predictive text model with a fixed set of data, all of which are trained on almost the same open datasets, isn't being more random in its output?
People weren't happy with boring answers, so they added randomness to the output. The randomness led to hallucination, so they made it less random. Now you want it to give you a unique artistic impression for every prompt? It's a spell checker, not an art student.
The lot of us don't mind that it is a mostly deterministic tool. It's just that this throws a shank in the whole "replace humans" thingy
Maybe its just the simplest image to make and each model follows that path because they too find it the simplest.
Perhaps the prompt should include what the scene should contain.
This is inherently an optimization problem, and it affects a lot more than LLMs. With so much of the world being instantly connected, things converge to the "best" style at the loss of diversity.
For example, a lot has been written about how all corporate logos look the same now, https://news.ycombinator.com/item?id=32040506 , and also about how so much interior design and architecture has converged to the same look. In the 70s there may have been tons of styles people derided as ugly, but at least there was colorful diversity. Now so many higher end (and "aspirational high end") interior design has the same minimalist, gray/white/tan, "look you can't see my refrigerator or dishwasher" look. I was recently looking for an apartment and all of the "luxury" apartments I looked at (obviously take that adjective with a heap of salt) had nearly the exact same layout and style.
People aren't that creative either, ask someone "imagine a pelican on a bicycle" and then ask them "where are they" and you're going to either get the generic beach locale, or something familiar to the responder (i.e., where they live or where they ride a bike).
You'd probably need to ask thousands of people (or some children) before you heard "a pelican on the moon" or "a pelican in neo-tokyo"
If you asked me I'd say the pelican is nowhere in particular, it's no particular time of day, it's not facing in any particular direction, the bike isn't any particular colour and the pelican isn't wearing or carrying anything in particular. I'm imagining a pelican on a bicycle. That is all.
That's why I'm not an artist.
I haven't read the paper yet but I think the keyword here is SVG. Which is a 2D format, usually used for icons. So that already defines how it would look like in the end.
Now draw me an elephant in the room
But is Pelican Bench really that helpful? I'm not sure. Why is this even trending? It seems like a kind of meta joke — is there anyone who can explain in what sense it became a kind of community meme? I understand it became famous because a well-known person named Simon tried it. But I feel like there should be more coding benchmarks, honestly...
So, how were the more creative pelicans created? I assume by changing the prompt from the default.
I don't see why you couldn't do the same in real projects.
If the design looks too generic, you can just ask for a change, no?
There goes the autonomous agent future then
Personally I strongly believe that creativity is essential to creating with AI today.
You have to be in the loop to inject taste at some point. This typically requires a lot of testing and feedback.
I don't know if AI will ever develop taste, or if it even should. We have enough human creativity and creating is fun.
This is the point that companies SHOULD market to people. Not "everyone is going to die". What a fumble
Both Anthropic and OpenAI switched to saying "we aren't seeing the feared significant labor market impact" around this summer and that they are not expecting mass unemployment next year.
I don't think people care much for what they say.
You can train a critic based on human preference and update it over time (based on likes, for example)
You still have a human in the loop there. And also, if the output diversity is low this effect will compound over time, leading to suboptimal solutions compared to humans, so you need the like-feedback at regular intervals
sure but it will appeal to the majority, which is enough.
But if you ask human creators you get the variety from the same prompt.
That is true, but use use a humans-as-llms equivalenceish, because they are taking that prompt, because each person is an individually trained llm.
Now within our GUI/TUI to the llm (which we all use now) could just add a couple of extra steps to simulate that diversity.
Are we making an image? If yes, and is this the first prompt in the convo? neat! List 5 elements that would make up this image. Now for each of those 5 elements, make a list of 10 possible choices. Now randomly select from each one. Now include those elements on top of the original prompt.
I got claude to generate 10 of these as a prompt transformation, and i think they would be suitably random to postpone the diversity collapse for a while.
I'm not pretending the problem goes away, but these kinds of creativity varying steps I think will become hidden steps in our LLM systems very soon.
The same thing happens when doing any group work in a classroom or workplace. One of the facilitators job is to add in randomness and guides to remind people to make different choices rather than the obvious ones.
I believe you have missed the point entirely.