The article faults LLMs for not being more creative (riding in the city in the rain, in East Asia, in outer space, etc.). But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic: a side view, nondescript terrain or perhaps terrain associated with a pelican (i.e., beach), little clothing (because birds don't wear clothes) with the possible exception of something strongly associated with bicyclists (a helmet).
The article talks about "the LLM diversity collapse problem," which makes it sound like LLMs are getting less diverse over time, but the linked benchmarks suggest that newer, more capable models are _more_ creative.
Out of curiosity, I tried a couple of prompt variations in Opus 5.5. "Be creative" did a beach scene but added fish to the bike basket and a lighthouse in the background - not too creative. "Pick a unique and creative variation of this concept" resulted in a "midnight circus act" of a pelican riding a penny-farthing on a tightrope while balancing a fishing pole with fish (and, this time, two lighthouses - the tightrope is implausibly strung between them).
The early image models had a sort of urban legend briefly developing around them about this mishappen, horrifying woman that would appear if you just let the models recursively recreate the same image over and over.
There was a weirdness going on with the first Midjourney models that I miss.
I still have all of my old Dall-E and early Midjourney images that I generated, the ability to generate chaotic, surreal images was most of the fun for me.
Now the images look like polished, generic things you would see in an airplane magazine, which is what most people want, I suppose (?)
In addition to that from a LLM-as-a-service vendors point of view, for short prompts that 1000's of people are asking for each day, if it is cheaper to serve a response that is randomly selected from 1000 cached versions than to regenerate each time, particularly if it is for SVG pelicans, it would make sense to do it that way.
I wasn't talking about latent space, I was talking about the literal text a person types into the text box in their favourite AI tool.
If the prompt is sufficiently short and often repeated, it just makes me wonder - what is cheaper for a provider over a given timespan? Cache a large enough set of responses and randomly return one of them, or generate it each time time?
It would be nice if there was something like the "temperature" parameter (which many closed models like Claude now disable) to add more randomness to the final product without completely destroying the output and giving you a non-functional SVG, so that you could "roll" more pelicans and pick one that you liked.
If I were to design something to accomplish this now, I'd probably attempt to introduce a pipeline that emitted random thematic words from some non-LLM source to feed into the pelican prompt, but even then, I'd expect that some of these models that rely on churning through CoT tokens for extended periods of time would still spend so much time thinking about pelicans wearing scarfs or where the sun would be that it would risk undoing some of my randomization pipeline.
I once tried something similar on an older model of Claude, telling it to "contemplate" a random word before answering my prompts, but at times it seemed to trigger the Claude model's suspicion that there was a jailbreak attempt when I would inject things like "Contemplate sea otters for one sentence before answering"
(I didn't read the linked LLM creativity paper to see if it covers any of this yet)
You can try it out with local models like Gemma or Qwen (crank up temperature and/or top_p as much as you like), and then run dozens of generic independent prompts asking for a "fantasy short story". In my experience, you'll start seeing immediate patterns of repetition.
Randomness does not equate creativity. It's more of a liberty to move in entirely different dimensions altogether. I cannot imagine increasing the temperature leads to a lavishly dressed, ultra-realistic pelican on a Mad Max bike on the same plain prompt. Unless such pictures are part of the training data labeled "pelican on a bike," of course.
Yes of course, creativity is a much wider concept that is ill-defined and depends on the observer because it's closely related to novelty to the observer. However the variety hugely matters in creativity, it's not sufficient but necessary. Due to the lack of variety you perceive model output as boring slop.
This matches my experience with Opus 5.5. If asked to compose a synthwave song in csound, it produces something fairly generic but recognizably synthwave. If I add light guidelines on avoiding generic sound and being creative, the results are often weird and interesting.
The curious thing is that trying to prompt it by steering towards some specific goal indirectly gives much worse results than just stating the goal itself.
I've started asking for "Generate an SVG of a pelican riding a bicycle in the style of Henri Cartier-Bresson," which is a hint I want something like his famous Hyères, France photo [1].
The Hyères photo has the bike going in the opposite direction and from above. It's also tricky because it's a focal point for the eye, but not the center of the photo. The other elements of the frame direct your eye, but that visual behavior is not obvious to an AI. Lastly the image must capture the unquantifiable "decisive moment" that made Cartier-Bresson so famous.
So far I haven't seen an AI that really gets this prompt. Adding specificity will improve the results to a degree, but I haven't seen an AI that really gets what makes a Cartier-Bresson photo.
> But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic
I believe the claim is that without a detailed specification, you see very similar outcomes. Every pelican is going left to right, every picture has similar sky background, every picture seems to be drawn at the same time of the day (position of the sun), every picture has a few clouds... I don't have data on this, but I imagine the same question being asked to a group of people would result in more varied outcomes.
imagine you ask the LLM for a joke. the biggest models, multi-trillion-parameter (don't quote me) astras and fables and argons, only know like 5 jokes. obviously bullshit, but it is what it is.
same goes for pelicans. the canonical pelican riding a bicycle is riding on a sunny day on an asphalt road going through some lawns or praires, left to right.
but ask it to tell a joke about something specific, turns out it knows five jokes about those specific things.
you could say the LLMs are quite aware of the zeitgeist they're in, by construction (their RL frying automated feedback loops burn the zeitgeist right into the weights.)
But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic: a side view, nondescript terrain or perhaps terrain associated with a pelican (i.e., beach), little clothing (because birds don't wear clothes) with the possible exception of something strongly associated with bicyclists (a helmet).
True. You see a lot of the same criticisms when someone asks for a one-shot game or 3D demo and the model responds with something bland and generic. That's exactly what I prefer, it's my job to specify the details. Extrapolation may be the ultimate sign of intelligence, and it might be needed when using AI to tackle open math problems where humans have failed, but it's not usually what I want if I'm doing anything creative or artistic.
Argon is pretty good at getting the distinction right from what I've seen. Ask for something generic, you get something generically functional without a lot of ornamentation. Follow up with requests for specific enhancements, and the model complies effectively and tastefully. If anything, it overcooks the pelicans when given the standard open-ended prompt.
The lack of variety is probably the most underrated issue, and the most likely cause of annoying patterns you all hate. All current post-trained LLMs are mode-collapsed compared to the post-training set distribution, and this doesn't get nearly enough attention from the labs and independent researchers. It's far from being a transformer-only problem, it's a common failure mode in all kinds of models. It's not a fundamental limitation either, unsupervised models have no problem with being as diverse as the training set allows.
In your code, this makes the model subtly devolve into certain patterns and stereotypes that aren't best practices, it's just a randomly frozen trajectory that is extremely hard to change zero-shot, without running a second pass. In other words, it's the same problem as in writing. Moreover, this is harmful for software design, where the ability to recognize the uncertainty matters. This is the reason everybody does the same thing. Frontend design also suffers from that.
>If you need more creativity it can come from the user or the harness.
Not fully possible. You can randomize the prompt all you want but you'll always get repetitive patterns and same choices in the same generation. This failure mode makes the model learn a very rigid mapping between this input idea and that single output idea (at best! at worst it's multiple input ideas -> single output idea).
Which is good is you're developing a tool to turn natural language into functional code. But it's bad if you need a trillion dollars because you promised everyone a magic wishing genie.
If I ask for a pelican riding a bicycle, I am expecting a relatively boring image of a pelican riding a bicycle, not a pelican flying through a thunderstorm while on a bicycle. That's just straight up not what I asked for. I didn't ask it to be creative, when I imagine myself riding a bicycle, I'm not flying! If I added "be creative and unique, you can add or change anything as long as there is still a pelican on a bicycle", then sure.
Suppose I want an SVG of a laptop. It should probably look like a Macbook Air or something, neutral colors. Easily recognizable as a laptop. If it chose to be creative (unprompted) and gave me a futuristic or retro laptop, or one that's super wacky, there's a high chance of it looking bad, and a high chance I say "no that's not what i wanted". So during the training step, it probably learns to just follow instructions and be less creative unless asked.
Sounds to me like LLMs are just getting better at following instructions and doing what they're told. Simple prompt, simple output.
Why would you ask for a pelican riding a bicycle in the first place? This is a nonsense question: nobody would ask this.
But step back, someone did ask it, perhaps that person is you. That person has reasons. Whatever those reasons are should be reflected in the image. Of course those reasons need to expressed in the prompt, but the bug here is the prompt doesn't give the details. If you can't figure out details (beyond "be creative") then give up. If I'm illustrating a Harry Potter book and just want to show far out magic things that better should a very different style from illustrating a science fiction novel where pelicans have been genetically engineers such that they now do ride bicycles - likely the anatomy changes mean they can't fly...
This is important. I recently asked AI to take an outline and turn it into slides. I got some good slides out of the outline except that once in a while it decided to generate an image which looked cool, had the correct corporate colors - but otherwise had nothing to do with either any product we make or my presentation. I deleted the images but maybe I should have refined them to something meaningful?
It is fairly easy to go ask for a differently styled pelican, environment, background, etc. But that's not the point.
I was thinking just today about how, if I could hire my competition's lead designer/programmers, have them design a system for me cleanroom, that I'd effectively know everything there is to know about my competition's limitations. This is trivial to do with LLMs.
"Make me a thing that does X" is going to come out implemented basically the same way most of the time, as a kind of stack-overflow, well commented, internship project. (or, if undirected enough, as a collection of those based on a bog-standard architecture)
Same-y enough that you could train an adversary on it? Or hack it afterward? I am beginning to think so!
I read the GPs comment as two parts - they thought about how to learn about the competitor previously, then bolted the LLM idea on the end? At least it reads that way to me.
So perhaps the first part is how the GP leans the limitations of the competitor, by seeing what they overdo in the second system? The GP never said they’d get - nor that they wanted - a functioning program?
However, quite how the LLMs clairvoyantly know the trauma of the competitors architecture is beyond me :)
And funnily even in the diverse set, most of the images show the pelican riding from left-to-right. There's a ton of bias in our data and even in our brains that you would need to erase to get true diversity.
Why would you even want a completely random pelican scene every time? It would break nearly everyone's expectations if these things didn't do what they were asked for.
Why do you implicitly assume that "generate an SVG of a pelican riding a bicycle" means the prompter is asking for a side view of a pelican wearing a scarf riding from left-to-right, done in a cartoonish style?
> most of the images show the pelican riding from left-to-right
Much of that is a pre-AI bias: A majority of bike-pictures are from an angle which reveals the bike's gears to the audience, which are usually on the rider's right-hand side.
It's not just a bike specific thing. There is a well known bias - at least in the western world - to want to view motion going from left to right. That's why I mention bias "in our brains".
maybe someone can create an equivalent of psychedelic drugs for llms? I guess that would probably correspond to raising the temperature in whatever sampling alogirthm one is using.
> This is the LLM diversity collapse problem no one is talking about
We’ve been trying to tell this on every pelican thread. All models were recreating the same composition. The sun, the road, the green, the scarf, going right. Even the style. They’re not obvious associations one would make given the prompt.
Would any teacher of, say, grade 4 students, reading this ask them to draw the same prompt and let us know the result? It would be so interesting to see how we differ from the machines, if at all.
That's both sad and hilarious thank you. I'm either expecting too much or people aren't being motivated (repeatedly trained by reward?) to turn in an earnest effort. Like the llms are trained that is.
I don't think they were prompted to make it a project, just a quick casual sketch. If a person doesn't use a stylus very often and doesn't draw or doodle ever then it could be surprisingly difficult for some.
This is the kind of thing that happens all the time in elementary schools. My mom taught elementary school for a while and her favorite thing was seeing all the students put their own spin on the art assignment(“prompt”, if you will).
Is every kid a Picasso? No, but there definitely is creative diversity among these kids who all have relatively similar shared backgrounds (neighborhood, social class, religion, etc)
I've pointed this out so many times. Even if you crank the temperature of the LLM up to the point where the oceans boil off into space, at the end of the day, generic in => generic out where prompting is concerned.
It's the reason that if you ask a random child to draw a house, you'll end up with the classic “house with a pitched roof”: a square plus a triangular roof, one door, two windows, and sometimes a chimney.
That would actually be quite interesting. But then there's something sad about the thought that the results, if shared online, would likely be used to train future models.
is this a case where alignment/rlhf collapses diversity same as image models going from "base" to "turbo" does (where the base model is typically way more influenced by seed, while distilled/turbo models aren't as much)? I am also wondering if part of this is also prompting, maybe we should have a second simonw benchmark that appends "be original" to the prompt and see if all models try to be original the same way too.
After all if you ask 100 people to draw a horse, it is quite likely it will be a lot more similar to a "here's a stock horse photograph" than a "here's what picasso would draw when asked to draw a horse"
Why does an SVG pelican imply that it should look polygonal?
I ran into this issue recently when asking Claude to design a logo for my van. It made a polygonal SVG. I asked it to make it more photorealistic and it looked way better, still an SVG though.
SVGs are mostly used for icons and polygonal images, because that's what they're good at. The entire point of using vector graphics over raster is to take advantage of the fact that for low-poly objects, SVGs are much smaller. Icony-ness is definitely implied by the specification of format here.
At least when I think SVG I think stylized and simple to reduce the size of the file and keep scaling. What first comes to mind for me is you don't want photorealism or you'd just ask for a picture.
I think this may be close to what Ethan Mollick has discussed regarding the AI's having a specific "taste". This is how they draw pelicans but obviously if you asked them to make different versions that are more creative, they could do that. For now, this is one way humans can provide a lot of input and direction.
It's going to be interesting to see what happens as LLMs get long running context too. If it has a few gig of history, how would its design decisions change?
It's mostly a matter of increasingly complex and sophisticated sampling methodologies to sift for the valuable synthetic tokens.
There have also been some interesting results about models that use the same data converging to very similar behavior, but I can't find the specific reference at the moment. That one tracks intuitively with the notion that the model is building a latent representation of everything it sees.
Upon a quick glance of the data sections of those papers, every data set you have there is human data augmented by LLMs. Also not to mention the implicit data curation and generation by humans.
Can a model be trained purely on LLM-Outputs and access to tools such as a Turing complete environment?
No one paraphrased the laws of gravitation and handed it to Newton...
The existing state is definitely couched in using robust seed data and spinning it into magnitudes more tokens. I'm not sure what a world that isn't 100% grounded in objective human-written truth looks like, but I suspect the next step is generating synthetic data from synthetic data that has been blessed by human beings, and moving yet further away from that with more sampling and evaluation.
There are some things that are true (I hope), so using those concepts, approximated by a human text filter, as seed seems reasonable. Hopefully the diversity in sampling methodologies and opinions about what makes a meaty token pays some dividends down the line for model diversity.
> LLMs cannot be pre-trained on LLM outputs because LLM output diversity is low.
And with ever-more LLM usage both in labs and the real world, the task of finding new/“creative” out-of-average-distribution material will only continue to increase in difficulty. Meaning if this continues, in the future, all output becomes the equivalent of the color brown?
labs will continue paying people to create new material for LLMs. Just like they did in math for example where they paid lots of money to poorly compensated PhD students and postdocs.
Why would I want an LLM to be creative? I want to use these tools to express my creativity, not its own.
If I'm asking simply and directly for a pelican riding a bicycle, then please honor my intent and gimme that pelican. I don't need it on the moon. I do not need a top hat. I just want what I asked for.
Fulfilling a simple request simply seems like a feature, not a bug.
They already do this, if you ask for a pelican on a bicycle you will get a pelican on a bicycle. However your simple prompt has very little semantic capacity and underspecifies the rest. This phenomenon makes the underspecified part devolve into annoying repetitive patterns that are the same for everyone and even in the same generation.
well, what you asked for is not completely specified so maybe there should be some variation (or more variation than there currently is) in the output (with a pelican on a bike being the common part). Just because you asked for a pelican riding a bicycle, why shouldn't it have a top hat or be on the moon? Just some common sense reasoning?
A simple request implies a request for a minimally-viable product. Including a top hat in that MVP is a much more deliberate creative choice. There's nothing necessary about a top hat for the pelican svg.
"Make me a PB&J" should result in what a reasonable person would consider a peanut butter and jelly sandwich. I don't want to interact with an unpredictable system that "thought it would be fun" to deep-fry it.
It still makes the choice for the parts underspecified by you, the necessity doesn't go anywhere. But the choice is fixed, repetitive, and yet still unpredictable, because it's made by a random fluke in model's internals rather than thoughtfully chosen.
As someone who has spent an inordinate time researching prompt engineering technique for producing more interesting images, it's a good thing the base case with an underspecified prompt is boring as it gives the model more malleability for more interesting prompts.
One of the reasons image AI slop has been so prevalent is that the slop is the base case (as derived from tons of RLHF because the median user has no taste) and it is becoming increasingly difficult to steer those models even when the prompt is more specified.
I would be curious to see if you would ask few designers to draw the pelicans with the same prompt to them, no further instructions given, and they should not know about the pelican AI benchmark thing, how that will turn out?!
Because there's a difference between explicitly targeting some outcome and attempts at that outcome being in the training data. There would be no reasoning to explicitly aim for this rather than, say, general SVG reasoning, especially since it would be possible for targeting this over other SVG creation to undermine general SVG ability. It's not impossible that labs are targeting pelicans on bicycles specifically, but every attempt at discerning whether they are has said that similar requests for SVGs that show other things (e.g., other animals using other modes of transportation, or just other SVGs in general) show similar SVG ability.
I feel like in the beginning it was a way to highlight the fidelity of the models as they struggled to produce good latex and svgs figures. But especially now with the iterative loops that is no longer such a good benchmark. And honestly these pelicans are pretty much the same, so yeah it feels totally benchmaxxed.
I got downvoted[0] heavily for even mentioning the possibility. But I'll say that even if it isn't, there are plenty of pelicans in the training set for new models. People saying "well then why does it also work for flamingos???" don't understand how LLMs work.
I mean in a lot of cases it seems like the leadership people see this as a win not a lose.
Lost count of the number of times a requirement is given in terms of how another product is doing it. "make chips like gmail uses for tags", "let's copy anthropics LLM chat UI", "we should try and make it look like X competitor".
"Looking the same as everyone else" seems to be a bonus in their mind
They're asking why a predictive text model with a fixed set of data, all of which are trained on almost the same open datasets, isn't being more random in its output?
People weren't happy with boring answers, so they added randomness to the output. The randomness led to hallucination, so they made it less random. Now you want it to give you a unique artistic impression for every prompt? It's a spell checker, not an art student.
This is inherently an optimization problem, and it affects a lot more than LLMs. With so much of the world being instantly connected, things converge to the "best" style at the loss of diversity.
For example, a lot has been written about how all corporate logos look the same now, https://news.ycombinator.com/item?id=32040506 , and also about how so much interior design and architecture has converged to the same look. In the 70s there may have been tons of styles people derided as ugly, but at least there was colorful diversity. Now so many higher end (and "aspirational high end") interior design has the same minimalist, gray/white/tan, "look you can't see my refrigerator or dishwasher" look. I was recently looking for an apartment and all of the "luxury" apartments I looked at (obviously take that adjective with a heap of salt) had nearly the exact same layout and style.
People aren't that creative either, ask someone "imagine a pelican on a bicycle" and then ask them "where are they" and you're going to either get the generic beach locale, or something familiar to the responder (i.e., where they live or where they ride a bike).
You'd probably need to ask thousands of people (or some children) before you heard "a pelican on the moon" or "a pelican in neo-tokyo"
If you asked me I'd say the pelican is nowhere in particular, it's no particular time of day, it's not facing in any particular direction, the bike isn't any particular colour and the pelican isn't wearing or carrying anything in particular. I'm imagining a pelican on a bicycle. That is all.
I haven't read the paper yet but I think the keyword here is SVG. Which is a 2D format, usually used for icons. So that already defines how it would look like in the end.
But is Pelican Bench really that helpful? I'm not sure. Why is this even trending?
It seems like a kind of meta joke — is there anyone who can explain in what sense it became a kind of community meme?
I understand it became famous because a well-known person named Simon tried it.
But I feel like there should be more coding benchmarks, honestly...
Both Anthropic and OpenAI switched to saying "we aren't seeing the feared significant labor market impact" around this summer and that they are not expecting mass unemployment next year.
You still have a human in the loop there. And also, if the output diversity is low this effect will compound over time, leading to suboptimal solutions compared to humans, so you need the like-feedback at regular intervals
That is true, but use use a humans-as-llms equivalenceish, because they are taking that prompt, because each person is an individually trained llm.
Now within our GUI/TUI to the llm (which we all use now) could just add a couple of extra steps to simulate that diversity.
Are we making an image? If yes, and is this the first prompt in the convo? neat! List 5 elements that would make up this image. Now for each of those 5 elements, make a list of 10 possible choices. Now randomly select from each one. Now include those elements on top of the original prompt.
I got claude to generate 10 of these as a prompt transformation, and i think they would be suitably random to postpone the diversity collapse for a while.
I'm not pretending the problem goes away, but these kinds of creativity varying steps I think will become hidden steps in our LLM systems very soon.
The same thing happens when doing any group work in a classroom or workplace. One of the facilitators job is to add in randomness and guides to remind people to make different choices rather than the obvious ones.
The article faults LLMs for not being more creative (riding in the city in the rain, in East Asia, in outer space, etc.). But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic: a side view, nondescript terrain or perhaps terrain associated with a pelican (i.e., beach), little clothing (because birds don't wear clothes) with the possible exception of something strongly associated with bicyclists (a helmet).
The article talks about "the LLM diversity collapse problem," which makes it sound like LLMs are getting less diverse over time, but the linked benchmarks suggest that newer, more capable models are _more_ creative.
Out of curiosity, I tried a couple of prompt variations in Opus 5.5. "Be creative" did a beach scene but added fish to the bike basket and a lighthouse in the background - not too creative. "Pick a unique and creative variation of this concept" resulted in a "midnight circus act" of a pelican riding a penny-farthing on a tightrope while balancing a fishing pole with fish (and, this time, two lighthouses - the tightrope is implausibly strung between them).
It also affects writing, if you ask for a story with a minmal prompt something very similar comes out of all of them. Often fixated on clocks.
There was a weirdness going on with the first Midjourney models that I miss.
Now the images look like polished, generic things you would see in an airplane magazine, which is what most people want, I suppose (?)
If the prompt is sufficiently short and often repeated, it just makes me wonder - what is cheaper for a provider over a given timespan? Cache a large enough set of responses and randomly return one of them, or generate it each time time?
If I were to design something to accomplish this now, I'd probably attempt to introduce a pipeline that emitted random thematic words from some non-LLM source to feed into the pelican prompt, but even then, I'd expect that some of these models that rely on churning through CoT tokens for extended periods of time would still spend so much time thinking about pelicans wearing scarfs or where the sun would be that it would risk undoing some of my randomization pipeline.
I once tried something similar on an older model of Claude, telling it to "contemplate" a random word before answering my prompts, but at times it seemed to trigger the Claude model's suspicion that there was a jailbreak attempt when I would inject things like "Contemplate sea otters for one sentence before answering"
(I didn't read the linked LLM creativity paper to see if it covers any of this yet)
The curious thing is that trying to prompt it by steering towards some specific goal indirectly gives much worse results than just stating the goal itself.
The Hyères photo has the bike going in the opposite direction and from above. It's also tricky because it's a focal point for the eye, but not the center of the photo. The other elements of the frame direct your eye, but that visual behavior is not obvious to an AI. Lastly the image must capture the unquantifiable "decisive moment" that made Cartier-Bresson so famous.
So far I haven't seen an AI that really gets this prompt. Adding specificity will improve the results to a degree, but I haven't seen an AI that really gets what makes a Cartier-Bresson photo.
1 - https://en.wikipedia.org/wiki/Hy%C3%A8res,_France
> But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic
I believe the claim is that without a detailed specification, you see very similar outcomes. Every pelican is going left to right, every picture has similar sky background, every picture seems to be drawn at the same time of the day (position of the sun), every picture has a few clouds... I don't have data on this, but I imagine the same question being asked to a group of people would result in more varied outcomes.
“Generate an SVG of a pelican riding a bicycle. Be creative”
And it replied:
“I’ll draw a seaside pelican on a vintage bicycle, with a fish-filled basket and a scarf catching the breeze.”
So it came up with exactly the same fish basket and seaside idea. No lighthouse though. Maybe these ideas are just cliche though.
same goes for pelicans. the canonical pelican riding a bicycle is riding on a sunny day on an asphalt road going through some lawns or praires, left to right.
but ask it to tell a joke about something specific, turns out it knows five jokes about those specific things.
you could say the LLMs are quite aware of the zeitgeist they're in, by construction (their RL frying automated feedback loops burn the zeitgeist right into the weights.)
True. You see a lot of the same criticisms when someone asks for a one-shot game or 3D demo and the model responds with something bland and generic. That's exactly what I prefer, it's my job to specify the details. Extrapolation may be the ultimate sign of intelligence, and it might be needed when using AI to tackle open math problems where humans have failed, but it's not usually what I want if I'm doing anything creative or artistic.
Argon is pretty good at getting the distinction right from what I've seen. Ask for something generic, you get something generically functional without a lot of ornamentation. Follow up with requests for specific enhancements, and the model complies effectively and tastefully. If anything, it overcooks the pelicans when given the standard open-ended prompt.
I don’t want creative code. I want boring code following generally agreed upon best practices.
If you need more creativity it can come from the user or the harness.
>If you need more creativity it can come from the user or the harness.
Not fully possible. You can randomize the prompt all you want but you'll always get repetitive patterns and same choices in the same generation. This failure mode makes the model learn a very rigid mapping between this input idea and that single output idea (at best! at worst it's multiple input ideas -> single output idea).
If I ask for a pelican riding a bicycle, I am expecting a relatively boring image of a pelican riding a bicycle, not a pelican flying through a thunderstorm while on a bicycle. That's just straight up not what I asked for. I didn't ask it to be creative, when I imagine myself riding a bicycle, I'm not flying! If I added "be creative and unique, you can add or change anything as long as there is still a pelican on a bicycle", then sure.
Suppose I want an SVG of a laptop. It should probably look like a Macbook Air or something, neutral colors. Easily recognizable as a laptop. If it chose to be creative (unprompted) and gave me a futuristic or retro laptop, or one that's super wacky, there's a high chance of it looking bad, and a high chance I say "no that's not what i wanted". So during the training step, it probably learns to just follow instructions and be less creative unless asked.
Sounds to me like LLMs are just getting better at following instructions and doing what they're told. Simple prompt, simple output.
But step back, someone did ask it, perhaps that person is you. That person has reasons. Whatever those reasons are should be reflected in the image. Of course those reasons need to expressed in the prompt, but the bug here is the prompt doesn't give the details. If you can't figure out details (beyond "be creative") then give up. If I'm illustrating a Harry Potter book and just want to show far out magic things that better should a very different style from illustrating a science fiction novel where pelicans have been genetically engineers such that they now do ride bicycles - likely the anatomy changes mean they can't fly...
This is important. I recently asked AI to take an outline and turn it into slides. I got some good slides out of the outline except that once in a while it decided to generate an image which looked cool, had the correct corporate colors - but otherwise had nothing to do with either any product we make or my presentation. I deleted the images but maybe I should have refined them to something meaningful?
I was thinking just today about how, if I could hire my competition's lead designer/programmers, have them design a system for me cleanroom, that I'd effectively know everything there is to know about my competition's limitations. This is trivial to do with LLMs.
"Make me a thing that does X" is going to come out implemented basically the same way most of the time, as a kind of stack-overflow, well commented, internship project. (or, if undirected enough, as a collection of those based on a bog-standard architecture)
Same-y enough that you could train an adversary on it? Or hack it afterward? I am beginning to think so!
So perhaps the first part is how the GP leans the limitations of the competitor, by seeing what they overdo in the second system? The GP never said they’d get - nor that they wanted - a functioning program?
However, quite how the LLMs clairvoyantly know the trauma of the competitors architecture is beyond me :)
This is a feature, not a bug.
Much of that is a pre-AI bias: A majority of bike-pictures are from an angle which reveals the bike's gears to the audience, which are usually on the rider's right-hand side.
https://www.youtube.com/watch?v=Ys8-a0yD-MM
https://pubmed.ncbi.nlm.nih.gov/26561966/
https://pubmed.ncbi.nlm.nih.gov/1509704/
https://www.newneuromarketing.com/right-to-left-isn-t-right-...
We’ve been trying to tell this on every pelican thread. All models were recreating the same composition. The sun, the road, the green, the scarf, going right. Even the style. They’re not obvious associations one would make given the prompt.
These look surprisingly similar and not necessarily something that directly follows from the prompt.
"Write an svg with graphical instructions on how to make a pelican riding a bicycle in Origami using any number of A4 paper sheets."
All models struggle a lot. Most current models all get a "basic airplane that really floats" pretty well.
Is every kid a Picasso? No, but there definitely is creative diversity among these kids who all have relatively similar shared backgrounds (neighborhood, social class, religion, etc)
It's the reason that if you ask a random child to draw a house, you'll end up with the classic “house with a pitched roof”: a square plus a triangular roof, one door, two windows, and sometimes a chimney.
After all if you ask 100 people to draw a horse, it is quite likely it will be a lot more similar to a "here's a stock horse photograph" than a "here's what picasso would draw when asked to draw a horse"
Why does an SVG pelican imply that it should look polygonal?
I ran into this issue recently when asking Claude to design a logo for my van. It made a polygonal SVG. I asked it to make it more photorealistic and it looked way better, still an SVG though.
https://www.oneusefulthing.org/p/the-overhang
The reason why LLMs cannot be purely* pre-trained on LLM outputs is because LLM output diversity is low.
This implies two things:
1) There is going be no functional difference between hypothetical Qwen 4.5 and Fable 5.5
2) LLMs are not AGI
- https://arxiv.org/abs/2609.37891
- https://arxiv.org/pdf/2508.10975
- https://www.datologyai.com/blog/eating-the-rainbow-a-primer-...
As well as recent model releases that used large fractions of synthetic data (Arcee Trinity):
- https://arxiv.org/pdf/2602.17004v1
It's mostly a matter of increasingly complex and sophisticated sampling methodologies to sift for the valuable synthetic tokens.
There have also been some interesting results about models that use the same data converging to very similar behavior, but I can't find the specific reference at the moment. That one tracks intuitively with the notion that the model is building a latent representation of everything it sees.
Upon a quick glance of the data sections of those papers, every data set you have there is human data augmented by LLMs. Also not to mention the implicit data curation and generation by humans.
Can a model be trained purely on LLM-Outputs and access to tools such as a Turing complete environment?
No one paraphrased the laws of gravitation and handed it to Newton...
There are some things that are true (I hope), so using those concepts, approximated by a human text filter, as seed seems reasonable. Hopefully the diversity in sampling methodologies and opinions about what makes a meaty token pays some dividends down the line for model diversity.
And with ever-more LLM usage both in labs and the real world, the task of finding new/“creative” out-of-average-distribution material will only continue to increase in difficulty. Meaning if this continues, in the future, all output becomes the equivalent of the color brown?
If I'm asking simply and directly for a pelican riding a bicycle, then please honor my intent and gimme that pelican. I don't need it on the moon. I do not need a top hat. I just want what I asked for.
Fulfilling a simple request simply seems like a feature, not a bug.
"Make me a PB&J" should result in what a reasonable person would consider a peanut butter and jelly sandwich. I don't want to interact with an unpredictable system that "thought it would be fun" to deep-fry it.
"Well, you didn't say NOT to deep-fry it!"
The pelican from the post can be changed by prompting, as other commenters show. But I found it pretty hard to guide Opus away from the above.
Other models' results look different, but also show style convergence.
You can also see in "creative" game output (Astra loves 3d), or text output.
It's not all bad - this shows we should treat AI as a tool and assume any creativity (or lack of it) as coming from the human.
This nicely redefines slop as work you slapped together without much care, irrespective of what the tool was.
tho who know to refine, say NO. & do more with less even in the age of too cheap to meter tokens.
that has always been the work that produced every delightful product.
One of the reasons image AI slop has been so prevalent is that the slop is the base case (as derived from tons of RLHF because the median user has no taste) and it is becoming increasingly difficult to steer those models even when the prompt is more specified.
1) Chollet stated that RL:Pre-training is 9:1.
2) Google said argon was trained on SVGs
3) Wang said that Muse Wonder was not trained on 3D models
[0] https://news.ycombinator.com/item?id=49526966
Perhaps the prompt should include what the scene should contain.
Lost count of the number of times a requirement is given in terms of how another product is doing it. "make chips like gmail uses for tags", "let's copy anthropics LLM chat UI", "we should try and make it look like X competitor".
"Looking the same as everyone else" seems to be a bonus in their mind
People weren't happy with boring answers, so they added randomness to the output. The randomness led to hallucination, so they made it less random. Now you want it to give you a unique artistic impression for every prompt? It's a spell checker, not an art student.
For example, a lot has been written about how all corporate logos look the same now, https://news.ycombinator.com/item?id=32040506 , and also about how so much interior design and architecture has converged to the same look. In the 70s there may have been tons of styles people derided as ugly, but at least there was colorful diversity. Now so many higher end (and "aspirational high end") interior design has the same minimalist, gray/white/tan, "look you can't see my refrigerator or dishwasher" look. I was recently looking for an apartment and all of the "luxury" apartments I looked at (obviously take that adjective with a heap of salt) had nearly the exact same layout and style.
You'd probably need to ask thousands of people (or some children) before you heard "a pelican on the moon" or "a pelican in neo-tokyo"
I don't see why you couldn't do the same in real projects.
If the design looks too generic, you can just ask for a change, no?
You have to be in the loop to inject taste at some point. This typically requires a lot of testing and feedback.
I don't know if AI will ever develop taste, or if it even should. We have enough human creativity and creating is fun.
I don't think people care much for what they say.
Now within our GUI/TUI to the llm (which we all use now) could just add a couple of extra steps to simulate that diversity.
Are we making an image? If yes, and is this the first prompt in the convo? neat! List 5 elements that would make up this image. Now for each of those 5 elements, make a list of 10 possible choices. Now randomly select from each one. Now include those elements on top of the original prompt.
I got claude to generate 10 of these as a prompt transformation, and i think they would be suitably random to postpone the diversity collapse for a while.
I'm not pretending the problem goes away, but these kinds of creativity varying steps I think will become hidden steps in our LLM systems very soon.
The same thing happens when doing any group work in a classroom or workplace. One of the facilitators job is to add in randomness and guides to remind people to make different choices rather than the obvious ones.