If I recall correctly, there were some papers which suggested that LLMs favor LLM-generated passages over human written ones. I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. It always picks its own :) I've also experienced that both Claude and Codex routinely include generated websites when I ask them to search for something. It also doesn't help that the web search tools that OAI and Anthropic have are deeply limiting: can't exclude keywords or domains.
The other day, I remember an article was posted to HN about something, but it came from a company that provides SEO services to companies by doing something like this:
1. For a given company, analyze their target audiences and the questions they are likely to ask LLMs about.
2. For each such question, ask it to each of the major LLMs, and compute the KL divergence between the pages they want to rank for the question vs. the LLM's response.
3. Rewrite the article to minimize said KL divergence.
In effect, they're performing an iterative optimization of some sort that moves the embedding space of their article closer to the question asked to the LLM, and any embedding model or generated responses are going to prefer said responses over others.
There is a term for the general practice of optimizing responses called GEO - Generative Engine Optimization [0]. A cousin of SEO and equally unsavory in how trust is being eroded through info shaping. Self-discovery by individuals is the victim.
And it's all because of ads. The incentives in an ad-funded internet are just always going to lead to this sort of thing. The most important thing is getting the user to load your page, not actually satisfying their query.
Let's hope the LLM model continues to be paying for credits, because any that move to ad revenue will become useless for real work.
> I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful.
Interesting. For me I've noticed it tends to do the opposite.
> asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored [...] It always picks its own
...is not the same as claiming...
> LLMs favor LLM-generated passages over human written ones
Here, you're using the same LLM to both produce and judge the resulting work. If anything, I would expect an LLM to tend to prefer its own work given that the same training is producing and judging.
It's not intuitive to me for why preference for its own writing would emerge, and during what type of training or tuning.
Perhaps something like: learning to identify what source files it has worked on by the code style alone, because tasks may give human code (public repos, etc) and ask to make changes.
Well it's not only this, or protection from LLMs training on LLM output. LLMs training on human output is also problematic.
I was traveling to an obscure small town, doing some "research" with LLMs beforehand. Every and each one told me enthusiastically to go to "Foobar square" (name changed) for the "best street food in XYZ town", some added a lot of colorful details.
There was no Foobar square in XYZ town. There was no Foobar square anywhere in the world. There was a SINGLE old Reddit comment, with no upvotes, to a unpopular post in an unpopular subreddit, where someone clearly badly misspelled the name of the square, and said something like "for street food go to Foobar square". Nothing about "the best" even.
I think this was a common game on city/town subs. It happened here, there was a post asking for a good restaurant and someone just made up a name. It went viral and people started posting made-up menus for the place, reviews, and for a couple of months any time someone asked about a restaurant this fictional place would get mentioned.
It was all done as a joke to see if they could get Gemini or ChatGPT to start recommending it.
I used one of the 12-month free Perplexity offers when they were everywhere. It felt slightly useful at first for simple queries where I didn’t want to go through the top 10 Google results manually. If I was looking for a specific recipe I remembered or a help page or user manual it would usually find it quickly.
Then they started optimizing for speed of responses over quality of results. I can enter a query and see my results appear in a second, but they’re garbage. The links and references it gives frequently don’t match the text right next to them. It feels like someone had a KPI to make responses as fast as possible and they optimized for that above all else.
They added a “Computer” option that’s supposed to do research for you. Half the time I can’t get it to trigger through the UI. Pressing the submit button doesn’t work. When I can get it to trigger, most of those sessions will work for a while and then just stop before an answer comes back.
The only reason I keep using it is to keep observing a company that has been heavily marketed and hyped, which should have had a market leading position for something. Even non-technical people I know who listen to Joe Rogan (where Perlexity is advertising heavily, I’m told) are asking me about it.
Now there are reports of people being billed at the end of their trial period without warning, despite them saying that they will warn before this happens. There are some alarmingly bad customer support screenshots where the customer support agent (AI? Probably) acknowledges that they didn’t send the email they promised but refuse to help anyway. It takes escalating it on Twitter to get it corrected.
If I want to do actual research or AI assisted web searching I have Claude or ChatGPT do it. The results are so much higher quality and it does exactly what I ask. It may take 45 seconds instead of the instant response from Perplexity but I save time overall because the response and links are more likely to be correct
I paid for perplexity pro for 3 years. I genuinely enjoyed using it and felt it was better overall than ChatGPT etc due to the way it showed sources etc. I liked being able to use different models depending on what I was looking for, and the deep research was helpful.
I think they probably damaged themselves by going for a land grab of user base through freebies. It meant the users weren’t ever going to convert to paid customers, so it was more to show investors that they had a user base. But, with an increased base of users who weren’t paying, it then meant they needed to find either new revenue streams or cheaper ways to provide the service. Unfortunately, it seems they went with the new revenue streams whilst also decreasing the functions paying members were able to access (something I find quite abhorrent- I paid a service level, but then they change what I receive mid-subscription). And then computer - rammed down my throat. One reason I pay for pro is to stop the nagging noise of paid tiers. And instead, they actually created a way of logging in and continually seeing gated functions.
So, after paying them upwards of $400-$500 and being a loyal customer, I walked.
I’ve personally found that Google’s AI mode is surprisingly capable and almost absurdly fast, although the hallucination rate is somewhat proportionally higher too to match its seeming over-responsiveness. Which means Perplexity in effect doesn’t have anything to differentiate it.
If they had some sort of tier that was like $5 to $10 and the main thing it had on it was their custom model for research, no other model, I might re-subscribe, but they were definitely burning way too much compute trying to give it away in the hopes others kept their subscription. I was using it to trim down on my direct Claude Compute usage since they ran me unmetered for a while.
I would draft a development plan with Claude on there, then feed it to Claude Code. This isn't sustainable, but given that I had x number of months pre-paid for, I just used it.
Well, was. They did a bad job of it these last few years which allowed any AI that could crawl the web to seem amazing for search because it could pick the best posts from Reddit or whatever other forum had the best context for your question, but now we're watching the AI snake eat its own tail.
Theoretically, Goggle doesn't care which websites run their ads, so they might as well give you the most useful ones. Search doesn't really work for engagementmaxxing.
If you send people to the optimal website containing exactly the information they are after, then you get fewer ad impressions than if you send them to a suboptimal website that has them going back and clicking on more links.
I rarely use image search, but I went to look something up recently and I was shocked at how many obviously AI generated images showed up. I couldn't even find an image of the thing I was looking for and eventually gave up. Bing has the same problem.
There's also the irony that this AI written piece criticizes how low the domains are on the tranco list when trellner.com doesn't even make the list, haha.
Agreed, I thought the subject matter was interesting enough to try and labour through the tedious prose, but once I got to "Their scale is the point." I just had to stop and just skim the rest.
Pretty sure that OP ("jakobgreenfeld") is the "founder". That user's last four submissions have all been similar "finding" reports from a Claude-generated mystery research group website.
I've been vary of using ai to search considering all the spam out there. I think I'd rather, perhaps naively, whitelist wikipedia, reddit, arxiv, some news sources, etc than include everything.
Is there nothing out there that does this? I'm paying for kagi and I can see that it has an api, is that maybe sufficient if configured properly?
If you use Kagi Assistant, you can pick one of your lenses (i.e. lists of domains to restrict searches to) in chats. Not sure if their API has that as well or some other way to restrict searches. Also not sure if the Assistant (or API) respects blocked domains when searching.
What protection do LLM search engines have against training off content generated by other LLMs?
Will we get to a point where AI-generated sites make up a majority of the internet, and LLMs are training upon their own regurgitations, with exponential amplification of all their lies and flaws?
Or will the pre-2022 corpus human knowledge be considered the low-background steel standard, and anything after that less and less reliable unless certified that it has been created by a human mind and untainted by hallucinations?
I've mostly stopped using the Internet to learn new things and have gone back to books from the library. The majority of technical books at the library were published pre-2020s and hopefully, publishing slop physically won't be profitable enough to flood that market, too. Now that the Internet has largely been destroyed by slop manufacturers, whether or not the words are(/were) worth putting on paper becomes a useful discriminator.
I've seen an extremely aggressive uptick in API key requests and sales that I'm not sure where it's coming from. Like it's up 5x over the summer. Been a bit confused about this since I do basically zero traditional marketing or SEO, but I think it's AI search tools that's suggesting my services.
The irony of this article being fully AI generated...
Anyway, it's over for Perplexity. They never had a great a product and the only reason for using them, was when they offered Pro accounts for free. Many people joined. Me included. But with a "meh" product and the general AI business not being very sticky, they lost quite harshly.
I thought they might be able to make money as a search api/index, but this article closed the book.
Honestly, I will just flag every post that is entirely AI slop from now on. This has to stop.
The home page for this "independent research firm" is also 100% nonsense [1]. "The record a machine reads is not the one a company writes.". Ironically this low-effort spam is exactly what this report warns about, and does not belong in HN - or anywhere else.
Also, that user's last four (three of them in the last hour) submissions have all been similar "finding" reports from a Claude-generated mystery research group website. All which contain exclusively AI slop articles. Ugh.
They've raised $155M total now with the latest at a $1B valuation [1] from Lightspeed, Sequoia, Kleiner Perkins, Khosla, and NVIDIA.
And very impressive list of angels too: Guillermo Rauch (Vercel), Karim Atiyeh (Ramp), Andrew Karam (AppLovin) among others
Last I heard they're trying to reposition from AEO/GEO to "AI Marketer". No clue how that's going, I feel like the AEO/GEO stuff isn't super defensible at that valuation if for no other reason than I assume (hope) the spamming stops working.
SEO companies were already doing this. There were already tools applying ML to the problem before LLMs too, that would recommend places you could post relevant content, like Reddit, yahoo answers (lmao), quora, etc.
If I recall correctly, there were some papers which suggested that LLMs favor LLM-generated passages over human written ones. I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. It always picks its own :) I've also experienced that both Claude and Codex routinely include generated websites when I ask them to search for something. It also doesn't help that the web search tools that OAI and Anthropic have are deeply limiting: can't exclude keywords or domains.
The other day, I remember an article was posted to HN about something, but it came from a company that provides SEO services to companies by doing something like this:
1. For a given company, analyze their target audiences and the questions they are likely to ask LLMs about.
2. For each such question, ask it to each of the major LLMs, and compute the KL divergence between the pages they want to rank for the question vs. the LLM's response.
3. Rewrite the article to minimize said KL divergence.
In effect, they're performing an iterative optimization of some sort that moves the embedding space of their article closer to the question asked to the LLM, and any embedding model or generated responses are going to prefer said responses over others.
I believe we will keep seeing more of this stuff.
There is a term for the general practice of optimizing responses called GEO - Generative Engine Optimization [0]. A cousin of SEO and equally unsavory in how trust is being eroded through info shaping. Self-discovery by individuals is the victim.
0: https://en.wikipedia.org/wiki/Generative_engine_optimization
And it's all because of ads. The incentives in an ad-funded internet are just always going to lead to this sort of thing. The most important thing is getting the user to load your page, not actually satisfying their query.
Let's hope the LLM model continues to be paying for credits, because any that move to ad revenue will become useless for real work.
Well, the one it generated is based on how it thought the best way to solve the problem was.
I am sure most humans would pick code written in their style, too.
If by root urls you mean domains, openai at least supports this.
https://developers.openai.com/api/docs/guides/tools-web-sear...
That is what I meant! Couldn't remember the word 'domain' while I was writing out my comment. Thank you
>It always picks its own
Makes sense to me, in that its own output would align closer to its own training set
> I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful.
Interesting. For me I've noticed it tends to do the opposite.
> asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored [...] It always picks its own
...is not the same as claiming...
> LLMs favor LLM-generated passages over human written ones
Here, you're using the same LLM to both produce and judge the resulting work. If anything, I would expect an LLM to tend to prefer its own work given that the same training is producing and judging.
It's not intuitive to me for why preference for its own writing would emerge, and during what type of training or tuning.
Perhaps something like: learning to identify what source files it has worked on by the code style alone, because tasks may give human code (public repos, etc) and ask to make changes.
Well it's not only this, or protection from LLMs training on LLM output. LLMs training on human output is also problematic.
I was traveling to an obscure small town, doing some "research" with LLMs beforehand. Every and each one told me enthusiastically to go to "Foobar square" (name changed) for the "best street food in XYZ town", some added a lot of colorful details.
There was no Foobar square in XYZ town. There was no Foobar square anywhere in the world. There was a SINGLE old Reddit comment, with no upvotes, to a unpopular post in an unpopular subreddit, where someone clearly badly misspelled the name of the square, and said something like "for street food go to Foobar square". Nothing about "the best" even.
It's all a lie.
I think this was a common game on city/town subs. It happened here, there was a post asking for a good restaurant and someone just made up a name. It went viral and people started posting made-up menus for the place, reviews, and for a couple of months any time someone asked about a restaurant this fictional place would get mentioned.
It was all done as a joke to see if they could get Gemini or ChatGPT to start recommending it.
I used one of the 12-month free Perplexity offers when they were everywhere. It felt slightly useful at first for simple queries where I didn’t want to go through the top 10 Google results manually. If I was looking for a specific recipe I remembered or a help page or user manual it would usually find it quickly.
Then they started optimizing for speed of responses over quality of results. I can enter a query and see my results appear in a second, but they’re garbage. The links and references it gives frequently don’t match the text right next to them. It feels like someone had a KPI to make responses as fast as possible and they optimized for that above all else.
They added a “Computer” option that’s supposed to do research for you. Half the time I can’t get it to trigger through the UI. Pressing the submit button doesn’t work. When I can get it to trigger, most of those sessions will work for a while and then just stop before an answer comes back.
The only reason I keep using it is to keep observing a company that has been heavily marketed and hyped, which should have had a market leading position for something. Even non-technical people I know who listen to Joe Rogan (where Perlexity is advertising heavily, I’m told) are asking me about it.
Now there are reports of people being billed at the end of their trial period without warning, despite them saying that they will warn before this happens. There are some alarmingly bad customer support screenshots where the customer support agent (AI? Probably) acknowledges that they didn’t send the email they promised but refuse to help anyway. It takes escalating it on Twitter to get it corrected.
If I want to do actual research or AI assisted web searching I have Claude or ChatGPT do it. The results are so much higher quality and it does exactly what I ask. It may take 45 seconds instead of the instant response from Perplexity but I save time overall because the response and links are more likely to be correct
I paid for perplexity pro for 3 years. I genuinely enjoyed using it and felt it was better overall than ChatGPT etc due to the way it showed sources etc. I liked being able to use different models depending on what I was looking for, and the deep research was helpful.
I think they probably damaged themselves by going for a land grab of user base through freebies. It meant the users weren’t ever going to convert to paid customers, so it was more to show investors that they had a user base. But, with an increased base of users who weren’t paying, it then meant they needed to find either new revenue streams or cheaper ways to provide the service. Unfortunately, it seems they went with the new revenue streams whilst also decreasing the functions paying members were able to access (something I find quite abhorrent- I paid a service level, but then they change what I receive mid-subscription). And then computer - rammed down my throat. One reason I pay for pro is to stop the nagging noise of paid tiers. And instead, they actually created a way of logging in and continually seeing gated functions.
So, after paying them upwards of $400-$500 and being a loyal customer, I walked.
I’ve personally found that Google’s AI mode is surprisingly capable and almost absurdly fast, although the hallucination rate is somewhat proportionally higher too to match its seeming over-responsiveness. Which means Perplexity in effect doesn’t have anything to differentiate it.
If they had some sort of tier that was like $5 to $10 and the main thing it had on it was their custom model for research, no other model, I might re-subscribe, but they were definitely burning way too much compute trying to give it away in the hopes others kept their subscription. I was using it to trim down on my direct Claude Compute usage since they ran me unmetered for a while.
I would draft a development plan with Claude on there, then feed it to Claude Code. This isn't sustainable, but given that I had x number of months pre-paid for, I just used it.
Perplexity is about to learn that Google is an anti-spam company first, search engine second
Well, was. They did a bad job of it these last few years which allowed any AI that could crawl the web to seem amazing for search because it could pick the best posts from Reddit or whatever other forum had the best context for your question, but now we're watching the AI snake eat its own tail.
Hard to conclusively beat spam when your primary means of making money is selling the very ads that the spammers are using to make money.
Theoretically, Goggle doesn't care which websites run their ads, so they might as well give you the most useful ones. Search doesn't really work for engagementmaxxing.
If you send people to the optimal website containing exactly the information they are after, then you get fewer ad impressions than if you send them to a suboptimal website that has them going back and clicking on more links.
I rarely use image search, but I went to look something up recently and I was shocked at how many obviously AI generated images showed up. I couldn't even find an image of the thing I was looking for and eventually gave up. Bing has the same problem.
It's difficult to read more than a few sentences, when this itself is clearly a Claude artifact.
There's also the irony that this AI written piece criticizes how low the domains are on the tranco list when trellner.com doesn't even make the list, haha.
Agreed, I thought the subject matter was interesting enough to try and labour through the tedious prose, but once I got to "Their scale is the point." I just had to stop and just skim the rest.
Indeed. A google/brave search on that founder generated nada.
Pretty sure that OP ("jakobgreenfeld") is the "founder". That user's last four submissions have all been similar "finding" reports from a Claude-generated mystery research group website.
> The result covers Perplexity only. We have not measured ChatGPT, Gemini, Copilot or Google’s AI Mode
Why only test Perplexity...? Isn't it the least popular among these?
I've been vary of using ai to search considering all the spam out there. I think I'd rather, perhaps naively, whitelist wikipedia, reddit, arxiv, some news sources, etc than include everything.
Is there nothing out there that does this? I'm paying for kagi and I can see that it has an api, is that maybe sufficient if configured properly?
If you use Kagi Assistant, you can pick one of your lenses (i.e. lists of domains to restrict searches to) in chats. Not sure if their API has that as well or some other way to restrict searches. Also not sure if the Assistant (or API) respects blocked domains when searching.
Reddit is full of ai accounts now tho
What protection do LLM search engines have against training off content generated by other LLMs?
Will we get to a point where AI-generated sites make up a majority of the internet, and LLMs are training upon their own regurgitations, with exponential amplification of all their lies and flaws?
Or will the pre-2022 corpus human knowledge be considered the low-background steel standard, and anything after that less and less reliable unless certified that it has been created by a human mind and untainted by hallucinations?
I have a feeling we're already there.
The weird babbling reported from Opus 5 might be a result of either a bad system prompt or bad training data.
They’ll train on prompts and anything else you send in. Many LLM responses are sorta finger printable: I assume this is intentional
> What protection do LLM search engines have against training off content generated by other LLMs?
You're talking about a scenario that won't blow itself up in the next few quarters, so it's of no interest to them.
I've mostly stopped using the Internet to learn new things and have gone back to books from the library. The majority of technical books at the library were published pre-2020s and hopefully, publishing slop physically won't be profitable enough to flood that market, too. Now that the Internet has largely been destroyed by slop manufacturers, whether or not the words are(/were) worth putting on paper becomes a useful discriminator.
Begun, the AI SEO wars have.
It does seem rather impactful.
I've seen an extremely aggressive uptick in API key requests and sales that I'm not sure where it's coming from. Like it's up 5x over the summer. Been a bit confused about this since I do basically zero traditional marketing or SEO, but I think it's AI search tools that's suggesting my services.
Has the industry already started to search for a better name than AI SEO for this?
Yes GEO (Generative Engine Optimisation) is one I've seen around.
GEO seems to be winning, Generative Engine Optimization.
Answer engine optimization (AEO)
"Sharing a nameserver pair is strong circumstantial evidence of a common Cloudflare account rather than proof of ownership"
-when you read one statement that let's you know to believe no other assertions in the article....
The irony of this article being fully AI generated...
Anyway, it's over for Perplexity. They never had a great a product and the only reason for using them, was when they offered Pro accounts for free. Many people joined. Me included. But with a "meh" product and the general AI business not being very sticky, they lost quite harshly.
I thought they might be able to make money as a search api/index, but this article closed the book.
Searching for products has become impossible. If you don't already know what you are looking for, you're screwed.
Makes sense. Manipulating training data so that models will recommend your product is undoubtedly a big industry.
Honestly, I will just flag every post that is entirely AI slop from now on. This has to stop.
The home page for this "independent research firm" is also 100% nonsense [1]. "The record a machine reads is not the one a company writes.". Ironically this low-effort spam is exactly what this report warns about, and does not belong in HN - or anywhere else.
[1] https://trellner.com/
Also, that user's last four (three of them in the last hour) submissions have all been similar "finding" reports from a Claude-generated mystery research group website. All which contain exclusively AI slop articles. Ugh.
An SEO tale as old as time
Who could have seen this coming?
my friend works for a company called 'profound' whose whole job is 'get found by ai' by spamming reddit and other talk sites ( among other things)
They've raised $155M total now with the latest at a $1B valuation [1] from Lightspeed, Sequoia, Kleiner Perkins, Khosla, and NVIDIA.
And very impressive list of angels too: Guillermo Rauch (Vercel), Karim Atiyeh (Ramp), Andrew Karam (AppLovin) among others
Last I heard they're trying to reposition from AEO/GEO to "AI Marketer". No clue how that's going, I feel like the AEO/GEO stuff isn't super defensible at that valuation if for no other reason than I assume (hope) the spamming stops working.
[1] https://dealroom.co/news/126181-profound-raises-96m-at-1b-va...
SEO companies were already doing this. There were already tools applying ML to the problem before LLMs too, that would recommend places you could post relevant content, like Reddit, yahoo answers (lmao), quora, etc.
AI;DR
Yea. A google on that founder generated zero results.