There’s a fun variation in W-Europe that google needs to spend some time on:
Northern Belgium and the Netherlands have web content in the same language. But google uses the content in one lump. Problem is when you search for employment/fiscal/legal/… you constantly get content that applies to the wrong nationality.
If you are building your own harness that does correct citations, is the correct thing to give AI access to some deterministic tool that allows is to actually deterministically copy paste parts of documents its reading (with links), rather than stochastic copy paste that they do by default?
I noticed this personally. Saw a citation with a preview for source A, which I knew was reliable. Checked, and it referenced a Reddit article and various other less reliable sources. Was a direct citation too that actually wasn't.
This is why lawyers have been getting in trouble using AI to review case law or (worse) to generate documents.
It creates citations and references that look close enough to be plausible but are just made up of thin air. CA passed a law explicitly requiring lawyers to review AI-generated documents that is now before the governor for signing (previously, lawyers were ethically expected to review documents submitted to the court or provided to clients but that doesn't have the same level of force as an explicit requirement).
What's surprising about this is that you can get the bullshit machine to produce correct externally validate citations. It's not particularly hard either—it's one of the first things you build when you give an LLM access to a body of documents/search. So for a large public service to whiff like this is certainly a stain on their credibility.
On the other hands it's a boost to their credibility that they make their mistakes easier to evaluate than their competition does. It would be worse if they had a similar error rate without openly providing references. Kudos to Perplexity for including more empirical attack surface.
Same is true of Google search “summaries” where quite often I click on the link provided and it doesn’t support the statement Google made.
BTW, this post would be more convincing if it wasn’t written in Claude voice itself!
There’s a fun variation in W-Europe that google needs to spend some time on:
Northern Belgium and the Netherlands have web content in the same language. But google uses the content in one lump. Problem is when you search for employment/fiscal/legal/… you constantly get content that applies to the wrong nationality.
If you are building your own harness that does correct citations, is the correct thing to give AI access to some deterministic tool that allows is to actually deterministically copy paste parts of documents its reading (with links), rather than stochastic copy paste that they do by default?
I noticed this personally. Saw a citation with a preview for source A, which I knew was reliable. Checked, and it referenced a Reddit article and various other less reliable sources. Was a direct citation too that actually wasn't.
100% my experience with this service. The intent is good, but it seems they're still in the "fake it until you make it" stage.
>The intent is good, but it seems they're still in the "fake it until you make it" stage
Please notice the internal contradictions here.
I felt that myself. And I have the same problem with gemini
This plus AI just citing AI slop. Theres a real downward spiral unfolding with the quality of information available on the internet.
This is why lawyers have been getting in trouble using AI to review case law or (worse) to generate documents.
It creates citations and references that look close enough to be plausible but are just made up of thin air. CA passed a law explicitly requiring lawyers to review AI-generated documents that is now before the governor for signing (previously, lawyers were ethically expected to review documents submitted to the court or provided to clients but that doesn't have the same level of force as an explicit requirement).
Even worse than lawyers is the government doing it: https://newrepublic.com/post/215001/judge-rfk-jr-hhs-fake-ai...
BS machine produces BS; in other news water is wet and sky is blue.
What's surprising about this is that you can get the bullshit machine to produce correct externally validate citations. It's not particularly hard either—it's one of the first things you build when you give an LLM access to a body of documents/search. So for a large public service to whiff like this is certainly a stain on their credibility.
On the other hands it's a boost to their credibility that they make their mistakes easier to evaluate than their competition does. It would be worse if they had a similar error rate without openly providing references. Kudos to Perplexity for including more empirical attack surface.
> you can get the bullshit machine to produce correct externally validate citations
How?