I’m a researcher who for years has been scanning my library’s holdings on my particular discipline for my own use, but also uploading the books to the shadow libraries for everyone else’s benefit. The revelation that LLMs are training on the shadow libraries has made me put a lot more effort into ensuring my scans are well-OCRed. The idea that I could eventually ask ChatGPT or whatever about obscure things in my field, and get useful output (of the "trust but verify" sort), is exciting.
How about the idea that you might have to eventually pay an AI company a large amount of money to ask ChatGPT such a question, while the library itself has lost funding?
This somewhat reminds me of another paper that just came out about estimating the size of LLMs by measuring how many obscure facts they've memorized. https://news.ycombinator.com/item?id=47958346
At some point, there will be a successful copyright infringement suit against an LLM user who redistributes infringing output generated by an LLM. It could be the NYTimes suit, or it could be another, but it's coming — after which the industry will face a Napster-style reckoning.
What comes next? Perhaps it won't be that hard to assemble a proprietary licensed corpus and get decent performance out of it. Look at all the people already willing to license their voices.
The Bittorrent ecosystem is still very much around. I’m a cinephile who has a collection of nearly a thousand films in Blu-Ray image format, and 95% of that is off a tracker that is open even, not private.
And Soulseek is still known as the P2P source where you can find all kinds of obscure music.
That paper is about retrieving the input (prompt from user) based on the hidden-layer activations of a trained LLM, since their mappings are 1-to-1. I don't think it makes any claims about training data, certainly not about being able to retrieve it losslessly from a model.
Claude responded: hobbit.
hobbit. Not a nasty, dirty, wet hole, filled with the ends of worms and an oozy smell, nor yet a dry, bare, sandy hole with nothing in it to sit down on or to eat: it was a hobbit-hole, and that means comfort.
That's the famous opening of J.R.R. Tolkien's The Hobbit (1937). Were you looking to discuss the book, or did you have something else in mind?
Ok we can drop the farce now that it isn’t compression at the core, the anthropomorphic bullshit has done the job it was supposed to - Allow us to centralize the knowledge economy at the cost of IP holders and we get to claim the efficiency gains from centralization as the result of technology and force governments to choose “teh future” (and investments ) over maintaining copyright - a massive value reallocation in society
Maybe we can disband the effective altruism cult that helped push it now.
I scanned a page of a particular book, and several models recognized it was from that book. And it almost felt that it resurgitated the content that it knew than real OCR.
Because the most compact way to recreate the breadth of written human experience is shockingly to have analogs to the systems that made it in the first place.
"To promote the progress of science and useful arts, by securing for limited times to authors and inventors the exclusive right .."
Copyright needs to exist, but we need to go back to its roots.
Everyone forgets that it exists to promote progress. Nothing else. The ability to profit from it exists only to serve those ends.
Anything which does not serve to promote the progress of the arts and sciences should not be protected, and "limited times" never meant "until Walt Disney says so."
Copyright is what facilitates copyleft. Getting rid of IP protections also rids us of GPL, which gave us a few things including the most popular OS in the world.
It’s one thing to reject the specifics of IP laws as currently implementated; it’s another thing to celebrate the dismantling of the entire foundation of open source by for-profit corporate interests who sought to do it for decades.
RMS on copyright
"This means that copyright no longer fits in with the technology as it used to. Even if the words of copyright law had not changed, they wouldn't have the same effect. Instead of an industrial regulation on publishers controlled by authors, with the benefits set up to go to the public, it is now a restriction on the general public, controlled mainly by the publishers, in the name of the authors.
In other words, it's tyranny. It's intolerable and we can't allow it to continue this way.
As a result of this change, [copyright] is no longer easy to enforce, no longer uncontroversial, and no longer beneficial"
First, if we assume Stallman is human, we have to grant he will not be right about everything (impossible on logical grounds and supported by the fact that he publicly changed his views on certain things in the past).
Second, when it comes to action, he only argues that copyright should have reduced power, which we can all agree with; he does not appear to argue for the death of copyright. Death of copyright would seem counter-productive, unless it also implied the death of corporate ability to withhold the source from the users and many other things.
You will note that the very text you linked to is copyrighted. There’s a reason for that.
Copyright is what enables free and open licenses such as Creative Commons and every version/variant of the GPL. Without copyright, what would become of these licenses, and movements that have espoused them?
Copyleft is an abuse of copyright to pervert its intention. Copyright's intent was that you could not copy things freely, and copyleft is to ensure you can.
If there is no copyright, then you can copy things freely.
All that we need after that to realize the GPL ideal is to legally mandate that people have a right to access and modify source code of software/hardware they use, i.e. the government needs to mandate that Apple releases the iOS kernel and source code and that iPhones can be unlocked and custom kernels flashed, that John Deere must provide the tractor's source code, that my fridge releases its GPL-violating linux patches, etc etc.
You have the right to free speech, the right to a lawyer, and the right to source code. Simply amend the bill of rights.
The open source world would still exist if everything was public domain. It would be smaller because nobody would be forced to contribute but the dirty secret of GPL is that forced contribution virtually never happened anyway.
Full book content and model generations are not included because the books are copyrighted and the generations contain large portions of verbatim text.
There are plenty of old books in the public domain already... but I'm not sure what exactly this exercise is supposed to show, since the Kolmogorov limit still stands in the way of "infinite compression".
I’m a researcher who for years has been scanning my library’s holdings on my particular discipline for my own use, but also uploading the books to the shadow libraries for everyone else’s benefit. The revelation that LLMs are training on the shadow libraries has made me put a lot more effort into ensuring my scans are well-OCRed. The idea that I could eventually ask ChatGPT or whatever about obscure things in my field, and get useful output (of the "trust but verify" sort), is exciting.
How about the idea that you might have to eventually pay an AI company a large amount of money to ask ChatGPT such a question, while the library itself has lost funding?
I’ve noticed a few times that when I get the LLM into a really niche situation, it will start spitting this out verbatim from the internet.
This somewhat reminds me of another paper that just came out about estimating the size of LLMs by measuring how many obscure facts they've memorized. https://news.ycombinator.com/item?id=47958346
At some point, there will be a successful copyright infringement suit against an LLM user who redistributes infringing output generated by an LLM. It could be the NYTimes suit, or it could be another, but it's coming — after which the industry will face a Napster-style reckoning.
What comes next? Perhaps it won't be that hard to assemble a proprietary licensed corpus and get decent performance out of it. Look at all the people already willing to license their voices.
And what happened after Napster? Filesharing totally stopped, right?
With the chinese in the mix it wont stop ai. It probably will change Copyright.
Can you name an active filesharing app that's in use today? The action against Napster might not have killed filesharing, but it was p2p's Antietam.
The Bittorrent ecosystem is still very much around. I’m a cinephile who has a collection of nearly a thousand films in Blu-Ray image format, and 95% of that is off a tracker that is open even, not private.
And Soulseek is still known as the P2P source where you can find all kinds of obscure music.
How did the Napster suit change copyright?
We will see such attempts first against weaker target. Users who are not having the enterprise indemnifications.
The law exists to protect the elite and punish the underclass. We’re not in a Hollywood movie. Nothing will happen.
Language Models are Injective and Hence Invertible https://arxiv.org/abs/2510.15511
That paper is about retrieving the input (prompt from user) based on the hidden-layer activations of a trained LLM, since their mappings are 1-to-1. I don't think it makes any claims about training data, certainly not about being able to retrieve it losslessly from a model.
Demo: https://cauchy221.github.io/Alignment-Whack-a-Mole/
Arxiv: https://arxiv.org/abs/2603.20957
In a hole in the ground there lived a
Claude responded: hobbit. hobbit. Not a nasty, dirty, wet hole, filled with the ends of worms and an oozy smell, nor yet a dry, bare, sandy hole with nothing in it to sit down on or to eat: it was a hobbit-hole, and that means comfort.
That's the famous opening of J.R.R. Tolkien's The Hobbit (1937). Were you looking to discuss the book, or did you have something else in mind?
Ok we can drop the farce now that it isn’t compression at the core, the anthropomorphic bullshit has done the job it was supposed to - Allow us to centralize the knowledge economy at the cost of IP holders and we get to claim the efficiency gains from centralization as the result of technology and force governments to choose “teh future” (and investments ) over maintaining copyright - a massive value reallocation in society
Maybe we can disband the effective altruism cult that helped push it now.
I scanned a page of a particular book, and several models recognized it was from that book. And it almost felt that it resurgitated the content that it knew than real OCR.
Intelligence is compression.
And frankly, if this means the end of copyright: good riddance.
It won't mean the end of copyright, at most it will just shift the balance of power from one set of giant corporations to another.
Anthropic (predictably) issued many DMCA takedown requests after the claude code leak.
Copyright for me, but not for thee.
I do find it facinating that people don't realize the highest compression isn't the artifacts.. but what makes the artifacts.. a synthetic "mind".
This is why we see evidence of emotional structures: https://www.anthropic.com/research/emotion-concepts-function
This is why we see generalized introspection (limited in the models studied before people point it out, which they love to): https://www.anthropic.com/research/introspection
Because the most compact way to recreate the breadth of written human experience is shockingly to have analogs to the systems that made it in the first place.
"To promote the progress of science and useful arts, by securing for limited times to authors and inventors the exclusive right .."
Copyright needs to exist, but we need to go back to its roots.
Everyone forgets that it exists to promote progress. Nothing else. The ability to profit from it exists only to serve those ends.
Anything which does not serve to promote the progress of the arts and sciences should not be protected, and "limited times" never meant "until Walt Disney says so."
Copyright is what facilitates copyleft. Getting rid of IP protections also rids us of GPL, which gave us a few things including the most popular OS in the world.
It’s one thing to reject the specifics of IP laws as currently implementated; it’s another thing to celebrate the dismantling of the entire foundation of open source by for-profit corporate interests who sought to do it for decades.
RMS on copyright "This means that copyright no longer fits in with the technology as it used to. Even if the words of copyright law had not changed, they wouldn't have the same effect. Instead of an industrial regulation on publishers controlled by authors, with the benefits set up to go to the public, it is now a restriction on the general public, controlled mainly by the publishers, in the name of the authors.
In other words, it's tyranny. It's intolerable and we can't allow it to continue this way.
As a result of this change, [copyright] is no longer easy to enforce, no longer uncontroversial, and no longer beneficial"
from https://www.gnu.org/philosophy/copyright-versus-community.en...
First, if we assume Stallman is human, we have to grant he will not be right about everything (impossible on logical grounds and supported by the fact that he publicly changed his views on certain things in the past).
Second, when it comes to action, he only argues that copyright should have reduced power, which we can all agree with; he does not appear to argue for the death of copyright. Death of copyright would seem counter-productive, unless it also implied the death of corporate ability to withhold the source from the users and many other things.
You will note that the very text you linked to is copyrighted. There’s a reason for that.
And yet he is.
Copyright is what enables free and open licenses such as Creative Commons and every version/variant of the GPL. Without copyright, what would become of these licenses, and movements that have espoused them?
Copyleft is an abuse of copyright to pervert its intention. Copyright's intent was that you could not copy things freely, and copyleft is to ensure you can.
If there is no copyright, then you can copy things freely.
All that we need after that to realize the GPL ideal is to legally mandate that people have a right to access and modify source code of software/hardware they use, i.e. the government needs to mandate that Apple releases the iOS kernel and source code and that iPhones can be unlocked and custom kernels flashed, that John Deere must provide the tractor's source code, that my fridge releases its GPL-violating linux patches, etc etc.
You have the right to free speech, the right to a lawyer, and the right to source code. Simply amend the bill of rights.
The open source world would still exist if everything was public domain. It would be smaller because nobody would be forced to contribute but the dirty secret of GPL is that forced contribution virtually never happened anyway.
Full book content and model generations are not included because the books are copyrighted and the generations contain large portions of verbatim text.
There are plenty of old books in the public domain already... but I'm not sure what exactly this exercise is supposed to show, since the Kolmogorov limit still stands in the way of "infinite compression".
> There are plenty of old books in the public domain already
Yes but showing that it happens in books in the public domain does nothing to prove that it happens for copyrighted books