The ruling is fine. The judge is not Alsop but he’s not technically incompetent either, which is good.
The torrent comments in general are nothing to get het up about; in summary
1) Meta wanted to download but not upload libgen and Anna’s after they couldn’t find anyone with rights to license that would talk to them.
2) they didn't want to distribute; just download. An engineer put in evidence that they restricted seeding successfully.
3) late in the case Silverman et al claimed while they hadnt been seeding they had been leeching and that counts as distribution (?!)
Judge commented as follows
1. just downloading is probably fine because it could be for purposes of fair use, and fair use concerns generally trump even good faith and fair dealing
2. Nobody could get llama to spit out more than a 60 token quote from a plaintiff book; thus llama is not made for infringement
3. We will need more briefing on this leeching thing which it is alleged is a form of distribution.
The judge lays out what he thinks a workable claim to get to the supreme court would be, which is that these llms defeat the purpose of our copyright laws by reducing the amount of human creativity and expression available to those who want to create economic value through creativity. Eg where will the jobs for biographers go?
I will say that debate is an active topic worldwide right now and a good question, with answers ranging from: “this maximizes human creativity bro” to “laser printers disrupted lead type foundries, that was great” to “nobody will ever write again and we are murdering our creative class and burning down their craftsman mid century modern homes.”
It seems to me this will get taken up next session with SCOTUS but also that it’s a little early; we just don’t know where this is going exactly. Either way, I expect our current judge will learn that leeching is precisely NOT seeding once the defense legal team has time to brief him.
> 1. just downloading is probably fine because it could be for purposes of fair use, and fair use concerns generally trump even good faith and fair dealing
This smells a bit strange to me, it's a "for-profit" company.. Fair use is a bit of pipe-dream here. Also there is no conditions on the source of the content ? If the source was obtained from illegal sources IE illegal distribution of copyrighted materials does that not play a part ?
Also will this set a precedent that if I download HBO's collection but don't seed or use for any commercial reasons it will be considered Fair Use ?
This whole thing just reeks of "rules for thee but not for me".
> This smells a bit strange to me, it's a "for-profit" company.. Fair use is a bit of pipe-dream here
Why do you think that? For-profit companies use fair use all the time. Its not unusual.
Yes, a usage being non-commercial can be a factor in favour of fair use, but its just one factor. Its definitely not a neccesary condition nor is it a sufficient condition.
> If the source was obtained from illegal sources IE illegal distribution of copyrighted materials does that not play a part ?
Why would it? That isn't really how copyright works. Its about the right to "copy" (or not to), not about distribution methods.
> Also will this set a precedent that if I download HBO's collection but don't seed or use for any commercial reasons it will be considered Fair Use ?
No. That's not the reason this is potentially fair use.
[Although as an aside it uses to be in Canada that only uploading was illegal].
> Why would it? That isn't really how copyright works. Its about the right to "copy" (or not to), not about distribution methods.
Okay, but if there was no permission to "copy" the content by the owners. I wish I knew more about it all, but seems to me that quoting a snippet from a book while offering comment on it would be classic fair use. Consuming the entire collection for free to charge for transformative services really doesn't feel 'fair'.
And again I can't shake the feeling that if I did this, was brought to court. I would be laughed at for claiming fair use.
My (limited) understanding was that in the USA it was not illegal to read a book you don't own but it is illegal to make a copy (download) of a book you don't own.
I still don't fully grok how Meta can legally download a pirated book as fair use when an individual doing the same would be deemed a criminal act.
It would seem that Meta still don't have the right to make copies of books that they haven't paid for no matter what they do with it.
It's because in the US, you are granted the right to copy (copyright) broadly and under a number of circumstances. The creator is given a right to prevent copying under a limited (albeit very broad) set of circumstances.
Since we have a usage based assessment system on the major chip in the right to prevent copying, "fair use", which by the way is designed specifically for the common good -- enhancing the overall value to society of works that are limited by their creators -- its not about the copying. Its about the usage. Reading by an llm is fair usage in this case according to this judge's early speculations.
> This smells a bit strange to me, it's a "for-profit" company.. Fair use is a bit of pipe-dream here.
I think you're conflating "fair use" with "non-profit", or at least nudging that direction. That's too simplistic. Fair use is a four-factor test, and profitability is not among the four.
> if I download HBO's collection but don't seed or use for any commercial reasons it will be considered Fair Use ?
Fair use is just about how you use it, not how you get it. If you download HBO's collection and just watch it, that is fair use even if you are dinged about how you acquired the collection.
If you make a parody of a work in HBO's collection and publish/distribute it, the only question is whether or not the parody is a copy of their work. If it is, then you cannot distribute it; if not, then you can. You may "use" elements of their work in yours (as that is the point of a parody) but that can still be "fair".
Notice how none of that has anything to do with how you acquired the work. It is just not relevant to the question of fair use.
Fair Use is largely about reproduction (how much of a work you are allowed to copy and use, and for what purpose). It doesn’t deal with the legality of getting the work in the first place.
> 1. just downloading is probably fine because it could be for purposes of fair use, and fair use concerns generally trump even good faith and fair dealing
Hello MPAA, RIAA ? Where are you now ? Twenty years ago they were screaming bloody murder for such things.
But i guess, being a "pirate", is one thing, being a business man, is another.
I'm not sure how llms count as fair use.
It's just that we can't show HOW they've been encoded in the model, means it's fair use? Or that statistical representations are fair use? Or is it the generation aspect? I can't sell you a Harry potter book, but I can sell you some service that let's you generate it yourself?
I feel like this has really blown a hole in copyright.
> It's just that we can't show HOW they've been encoded in the model, means it's fair use?
Describing training as “encoding them in the model” doesn’t seem like an accurate description of what is happening. We know for certain that a typical copyrighted work that is trained on is not contained within the model. It’s simply not possible to represent the entirety of the training set within a model of that size in any meaningful way. There are also papers showing that memorisation plateaus at a reasonably low rate according to the size of the model. Training on more works doesn’t result in more memorisation, it results in more generalisation. So arguments based on the idea that those works are being copied into the model don’t seem to be founded in fact.
> I can't sell you a Harry potter book, but I can sell you some service that let's you generate it yourself?
That’s the reason why cases like this are doomed to fail: No model can output any of the Harry Potter books. Memorisation doesn’t happen at that scale. At best, they can output snippets. That’s clearly below the proportionality threshold for copyright to matter.
> That’s clearly below the proportionality threshold for copyright to matter.
This type of reasoning keeps coming up with seemingly zero consideration for why copyright actually exists. The goal of copyright, under US law, is "To promote the progress of science and useful arts".
The goal of companies creating these LLMs is to supersede the use of source material they draw from, like books. You use an LLM because it has all the answers without having to spend the money compensating the original authors, or put in the work digesting it yourself, that's their entire value proposition.
Their end game is to create a product so good that nobody has a reason to ever buy a book again. A few hours after you publish your book, the LLM will gobble it up and distribute the insights contain within to all of their users for free, "it's fair use", they say. There won't be any economic incentive to write books at that point, and so "the progress of science and useful arts" will crawl to a halt. Copyright defeated.
If LLM companies are allowed to produce market substitutes of original works then the goal of copyright is being defeated on a technicality and this ought to be a discussion about whether copyright should be abolished completely, not a discussion about whether big tech should be allowed to get away with it.
> The goal of companies creating these LLMs is to supersede the use of source material they draw from, like books.
Nobody is going to stop buying Harry Potter books because they can get an LLM to spit out ~50 words from one of the books. The proportionality factor is very clearly relevant here.
> If LLM companies are allowed to produce market substitutes of original works
Did Meta publish a book written by an LLM?
> The goal of copyright, under US law, is "To promote the progress of science and useful arts".
I would consider training LLMs to be very much in line with those goals.
> Nobody is going to stop buying Harry Potter books because they can get an LLM to spit out ~50 words from one of the books.
Not yet, but they'll stop buying books on niche technical subjects.
> Did Meta publish a book written by an LLM?
They don't need to publish a book to substitute original works. They substitute the original work every time they generate a response that is based on the book they substituted.
> I would consider training LLMs to be very much in line with those goals.
Because you're misunderstanding the premise. Original works are the ones that advance art and science. Those are the ones that are supposed to be protected by copyright.
Quoting Judge Alsup from his recent ruling in Bartz v. Anthropic.
> Instead, Authors contend generically that training LLMs will result in an explosion of works competing with their works — such as by creating alternative summaries of factual events, alternative examples of compelling writing about fictional events, and so on. This order assumes that is so (Opp. 22–23 (citing, e.g., Opp. Exh. 38)). But Authors’ complaint is no different than it would be if they complained that training schoolchildren to write well would result in an explosion of competing works. This is not the kind of competitive or creative displacement that concerns the Copyright Act. The Act seeks to advance original works of authorship, not to protect authors against competition.
Copyright was build to protect the artist from unauthorized copy by a human not by a machine (a machine wildly beyond their imagination at the time I mean), so the input and output limitations of humans were absolutely taken into account when writing such laws, if LLMs were treated in similar fashion authors would have had a say in wether their works can be used as inputs in such models or if they forbid it.
Yes it does, the spirit of the law matters in many one cases. A fair ruling would have declared that authors must be able to forbid the usage of their work as training data for any given model because the "transformative" processes that are being executed are wildly beyond what the writers of the law knew were even possible at the time of the writing of such laws.
> Copyright was build to protect the artist from unauthorized copy by a human not by a machine (a machine wildly beyond their imagination at the time I mean), so the input and output limitations of humans were absolutely taken into account when writing such laws
Copyright law was spurred by the spread of the printing press, a machine which has ability to output full replicas. It does not assume human-like input/output limitations.
> A fair ruling would have declared that authors must be able to forbid the usage of their work as training data for any given model because the "transformative" processes that are being executed are wildly beyond what the writers of the law knew were even possible
Copyright's basis in the US is "To promote the Progress of Science and useful Arts". Declaring a transformative use illegal because it's so novel would seem to run directly counter to that.
To my understanding it's generally the opposite (a pre-existing use with an established market that the rightsholder had expected to exploit) that would weigh against a finding of fair use.
The spirit of the law matters, but there are limits to how much existing statutes can be stretched to cover novel scenarios. Seems to me like new laws may be necessary to keep up (whatever the people would prefer them to be).
I kind of assumed I could ask it for verses from the bible one by one till i have the full book?
When i ask chatgpt for a specific page or so from HP I get the impression that the model would be perfectly capable of doing so but is hindred by extra work openAI put in to prevent the answer specifically because of copyright.
In which case the question: What if someone manages to do some prompt trickery again to get past it? Are they then responsible?
No, it can’t recreate a book. Well, maybe it could get most of the way for the Bible. That is an exceptional case because its adherents are constantly quoting verses religiously. I expect it’s the most reproduced, quoted, and translated book in history by a very significant margin. It’s also not copyrighted.
Can you do this for the general case? No, not even for extremely popular books. People might quote Harry Potter a lot, but they don’t quote the entire thing over and over, chapter and verse, on hundreds of thousands of different websites. The number of times Bible verses appear in the training data is going to absolutely dwarf the number of times Harry Potter quotes appear, and people aren’t quoting all parts of Harry Potter, just the interesting parts.
> When i ask chatgpt for a specific page or so from HP I get the impression that the model would be perfectly capable of doing so but is hindred by extra work openAI put in to prevent the answer specifically because of copyright.
They do put extra work in to filter this stuff out, but even if they didn’t the model wouldn’t be able to reproduce entire chapters, let alone entire books.
You can test this for yourself. Remember, this lawsuit isn’t against OpenAI, it’s against Meta. Download Llama and try to get it to reproduce Harry Potter. There won’t be any guardrails imposed on top of the model if you run it locally.
>People might quote Harry Potter a lot, but they don’t quote the entire thing over and over, chapter and verse, on hundreds of thousands of different websites.
I'm fairly certain I could find the entire thing in plain text in multiple places online.
A quick google gives the philosophers stone as the second result in pdf format on the internet archive but i'm sure with a bit of looking i'd bump into a lot of plaintext copies.
They might have taken measures to prevent this from being anywhere their training data (i think it would be fairly easy and something they'd likely do) but if they at any point failed for a book or so that they didn't consider wouldn't my original question stand?
You’re missing the point. An LLM is not going to memorise a whole book just because it’s seen a few copies. An LLM might be able to memorise the Bible in particular simply because Bible quotes are everywhere. There is a vast difference between being able to find a handful of copies online and having it constantly quoted everywhere that humans communicate. Bible quotes get literally everywhere. People put them on bumper stickers, tattoo themselves with it, put it in their email signatures, etc. Bible quotes are so omnipresent, they have become part of our language – a lot of idioms people use every day come from the Bible.
The Bible isn’t just a book, it’s been a massive part of human culture for millennia, to the point of it shaping language itself. LLMs might be able to memorise the Bible, but it’s not because they can memorise books, it’s because the Bible is far more than just a book.
I went to check and it seems like it works fine for plenty of other public domain books. The picture of Dorian Grey, Pride and prejudice and what have you.
I can ask for x amount of paragraphs from a specific and such.
I doubt every part of those books get quoted everywhere on a numbered basis like the bible might be.
For only recently public domain books it seems to be overly cautious trough the retroactively applied filtering where it refuses if it suspects there might be a single country where copyright still applies.
Don't have access to the account i was using before right now but when i'm using chatgpt free tier which i believe is GPT-4o I at first thought i got it right again.
I decided to ask it: Can you give me the first 4 paragraphs of chapter 3 of the book The picture of Dorian Grey?
And it gave me something and it looked alright to me.
It read right and i went to gutenberg and glanced over it and the first lines of each paragraph seemed correct but only the short ones were. The first paragraph which was longer after the opening lines suddenly had an entire section randomly replaced with hallucination.
A followup asking it to not hallucinate had it search the web to fetch the correct thing which isn't valid in this context.
I suspect it starts hallucinating once the bit of text gets long so i asked for specific sentences of chapters (and to do so without web search). the 1st, 2nd, 3rd and such.
It managed to not outright hallucinate lines then but did get the chapter i asked for wrong sometimes. I presume that with sufficiently careful prompting one can get the book out properly in sequential order with a lot of prompts but it takes quite some effort to get there. But that's where my curiosity ends for the night. My bed calls.
> proportionality threshold for copyright to matter.
This is the part I have a problem with, that threshold was put there for humans based on their capabilities, it's an extremely dishonest assessment that the same threshold must apply for a LLM and it's outputs, those works were created to be read by humans not a for-profit statistical inference machine, the derivative nature were also expected to be caused by the former no the later, so the judge should have admitted that the context of the law is insufficient and that copyright must include the power of forbidding the usage of one's work into such model for copyright to continue fulfilling it's intended purpose (or move the case to the supreme court I guess)
> that threshold was put there for humans based on their capabilities
It wasn’t. It’s there because a small proportion being reproduced doesn’t harm the copyright holder in the same way a full reproduction does.
Nobody is going to stop buying Harry Potter books because they can get an LLM to spit out ~50 words from the book. This is entirely in line with the spirit of the law. This is exactly why proportionality is a factor in fair use.
The judge is claiming that because the use is of the books are “so transformative,” the usage of these books to train an llm is fair use.
I’m not familiar with the facts of the case and IANAL, and its late, but how did the plaintiffs determine their books were being used for training of the llm? Was the model spitting out language that was similar or verbatim to their works?
> Maybe I'm mistaken but shouldn't the source come from a legal source
There is no such thing as a legal or illegal source, only legal or illegal uses.
If the use was legal, then it doesn't matter where you got the material from. Similary if you got the material via more conventional means it would still be copyright infringement if you used it in an illegal way.
> Again if I download the entire works of HBO tv shows, then make a "transformative" version on my iphone, how can that be considered fair use?
That wouldn't be considered transformative. In this context "transformative" means you transformed it into something with a different purpose than the original.
However if you for example made a video essay for youtube talking about the themes (or whatever) of the tv show including clips from it, that would be transformative and probably fine.
> However if you for example made a video essay for youtube talking about the themes (or whatever) of the tv show including clips from it, that would be transformative and probably fine.
Then I get a free pass when downloading the entire disney collection but making a youtube essay video for each downloaded item? So long as I don't seed of course.
I'm not trying to be argumentative, but surely you can see how that will never pass muster.
downloading is not the illegal part. distribution is the illegal part.
if you watch a video from a site that has a Disney movie on there, the site distributing the content to you is doing the illegal part. You are not doing anything illegal (AFAIK).
>The judge is claiming that because the use is of the books are “so transformative,” the usage of these books to train an llm is fair use.
"you're doing something so critical to our (country's) success, that we're ok to waive copyright. I get that, if the US doesn't do it, then China will(is).
Interesting judgement, and it's implications, if you are correct haha.
Same. If I invented a novel new way of encoding video and used it to pack a bunch of movies into a single file, I would fully expect to be sued if I tried distributing that file, and equally so if I let people use a web site that let them extract individual videos from that file. Why should text be treated differently?
You are allowed to quote from copyrighted works without needing permission. Trying to assert copyright because of a quote of, say, a mere 60 words in length would get you thrown out of any judge’s court.
It was shown, in this case, that the llms wouldn’t generate accurate quotes more than 60 words in length.
This is not comparable to encoding a full video file.
I think the better analogy is if you had someone with a superhuman, but not perfect memory read a bunch of stuff, then you were allowed to talk to the person about the things they’d read, does that violate copyright? I’d say clearly no.
Then what if their memory is so good, they repeat entire sections verbatim when asked. Does that violate it? I’d say it’s grey.
But that’s a very specific case - reproducing large chunks of owned work is something that can be quite easily detected and prevented and I’m almost certain the frontier labs are already going this.
So I think it’s just very not clear - the reality is this is a novel situation, the job of the courts is now to basically decide what’s allowed and what’s not. But the rational shouldn’t be ‘this can’t be fair use it’s just compression’. Because it’s clearly something fundamentally different and existing laws just aren’t applicable imo
That's not a great analogy. A person is expected to use their discretion, and can be held legally liable for their actions. A machine is not, and cannot.
> Then what if their memory is so good, they repeat entire sections verbatim when asked. Does that violate it? I’d say it’s grey.
That's an unambiguous "yes". Performing a copyrighted play or piece of music without the rights to do so is universally considered a copyright violation, even if the performers are performing from memory. It's still a copyright violation if they don't remember their parts perfectly and have to ad-lib sometimes, or if they don't perform the entire work from start to finish.
This a strawman, in the sense that it is not accurate to think about AI models as a compressed form of their training data, since the lossiness is so high. One of the insights from the trial is the LLMs are particularly poor at reproducing original texts (60 tokens was the max found in this trial, IIRC). This is taken into account when considering fair use based on the fourth fair use factor: how the work impacts the market for the original work. It's hard to make an argument that LLMs are replacing long-form text works, since they have so much trouble actually producing them.
There's a whole related topic here in the realm of news (since it's shorter form), but it also has a much shorter half-life. Not sure what I think there yet.
People can process only fraction of that while using much more time to do that. And I'm using 'process' here to meet you in your nihilist argument that these algorithms are same as humans. Which is pretty strange because people barely acknowledge similarities with other mammals but suddenly software is equal.
The word transformative was put there in a time of manual transformative processes, like when you paint something similar to what you saw in a painting by another artist, with all the implied limitations that entails, like the time it took from you to watch that painting, and the time it takes you to create that new painting, nothing to do at all with the way LLMs operate, an honest assessment would have found that the word was meant for a wildly different use case and therefore it required a bigger and more nuanced discussion.
> The word transformative was put there in a time of manual transformative processes, like when you paint something similar to what you saw in a painting by another artist
Do you have any citation that that is how the word "transformation" was understood historically? Because what your suggesting seems to be the opposite of what i've read.
My understanding is even back in the 1800s (e.g. https://en.wikipedia.org/wiki/Folsom_v._Marsh ) your example would not be considered transformative, if your intention was to make a similar painting to serve a similar purpose.
I'm inclined to agree here. LLMs do not use just a paragraph here and there in accordance with fair use, but rather uses the entire body of work to train itself.
That’s one hell of a headline for a story about Meta winning summary judgement for most of the claims against them. You’d be forgiven for thinking Meta lost this case, going by the headline.
It's not a story about Meta winning (that previous story is linked in the first sentence), but a story expanding on the part where they didn't. The headline is totally appropriate for that piece.
It’s crazy, yet so predictable, that while the system tries to bankrupt individuals for torrenting a single book or movie in this case the excuse “it was just to train an LLM” will fly. Imagine a private individual would argue that in court.
If this case reaches the Supreme Court, one key consideration is the potential pressure to support the growth of U.S. AI companies. Rather than imposing strict legal restrictions that could hinder their ability to compete globally, especially against companies in countries with more lenient regulations like China, the Court may be inclined to take a more permissive stance.
So the argument is that by torrenting ebooks, meta provided bandwidth to the torrent network, and thus provided (financial??!) benefit to pirate sites?
I got to be honest, that sounds extremely weak to me. The benefit to the pirate site of joining the torrent swam seems like it would be extremely slight.
The pirate sites? There is no pirate site hosting files here, BitTorrent is peer to peer.
If it’s fine for a large corp. like Meta to pirate books then it’s fine for everyone else. If it’s a crime for ordinary consumers then it’s a crime for Meta too.
Especially as Meta aren’t doing this for charity. They train LLMs for their own gain.
> The pirate sites? There is no pirate site hosting files here, BitTorrent is peer to peer.
Or "shadow library" or whatever you want to call it. The argument according to the article is that the entity that created the torrent, which also as far as i understand also operates a traditional website, benefits from meta's actions.
I mean it seems clear that Meta did not pirate the content to watch/read it. However, I guess according to the ruling you could pirate anything you want (but no seeding), produce a shitty haiku based on what you pirated and then claim fair use.
The system wants to destroy creativity and humans. The system wants artists and writers to depend on the patronage of the ultra rich who steal their output.
We are back to feudal society, except that monarchs or their advisers at least had taste as opposed to the Nouveau riche.
Just read (most) of the ruling.
The ruling is fine. The judge is not Alsop but he’s not technically incompetent either, which is good.
The torrent comments in general are nothing to get het up about; in summary
1) Meta wanted to download but not upload libgen and Anna’s after they couldn’t find anyone with rights to license that would talk to them.
2) they didn't want to distribute; just download. An engineer put in evidence that they restricted seeding successfully.
3) late in the case Silverman et al claimed while they hadnt been seeding they had been leeching and that counts as distribution (?!)
Judge commented as follows
1. just downloading is probably fine because it could be for purposes of fair use, and fair use concerns generally trump even good faith and fair dealing
2. Nobody could get llama to spit out more than a 60 token quote from a plaintiff book; thus llama is not made for infringement
3. We will need more briefing on this leeching thing which it is alleged is a form of distribution.
The judge lays out what he thinks a workable claim to get to the supreme court would be, which is that these llms defeat the purpose of our copyright laws by reducing the amount of human creativity and expression available to those who want to create economic value through creativity. Eg where will the jobs for biographers go?
I will say that debate is an active topic worldwide right now and a good question, with answers ranging from: “this maximizes human creativity bro” to “laser printers disrupted lead type foundries, that was great” to “nobody will ever write again and we are murdering our creative class and burning down their craftsman mid century modern homes.”
It seems to me this will get taken up next session with SCOTUS but also that it’s a little early; we just don’t know where this is going exactly. Either way, I expect our current judge will learn that leeching is precisely NOT seeding once the defense legal team has time to brief him.
> 1. just downloading is probably fine because it could be for purposes of fair use, and fair use concerns generally trump even good faith and fair dealing
This smells a bit strange to me, it's a "for-profit" company.. Fair use is a bit of pipe-dream here. Also there is no conditions on the source of the content ? If the source was obtained from illegal sources IE illegal distribution of copyrighted materials does that not play a part ?
Also will this set a precedent that if I download HBO's collection but don't seed or use for any commercial reasons it will be considered Fair Use ?
This whole thing just reeks of "rules for thee but not for me".
> This smells a bit strange to me, it's a "for-profit" company.. Fair use is a bit of pipe-dream here
Why do you think that? For-profit companies use fair use all the time. Its not unusual.
Yes, a usage being non-commercial can be a factor in favour of fair use, but its just one factor. Its definitely not a neccesary condition nor is it a sufficient condition.
> If the source was obtained from illegal sources IE illegal distribution of copyrighted materials does that not play a part ?
Why would it? That isn't really how copyright works. Its about the right to "copy" (or not to), not about distribution methods.
> Also will this set a precedent that if I download HBO's collection but don't seed or use for any commercial reasons it will be considered Fair Use ?
No. That's not the reason this is potentially fair use.
[Although as an aside it uses to be in Canada that only uploading was illegal].
> Why would it? That isn't really how copyright works. Its about the right to "copy" (or not to), not about distribution methods.
Okay, but if there was no permission to "copy" the content by the owners. I wish I knew more about it all, but seems to me that quoting a snippet from a book while offering comment on it would be classic fair use. Consuming the entire collection for free to charge for transformative services really doesn't feel 'fair'.
And again I can't shake the feeling that if I did this, was brought to court. I would be laughed at for claiming fair use.
My (limited) understanding was that in the USA it was not illegal to read a book you don't own but it is illegal to make a copy (download) of a book you don't own.
I still don't fully grok how Meta can legally download a pirated book as fair use when an individual doing the same would be deemed a criminal act.
It would seem that Meta still don't have the right to make copies of books that they haven't paid for no matter what they do with it.
It's because in the US, you are granted the right to copy (copyright) broadly and under a number of circumstances. The creator is given a right to prevent copying under a limited (albeit very broad) set of circumstances.
Since we have a usage based assessment system on the major chip in the right to prevent copying, "fair use", which by the way is designed specifically for the common good -- enhancing the overall value to society of works that are limited by their creators -- its not about the copying. Its about the usage. Reading by an llm is fair usage in this case according to this judge's early speculations.
> Why would it? That isn't really how copyright works. Its about the right to "copy" (or not to), not about distribution methods.
Copyright covers four rights:
Copying and distribution are central to what copyright attempts to control.> This smells a bit strange to me, it's a "for-profit" company.. Fair use is a bit of pipe-dream here
Fair use can be for profit.
> if I download HBO's collection but don't seed or use for any commercial reasons it will be considered Fair Use
No, seeding is automatically not fair use. Leeching does not automatically mean its not fair use, just that it might be.
> Leeching does not automatically mean its not fair use, just that it might be.
What do you mean by might be? It either yes or no.
> This smells a bit strange to me, it's a "for-profit" company.. Fair use is a bit of pipe-dream here.
I think you're conflating "fair use" with "non-profit", or at least nudging that direction. That's too simplistic. Fair use is a four-factor test, and profitability is not among the four.
> if I download HBO's collection but don't seed or use for any commercial reasons it will be considered Fair Use ?
Fair use is just about how you use it, not how you get it. If you download HBO's collection and just watch it, that is fair use even if you are dinged about how you acquired the collection.
If you make a parody of a work in HBO's collection and publish/distribute it, the only question is whether or not the parody is a copy of their work. If it is, then you cannot distribute it; if not, then you can. You may "use" elements of their work in yours (as that is the point of a parody) but that can still be "fair".
Notice how none of that has anything to do with how you acquired the work. It is just not relevant to the question of fair use.
If you pirate a movie, watch it, and talk about it but don't distribute it then yes that's fair use. People literally do this all the time.
What? No. Piracy is illegal.
Fair Use is largely about reproduction (how much of a work you are allowed to copy and use, and for what purpose). It doesn’t deal with the legality of getting the work in the first place.
Sure piracy is illegal but fair use has nothing to do with how you watched it.
> 1. just downloading is probably fine because it could be for purposes of fair use, and fair use concerns generally trump even good faith and fair dealing
Hello MPAA, RIAA ? Where are you now ? Twenty years ago they were screaming bloody murder for such things. But i guess, being a "pirate", is one thing, being a business man, is another.
I'm not sure how llms count as fair use. It's just that we can't show HOW they've been encoded in the model, means it's fair use? Or that statistical representations are fair use? Or is it the generation aspect? I can't sell you a Harry potter book, but I can sell you some service that let's you generate it yourself?
I feel like this has really blown a hole in copyright.
> It's just that we can't show HOW they've been encoded in the model, means it's fair use?
Describing training as “encoding them in the model” doesn’t seem like an accurate description of what is happening. We know for certain that a typical copyrighted work that is trained on is not contained within the model. It’s simply not possible to represent the entirety of the training set within a model of that size in any meaningful way. There are also papers showing that memorisation plateaus at a reasonably low rate according to the size of the model. Training on more works doesn’t result in more memorisation, it results in more generalisation. So arguments based on the idea that those works are being copied into the model don’t seem to be founded in fact.
> I can't sell you a Harry potter book, but I can sell you some service that let's you generate it yourself?
That’s the reason why cases like this are doomed to fail: No model can output any of the Harry Potter books. Memorisation doesn’t happen at that scale. At best, they can output snippets. That’s clearly below the proportionality threshold for copyright to matter.
> That’s clearly below the proportionality threshold for copyright to matter.
This type of reasoning keeps coming up with seemingly zero consideration for why copyright actually exists. The goal of copyright, under US law, is "To promote the progress of science and useful arts".
The goal of companies creating these LLMs is to supersede the use of source material they draw from, like books. You use an LLM because it has all the answers without having to spend the money compensating the original authors, or put in the work digesting it yourself, that's their entire value proposition.
Their end game is to create a product so good that nobody has a reason to ever buy a book again. A few hours after you publish your book, the LLM will gobble it up and distribute the insights contain within to all of their users for free, "it's fair use", they say. There won't be any economic incentive to write books at that point, and so "the progress of science and useful arts" will crawl to a halt. Copyright defeated.
If LLM companies are allowed to produce market substitutes of original works then the goal of copyright is being defeated on a technicality and this ought to be a discussion about whether copyright should be abolished completely, not a discussion about whether big tech should be allowed to get away with it.
> The goal of companies creating these LLMs is to supersede the use of source material they draw from, like books.
Nobody is going to stop buying Harry Potter books because they can get an LLM to spit out ~50 words from one of the books. The proportionality factor is very clearly relevant here.
> If LLM companies are allowed to produce market substitutes of original works
Did Meta publish a book written by an LLM?
> The goal of copyright, under US law, is "To promote the progress of science and useful arts".
I would consider training LLMs to be very much in line with those goals.
> Nobody is going to stop buying Harry Potter books because they can get an LLM to spit out ~50 words from one of the books.
Not yet, but they'll stop buying books on niche technical subjects.
> Did Meta publish a book written by an LLM?
They don't need to publish a book to substitute original works. They substitute the original work every time they generate a response that is based on the book they substituted.
> I would consider training LLMs to be very much in line with those goals.
Because you're misunderstanding the premise. Original works are the ones that advance art and science. Those are the ones that are supposed to be protected by copyright.
Quoting Judge Alsup from his recent ruling in Bartz v. Anthropic.
> Instead, Authors contend generically that training LLMs will result in an explosion of works competing with their works — such as by creating alternative summaries of factual events, alternative examples of compelling writing about fictional events, and so on. This order assumes that is so (Opp. 22–23 (citing, e.g., Opp. Exh. 38)). But Authors’ complaint is no different than it would be if they complained that training schoolchildren to write well would result in an explosion of competing works. This is not the kind of competitive or creative displacement that concerns the Copyright Act. The Act seeks to advance original works of authorship, not to protect authors against competition.
That's unrelated to the reasoning that I provided.
Copyright was build to protect the artist from unauthorized copy by a human not by a machine (a machine wildly beyond their imagination at the time I mean), so the input and output limitations of humans were absolutely taken into account when writing such laws, if LLMs were treated in similar fashion authors would have had a say in wether their works can be used as inputs in such models or if they forbid it.
This reply doesn’t seem to relate to either of the points I made.
Yes it does, the spirit of the law matters in many one cases. A fair ruling would have declared that authors must be able to forbid the usage of their work as training data for any given model because the "transformative" processes that are being executed are wildly beyond what the writers of the law knew were even possible at the time of the writing of such laws.
> Copyright was build to protect the artist from unauthorized copy by a human not by a machine (a machine wildly beyond their imagination at the time I mean), so the input and output limitations of humans were absolutely taken into account when writing such laws
Copyright law was spurred by the spread of the printing press, a machine which has ability to output full replicas. It does not assume human-like input/output limitations.
> A fair ruling would have declared that authors must be able to forbid the usage of their work as training data for any given model because the "transformative" processes that are being executed are wildly beyond what the writers of the law knew were even possible
Copyright's basis in the US is "To promote the Progress of Science and useful Arts". Declaring a transformative use illegal because it's so novel would seem to run directly counter to that.
To my understanding it's generally the opposite (a pre-existing use with an established market that the rightsholder had expected to exploit) that would weigh against a finding of fair use.
The spirit of the law matters, but there are limits to how much existing statutes can be stretched to cover novel scenarios. Seems to me like new laws may be necessary to keep up (whatever the people would prefer them to be).
I made two points:
- It is not accurate to describe training as “encoding works into the model”.
– A model cannot recreate a Harry Potter book.
Neither of these have anything to do with “the spirit of the law”.
Can it not recreate a book?
I kind of assumed I could ask it for verses from the bible one by one till i have the full book?
When i ask chatgpt for a specific page or so from HP I get the impression that the model would be perfectly capable of doing so but is hindred by extra work openAI put in to prevent the answer specifically because of copyright. In which case the question: What if someone manages to do some prompt trickery again to get past it? Are they then responsible?
No, it can’t recreate a book. Well, maybe it could get most of the way for the Bible. That is an exceptional case because its adherents are constantly quoting verses religiously. I expect it’s the most reproduced, quoted, and translated book in history by a very significant margin. It’s also not copyrighted.
Can you do this for the general case? No, not even for extremely popular books. People might quote Harry Potter a lot, but they don’t quote the entire thing over and over, chapter and verse, on hundreds of thousands of different websites. The number of times Bible verses appear in the training data is going to absolutely dwarf the number of times Harry Potter quotes appear, and people aren’t quoting all parts of Harry Potter, just the interesting parts.
> When i ask chatgpt for a specific page or so from HP I get the impression that the model would be perfectly capable of doing so but is hindred by extra work openAI put in to prevent the answer specifically because of copyright.
They do put extra work in to filter this stuff out, but even if they didn’t the model wouldn’t be able to reproduce entire chapters, let alone entire books.
You can test this for yourself. Remember, this lawsuit isn’t against OpenAI, it’s against Meta. Download Llama and try to get it to reproduce Harry Potter. There won’t be any guardrails imposed on top of the model if you run it locally.
>People might quote Harry Potter a lot, but they don’t quote the entire thing over and over, chapter and verse, on hundreds of thousands of different websites.
I'm fairly certain I could find the entire thing in plain text in multiple places online. A quick google gives the philosophers stone as the second result in pdf format on the internet archive but i'm sure with a bit of looking i'd bump into a lot of plaintext copies.
They might have taken measures to prevent this from being anywhere their training data (i think it would be fairly easy and something they'd likely do) but if they at any point failed for a book or so that they didn't consider wouldn't my original question stand?
You’re missing the point. An LLM is not going to memorise a whole book just because it’s seen a few copies. An LLM might be able to memorise the Bible in particular simply because Bible quotes are everywhere. There is a vast difference between being able to find a handful of copies online and having it constantly quoted everywhere that humans communicate. Bible quotes get literally everywhere. People put them on bumper stickers, tattoo themselves with it, put it in their email signatures, etc. Bible quotes are so omnipresent, they have become part of our language – a lot of idioms people use every day come from the Bible.
The Bible isn’t just a book, it’s been a massive part of human culture for millennia, to the point of it shaping language itself. LLMs might be able to memorise the Bible, but it’s not because they can memorise books, it’s because the Bible is far more than just a book.
I went to check and it seems like it works fine for plenty of other public domain books. The picture of Dorian Grey, Pride and prejudice and what have you. I can ask for x amount of paragraphs from a specific and such.
I doubt every part of those books get quoted everywhere on a numbered basis like the bible might be. For only recently public domain books it seems to be overly cautious trough the retroactively applied filtering where it refuses if it suspects there might be a single country where copyright still applies.
I can’t reproduce that. What model were you using and what prompt?
Don't have access to the account i was using before right now but when i'm using chatgpt free tier which i believe is GPT-4o I at first thought i got it right again.
I decided to ask it: Can you give me the first 4 paragraphs of chapter 3 of the book The picture of Dorian Grey?
And it gave me something and it looked alright to me. It read right and i went to gutenberg and glanced over it and the first lines of each paragraph seemed correct but only the short ones were. The first paragraph which was longer after the opening lines suddenly had an entire section randomly replaced with hallucination.
A followup asking it to not hallucinate had it search the web to fetch the correct thing which isn't valid in this context.
I suspect it starts hallucinating once the bit of text gets long so i asked for specific sentences of chapters (and to do so without web search). the 1st, 2nd, 3rd and such.
It managed to not outright hallucinate lines then but did get the chapter i asked for wrong sometimes. I presume that with sufficiently careful prompting one can get the book out properly in sequential order with a lot of prompts but it takes quite some effort to get there. But that's where my curiosity ends for the night. My bed calls.
> I presume that with sufficiently careful prompting one can get the book out properly
You failed to get it to reproduce one paragraph. Why on earth would you presume you can do it for the entire book‽
Did you read what I said? I got plenty of correct paragraphs. They just had to be short. Breaking up the big paragraphs seems to help the issue.
> proportionality threshold for copyright to matter.
This is the part I have a problem with, that threshold was put there for humans based on their capabilities, it's an extremely dishonest assessment that the same threshold must apply for a LLM and it's outputs, those works were created to be read by humans not a for-profit statistical inference machine, the derivative nature were also expected to be caused by the former no the later, so the judge should have admitted that the context of the law is insufficient and that copyright must include the power of forbidding the usage of one's work into such model for copyright to continue fulfilling it's intended purpose (or move the case to the supreme court I guess)
> that threshold was put there for humans based on their capabilities
It wasn’t. It’s there because a small proportion being reproduced doesn’t harm the copyright holder in the same way a full reproduction does.
Nobody is going to stop buying Harry Potter books because they can get an LLM to spit out ~50 words from the book. This is entirely in line with the spirit of the law. This is exactly why proportionality is a factor in fair use.
The judge is claiming that because the use is of the books are “so transformative,” the usage of these books to train an llm is fair use.
I’m not familiar with the facts of the case and IANAL, and its late, but how did the plaintiffs determine their books were being used for training of the llm? Was the model spitting out language that was similar or verbatim to their works?
> The judge is claiming that because the use is of the books are “so transformative,” the usage of these books to train an llm is fair use.
Maybe I'm mistaken but shouldn't the source come from a legal source ? This is not public domain material.
Again if I download the entire works of HBO tv shows, then make a "transformative" version on my iphone, how can that be considered fair use?
> Maybe I'm mistaken but shouldn't the source come from a legal source
There is no such thing as a legal or illegal source, only legal or illegal uses.
If the use was legal, then it doesn't matter where you got the material from. Similary if you got the material via more conventional means it would still be copyright infringement if you used it in an illegal way.
> Again if I download the entire works of HBO tv shows, then make a "transformative" version on my iphone, how can that be considered fair use?
That wouldn't be considered transformative. In this context "transformative" means you transformed it into something with a different purpose than the original.
However if you for example made a video essay for youtube talking about the themes (or whatever) of the tv show including clips from it, that would be transformative and probably fine.
> However if you for example made a video essay for youtube talking about the themes (or whatever) of the tv show including clips from it, that would be transformative and probably fine.
Then I get a free pass when downloading the entire disney collection but making a youtube essay video for each downloaded item? So long as I don't seed of course.
I'm not trying to be argumentative, but surely you can see how that will never pass muster.
downloading is not the illegal part. distribution is the illegal part.
if you watch a video from a site that has a Disney movie on there, the site distributing the content to you is doing the illegal part. You are not doing anything illegal (AFAIK).
It'll be, but in a slightly different way. As it will be considered _fair_ for the Warner Bros to sue you dry.
> but how did the plaintiffs determine their books were being used for training of the llm?
I think facebook admited this. I don't think the fact of this is under dispute.
>The judge is claiming that because the use is of the books are “so transformative,” the usage of these books to train an llm is fair use.
"you're doing something so critical to our (country's) success, that we're ok to waive copyright. I get that, if the US doesn't do it, then China will(is).
Interesting judgement, and it's implications, if you are correct haha.
"Transformative" is a legal term with a specific meaning. The copyrighted work has to be transformed somehow rather than copied.
https://en.m.wikipedia.org/wiki/Transformative_use
Ahh, good pick up. Thanks, I definitely had the wrong understanding of transformative. Mostly around transforming the use case.
Same. If I invented a novel new way of encoding video and used it to pack a bunch of movies into a single file, I would fully expect to be sued if I tried distributing that file, and equally so if I let people use a web site that let them extract individual videos from that file. Why should text be treated differently?
You are allowed to quote from copyrighted works without needing permission. Trying to assert copyright because of a quote of, say, a mere 60 words in length would get you thrown out of any judge’s court.
It was shown, in this case, that the llms wouldn’t generate accurate quotes more than 60 words in length.
This is not comparable to encoding a full video file.
I think the better analogy is if you had someone with a superhuman, but not perfect memory read a bunch of stuff, then you were allowed to talk to the person about the things they’d read, does that violate copyright? I’d say clearly no.
Then what if their memory is so good, they repeat entire sections verbatim when asked. Does that violate it? I’d say it’s grey.
But that’s a very specific case - reproducing large chunks of owned work is something that can be quite easily detected and prevented and I’m almost certain the frontier labs are already going this.
So I think it’s just very not clear - the reality is this is a novel situation, the job of the courts is now to basically decide what’s allowed and what’s not. But the rational shouldn’t be ‘this can’t be fair use it’s just compression’. Because it’s clearly something fundamentally different and existing laws just aren’t applicable imo
That's not a great analogy. A person is expected to use their discretion, and can be held legally liable for their actions. A machine is not, and cannot.
> Then what if their memory is so good, they repeat entire sections verbatim when asked. Does that violate it? I’d say it’s grey.
That's an unambiguous "yes". Performing a copyrighted play or piece of music without the rights to do so is universally considered a copyright violation, even if the performers are performing from memory. It's still a copyright violation if they don't remember their parts perfectly and have to ad-lib sometimes, or if they don't perform the entire work from start to finish.
This a strawman, in the sense that it is not accurate to think about AI models as a compressed form of their training data, since the lossiness is so high. One of the insights from the trial is the LLMs are particularly poor at reproducing original texts (60 tokens was the max found in this trial, IIRC). This is taken into account when considering fair use based on the fourth fair use factor: how the work impacts the market for the original work. It's hard to make an argument that LLMs are replacing long-form text works, since they have so much trouble actually producing them.
There's a whole related topic here in the realm of news (since it's shorter form), but it also has a much shorter half-life. Not sure what I think there yet.
One should also keep in mind the countless people who got much of their education from pirated books.
That has nothing to do with whether LLMs are fair use.
This seems to be a bad faith argument, although it would be amusing to see Facebook use it.
"Your honor, it's fair use because students have downloaded educational books for years."
People can process only fraction of that while using much more time to do that. And I'm using 'process' here to meet you in your nihilist argument that these algorithms are same as humans. Which is pretty strange because people barely acknowledge similarities with other mammals but suddenly software is equal.
The word transformative was put there in a time of manual transformative processes, like when you paint something similar to what you saw in a painting by another artist, with all the implied limitations that entails, like the time it took from you to watch that painting, and the time it takes you to create that new painting, nothing to do at all with the way LLMs operate, an honest assessment would have found that the word was meant for a wildly different use case and therefore it required a bigger and more nuanced discussion.
> The word transformative was put there in a time of manual transformative processes, like when you paint something similar to what you saw in a painting by another artist
Do you have any citation that that is how the word "transformation" was understood historically? Because what your suggesting seems to be the opposite of what i've read.
My understanding is even back in the 1800s (e.g. https://en.wikipedia.org/wiki/Folsom_v._Marsh ) your example would not be considered transformative, if your intention was to make a similar painting to serve a similar purpose.
I'm inclined to agree here. LLMs do not use just a paragraph here and there in accordance with fair use, but rather uses the entire body of work to train itself.
Or am I misunderstanding something about LLMs?
That’s one hell of a headline for a story about Meta winning summary judgement for most of the claims against them. You’d be forgiven for thinking Meta lost this case, going by the headline.
It's not a story about Meta winning (that previous story is linked in the first sentence), but a story expanding on the part where they didn't. The headline is totally appropriate for that piece.
Surely it should be a whole separate copyright case with fines of up to $150,000 per work infringed.
It’s crazy, yet so predictable, that while the system tries to bankrupt individuals for torrenting a single book or movie in this case the excuse “it was just to train an LLM” will fly. Imagine a private individual would argue that in court.
Ironically, the Llama models enable people to fine-tune on their own material. A lot of people are doing exactly this.
If this case reaches the Supreme Court, one key consideration is the potential pressure to support the growth of U.S. AI companies. Rather than imposing strict legal restrictions that could hinder their ability to compete globally, especially against companies in countries with more lenient regulations like China, the Court may be inclined to take a more permissive stance.
Sorta not really. They said the plaintiff had a non relevant argument or something.
Feels more of political purpose as AI is now a "national strategy".
So the argument is that by torrenting ebooks, meta provided bandwidth to the torrent network, and thus provided (financial??!) benefit to pirate sites?
I got to be honest, that sounds extremely weak to me. The benefit to the pirate site of joining the torrent swam seems like it would be extremely slight.
The pirate sites? There is no pirate site hosting files here, BitTorrent is peer to peer.
If it’s fine for a large corp. like Meta to pirate books then it’s fine for everyone else. If it’s a crime for ordinary consumers then it’s a crime for Meta too.
Especially as Meta aren’t doing this for charity. They train LLMs for their own gain.
> The pirate sites? There is no pirate site hosting files here, BitTorrent is peer to peer.
Or "shadow library" or whatever you want to call it. The argument according to the article is that the entity that created the torrent, which also as far as i understand also operates a traditional website, benefits from meta's actions.
I think that is really far fetched.
The argument is that distribution is the infringement. This is basically the only thing torrenters get charged with.
That does not seem to be the argument that was presented in the article.
Wait. So I can pirate any stuff as long as I intend to train an LLM with it?
No, because the downloading was considered illegal. You can use any book or movie you bought to train an LLM though.
I mean it seems clear that Meta did not pirate the content to watch/read it. However, I guess according to the ruling you could pirate anything you want (but no seeding), produce a shitty haiku based on what you pirated and then claim fair use.
my brother in Christ, you can pirate any stuff for any reason.
Haha. I meant legally. In EU it's common to get fined for pirating movies.
When do the outputs cease to be a derived work?
When you're rich.
[dead]
The system wants to destroy creativity and humans. The system wants artists and writers to depend on the patronage of the ultra rich who steal their output.
We are back to feudal society, except that monarchs or their advisers at least had taste as opposed to the Nouveau riche.