Lexis especially makes even scraping public state laws and admin codes difficult, which is extra frustrating because they are the legal publisher of record in a number of states.
I've been considering trying to launch an OpenStates style scraper project for US laws and admin codes, but haven't had the time to attack 100 more scrapers. Even with AI help, the volume is significant.
Who will maintain the git repo per state [1] [2]? There is value in a pipeline that continually ingests this data from various sources and pushes it into the Internet Archive, but if you wish to treat it as authoritative, it must have a human minding it, because of entropy and decay. Even the Python Software Foundation has a budget of ~$5M/year. Hence my openlaws.us example.
If it was as easy as writing a scraper and dumping it all in a bucket or repo, it'd already be done. It's just the usual thankless hard work over time grind.
I know a thing or two about that, 2,400 commits to the scrapers powering openstates over the past 9 years.
Even with openstates, we have an API but don't "just" dump the bills to git for legacy nerd reasons.
The nice thing about laws is that the host websites (or PDFs) don't change templates _that_ often, so generally you can rescrape quarterly (or in some states, annually) without a ton of maintenance. With administrative codes you need to scrape more often, but the websites are still pretty stable.
The downside is that codes in particular are often big, so a single scrape might need to make 20,000 or more requests, so you have to be very careful about rate limiting and proxies, which goes to my original point that it sucks that accessing this stuff is such a mess.
Also the states themselves can’t update their own laws. In Nevada the code (NRS) from Nevada’s website is out of date. Very embarrassing imo and hard to get it to work bc AI can’t have a trusted source of data.
The answer is probaly embedded within the concept of codification of acts. Legislatures pass acts, which are kind of like diffs for statutory law. But there is no base document, just a series of diffs from the beginning. Somewhere along the way, someone did a lot of work to “codify” the law, and when you go look up 18 USC 1001, and then click “next,” you are taking advantage of the codification process.
But the person who did the codification has some rights thereto, meaning that while NV can post every act that passed the legislature, they can’t publish someone else’s codification of the statutes.
This matters very little because everyone just has Westlaw and no one uses the state legislature’s website to cite statutes.
The codification needs to become part of the process of passing acts. The government should be required to publish the updated code themselves along with any act that changed it. The whole concept that a commercial entity can have rights to the fully assembled text is terribly broken.
If anybody is worried about the jobs those businesses created, then tell them to pivot into publishing commented editions of the codes (add cross-references, references to relevant court decisions, etc.).
The codification happened hundreds of years ago, though.
But you could do it too! The Congressional Record is a thing, and it publishes all the acts of Congress, all the way back to the beginning.
The problem is that after you were done, the first thing someone would ask you is to cross-cite everything into the West Annotated code because no one else has your code and no one cares about it, because we all have Westlaw.
(Which publishes commented editions of the codes, with cross references, references to relevant court decisions, etc.)
It's all a little bit antiquated but it works fine. Someday it will change. I too thought it should work the way people are describing upthread when I was a computer guy but it is what it is.
I would imagine you either start with the first acts of your legislature and codify it from the beginning, or you start with some version of the code you figure you have rights to and go from there. It seems like it would be insane to do that job halfway, but that's not my area of expertise.
I have no idea whatsoever what is going on in Nevada.
I would argue it does matter because the public has to know all the law as ignorance of the law is no excuse. If Westlaw is limited or an unreasonable monetary burden on the populace (or possibly, depending on your argument, costs anything at all) then the argument kind of evaporates of how you can prosecute someone, because an essential part of law is that the party under criminal penalty must be put under clear notice of what is illegal.
IMO, this should also extend to opinions -- if there is precedent that guides what the law is, it needs to be publicly published free of charge so that the public is put under notice what the law is. (someone might mention something like PACER is free in small quantities, I would counter it would cost you a gazillion dollars to be fully informed of all the precedent that forms the full common law meaning of the laws.) This is especially important in mala prohibita crimes since there's no way to even guess through moral/ethical deduction.
> argument kind of evaporates of how you can prosecute someone, because an essential part of law is that the party under criminal penalty must be put under clear notice of what is illegal.
I reckon that’s why the sixth amendment exists but if you want to make a free PACER, go for it.
First, laws cannot be copyrighted. They're in the public domain.
Second, the case you link to was not about laws, but annotated law. The annotated version (OCGA) was written by LexisNexis, and all that work paid for by LexisNexis. Obviously, normally that would mean LexisNexis has the copyright on the annotations. This is unproblematic.
Another organization (Public.Resource.Org) posted the OCGA online. Georgia sued, arguing the annotations were the work of LexisNexis and therefore obviously copyrighted. But the problem was that Georgia had declared the annotations to be official.
The Supreme Court found this act of making them official made them essentially the law itself, and therefore uncopyrightable. Which meant LexisNexis had done all that expensive work for free, whoops.
So it's a weird edge case where Georgia should either not have accorded any special status to the annotations, or should have paid for annotations themselves if they wanted "official" ones, and made them public domain.
It’s not a weird edge case at all. For many specific areas like building codes, states have long delegated publication and “annotation” (essentially, writing down critical elements of the law that must be known to folks who want to obey the law) to private publishers. This isn’t literally “Georgia tried to copyright its laws” but it’s close enough to be morally the same. I remember being incensed about this in the late 1990s and I’m glad the wheel of justice (finally) turned.
ETA: A critical tell here are the words “The State of Georgia sued.” Georgia essentially tried to copyright its laws by having a private firm do the copyrighting, and they didn’t even make much effort to hide behind this fiction, since they were the plaintiff. Kudos to the courts for seeing past this low-effort obfuscation.
I’d guess Lexis did that to itself. Usually the “deal” is that West or Lexis provides codification and reporting services for zero dollars, if they are named the official printer.
So the Legislature doesn’t have to maintain and oversee their own nest of troublesome legal pedants, and picks up a few contributions from legal publishing “entrepreneurs.”
By making the Annotated code official, it meant that anyone looking to prove what a particular law says in court would have to get it from the expensive, $412 hardback book, not the free version. I’d guess Lexis asked for that provision as part of its deal with Georgia.
How does one decide what should be in the public domain for the good of society and what should be commercialized? These are a couple of examples that I learned recently
1. In the UK, Royal Mail owns the postal addresses data. I was looking at UK's open datasets - apparently lot of datasets that have addresses can't be used without paying Royal Mail. There are some exceptions - but I am no lawyer. It is depressing to learn that Royal Mail is no longer a public institution, it was sold against public will by the UK government to a private entity, and sold again and as of last year it is owned by a Czech billionaire. Similarly, Canadian postal code database is also not free.
2. CPT code descriptions are owned by AMA (apparently they're super litigious?). Sure they took the time to write them, they should be compensated - but imagine how many interesting projects can be built if this data was freely available
On one hand, multi Billion dollar companies like Bloomberg exist, thanks to free and open data. But also things that should be free (dictionaries, postal codes etc) aren't.
It's not a question of "how do you decide which public standards should be freely accessible". That's easy: all of them. The public benefits more from freely accessible standards whether they're building codes, legal codes, ISO standards, or HDMI. The effect of not having them publicly available is that people make-do without having read the standard and the public has no way to validate things against the standard afterwards.
The question we don't have an easy answer for is how to incentivise the people behind these things without locking their work behind paywalls? Compliance marks, homologation regimes, copyright, and other strategies all have their own downsides.
Do you think it’s good for UpToDate that OpenEvidence scrapes and paraphrases UpToDate and sells the same information in a GPT wrapper to make big investor bucks? I don’t know what the answer is. Go for it, tell me.
If you're trying to fix anything relating to public policy, politics, &c., without fixing this you are wasting your time. You will make a bigger difference adopting a local stretch of highway and picking up litter. It is maddening to see generation after generation learn this the hard way.
FLP's CourtListener RECAP hits hard, but it has no counterparts either worldwide or within the US. If it goes offline, democracy and rule of law across the entire world go offline with it. Federal statutes and regulations are a very small part of black letter law.
PACER is currently a "lesser concern" because of FLP, compared to state court dockets in backwards states like California and New York (affecting some 60 million people alone), though by no means solved.
And if by discouraging you mean encouraging people to adopt freeways and pick up litter, I definitely intended to set expectations about how much might be achieved without addressing fundamental free law movement issues.
We do need help on Wikipedia though, but that is more ancillary and general.
No counterparts worldwide? Here in Australia, all significant case law has been published on AustLII, which is freely accessible and searchable, since about 2000. I think CanLII and NZLII are equally comprehensive. Government law publishers provide free and authoritative access to all statutes. Lexis and Westlaw still make money by publishing commentary and summaries, but paywalled primary sources are an American problem.
AustLII, CanLII and NZLII are all clones of the original Legal Information Institute (LII) from Cornell University. [0]
Most Australian courts require or prefer that case law be cited using exact page numbers from the 'authorised reports', i.e. the ones published by Westlaw or LexisNexis.
> Having received my letter, Justice Souter notified the Reporter of the error, and at 535 U.S. i appears an erratum, directing that “Homer nodded” be inserted—eight or nine years after the opinion was delivered.
I would guess corrections so long after publishing are relatively rare - but if they're also marked as such, it seems a slight stretch to say it's still being "edited" 9 years later
I do a lot of IL criminal courts research and it's really, really fucking bad out there.
Transcripts: Multiple dollars per pages. Want that expedited? Multiply that amount by four. Don't know the court date? Can't get your transcript. Clerks put in the wrong date? Tough luck. Payment for those transcripts? Over Zelle because the court reporters themselves are contractors and get paid independently.
FOIA: IL Judicial system isn't FOIAable. There's a "data request" process that you can go down, but last time I tried by requesting through the chief judge of Cook County, they told me to request the data from the Clerk of Courts. The Clerk of Courts told me I need to speak to the chief judge's office. That went on for 6 months. The docket-level data I've seen from them is... beyond indecipherable. Literally never seen a dataset so indecipherable in my life -- a combination of esoteric codes, hidden encoding schemes, and length-delimited lines.
Court docs: Gotta go downtown if you're not a lawyer or if you're not a "criminal justice partner" to get any, then you have to print them (don't print past 4 or you'll get yelled at). Some journalists have access, but the IL Supreme Court's policies disallow journalists from getting it. So we have a system where only some journalists get access to court documents and fuck everyone else, I guess.
Civil Asset Forfeiture cases: the courts and state's attorneys do everything in their part to separate it from the initiating criminal case. So peoples' cars are being taken by the cops under dubious constitutional grounds and they can't get it back until the criminal case is over. And I've yet to find a consistent way to figure out the initiating criminal case tied to a civil asset forfeiture. I've yet to find anything like a policy doc that says it's intentional to obfuscate the criminal case from the civil case, but I can't think of any reason why it would be like this if it wasn't intentional. FOIA through the State's Attorney? Nah, they don't respond, or they give nonsense denials that require litigation.
I could go on, but. Fuck. It's bad out there and it hides so much suffering. I highly encourage folk to give court watching a try. You'll be shocked how much you'll learn.
Consider reaching out to https://www.reclaimtherecords.org/ if any headway can be made legally on these scenarios. They are aggressive, highly recommend. If the legal scaffolding exists to enable FOIA requests, https://www.muckrock.com/ can then do the rest.
Reclaim the Records and I do a loooot of the same work but it's separate enough that we don't really interact much. But I'm a big, big fan of their work and have had a couple light chats with the founder a couple times. She's great and yeah, wonderfully aggressive. Her work's the sort of foundational work that enables so much.
Muckrock and I go back quite a long way! Sending about 200 requests through them later this month :)
I'll tip my hand a bit, I suggested Reclaim the Records because while I am time poor, if they take up fighting anything you suggested, I can send a directed donation towards that work. If you have someone else in mind who will take on these legal challenges in Illinois, I am interested.
I've been frustrated by this stuff for many years, and come to the sad conclusion that corruption in American public life is simply endemic. Lawmakers could and should prioritize the public availability of public legislative and judicial proceedings, but habitually choose not to, preferring obfuscation over clarity.
I buy into Plato's concept (sometimes attributed to Tactitus) that 'the more corrupt the state, the more numerous the laws.'
> Transcripts: Multiple dollars per pages. Want that expedited? Multiply that amount by four. Don't know the court date? Can't get your transcript. Clerks put in the wrong date? Tough luck. Payment for those transcripts? Over Zelle because the court reporters themselves are contractors and get paid independently.
Is this because a person has to literally transcribe the stenographer's notes into plain English? Is it expensive because it is labor intensive?
Yes and yes. I'm sure there's historical or possibly ethical reasons for it being this way, but I don't understand why it's not the clerk of court's job. Something to figure out!
Over here in Sweden they justify the fees for printed cases by the paper cost. Cheaper than the number you mentioned but still quite costly if you want all nontrivial verdicts. And quite silly when they deliver pdfs..
Then every single case needs to have all PII redacted before it is made searchable, but that is a different story.
I assume there's some reason you can't, but it'd be nice if one could simply declare that the civil forfeiture case is invalid when no criminal case exists, and place the burden of proof back where it belongs.
Family court is even worse. You can't even know what's illegal for your kid to do because the civil standards on things like 'neglect' are incredibly nebulous and left for agencies like CPS and family law judges to review, so to find out how they're actually interpreted you would need to see cases and how they played out. Those are ~all wrapped up in secret court hearings.
So basically you have no fucking idea what the law actually is, how it's being enforced, what will happen to you if you don't follow their interpretation, and god forbid you wind up there they can use a lot of methods to keep you from talking about it.
You seem to have faith in the courts still. It was not a bribe. Here's a short exchange between me and the court reporter who did indeed sent me the transcripts after payment:
"Out of curiosity, is it normal to pay court reporters over venmo like this? To be honest I was pretty surprised when I heard this. And funny enough when I asked folk on twitter, a scammer tried to "help out" with some "scam recovery" service for $500."
"That's hilarious! Since the pandemic, it really took off because we were working remotely, and some of the Cook County judges and reporters remain remote to this day and may be permanently working on Zoom. I work at 26th Street so we've worked through most of the pandemic in person, but the courthouse was not open to the public. Zelle is a way you can get payment to us quickly to get your transcript faster.
Most private attorneys use Zelle now because they want the reporter to start transcription sooner rather than later, but we also accept checks or if you prefer to send a check in the mail, you can do that. Very rarely does an attorney come into our office these days.
It's just a matter of preference."
Its crazy insane that citizens who are expected to follow the law cannot have access to law ? what does that even mean ? Do we at-least know some high level numbers on how many law/opinions/cases texts are not freely available to citizens ?
Tangentially the State of Florida will charge you $75/day for your time in jail, regardless of whether you were found not guilty, the case was dismissed or charges dropped completely.
And not paying those fees is a second degree felony.
If someone wanted to do business research for Mexico or any other LATAM country, while in the US and using English source data what resources would you look to?
Depends on what kind of business research you need. But there are at least collections of all tax treaties that exist available from a single source somewhere. I don't remember what the source is off the top of my head unfortunately.
"The fees the federal government charges for court documents are significant. PACER charges users $0.10 for every page users search and access, and $30 per name or item searched plus $0.10 per page per document per month, albeit with a max of $3 per individual document."
It's funny how these numbers sound very reasonable on their face, especially if used for human review.
But if you consider programmatic searches, 10 cents per page is a lot. Consider the CPU cost of cycling through each pdf, assume 1000 letters, that's 1Khz which is like a billionth of a cent, in terms of electricity and hardware ammortization.
I'd bet that this incentivizes a 'hoarding' and reuse of information, rather than a "query when necessary" practice, which would in fact give place to monopolies like OP argues.
A newcomer legal analysis company would have to pay, say 200$ for research from their first customer. If they save those documents, then it is unlikely that their second customer will benefit from those documents.
However huge firms would make a business out of the byproduct of searches, they spend 200$ on documents, and they serve a client, but they add those documents to their own internal database, which of course has costs much lower than the nominal fees stipulated by the federal steward (PACER).
An argument could be that you can't eliminate the fees because there's costs that need to be paid. But isn't that what taxes are typically for? Especially corporate income tax. The costs are probably a nominal fee to avoid overbearing systems, similar to franchise taxes and incorporation clerical fees.
I don't think the idea of charging for documents is bad, who said that the system would need to be adopted for huge data hungry systems, I'm all for giving human analysis an advantage instead of adapting systems to play into the data munching AI fad. I'd rather see balancing that rewards human analysis rather than bringing data munching to the masses.
Your point makes sense, but for law-abiding citizens, it’s hard to justify: people are expected to understand and comply with the law, yet the practical record of how it’s enforced (dockets/filings) is paywalled. If the goal is to fund the system or deter abuse, that’s what general taxation and targeted anti-scraping limits are for, metering “public” records mostly punishes ordinary users and small competitors, not heavy institutional users.
I'd be interested to see some information on LLMs being used for legal research. I expect it would not go well. I don't imagine it's just a simple matter of summarizing the text that's there - one has to interpret it with legalese, put it in context of the jurisdiction, look up related cases, etc.
Lexis especially makes even scraping public state laws and admin codes difficult, which is extra frustrating because they are the legal publisher of record in a number of states.
I've been considering trying to launch an OpenStates style scraper project for US laws and admin codes, but haven't had the time to attack 100 more scrapers. Even with AI help, the volume is significant.
https://openlaws.us/
https://public.resource.org/
https://www.eff.org/press/releases/publicresourceorg-can-kee...
https://blog.archive.org/2022/10/19/2022-internet-archive-he...
Neither openlaws nor public.resource actually let you just pull the laws in a common format (or the scrapers) as far as I can tell?
I was thinking something more along the lines of a git repo per state.
Who will maintain the git repo per state [1] [2]? There is value in a pipeline that continually ingests this data from various sources and pushes it into the Internet Archive, but if you wish to treat it as authoritative, it must have a human minding it, because of entropy and decay. Even the Python Software Foundation has a budget of ~$5M/year. Hence my openlaws.us example.
If it was as easy as writing a scraper and dumping it all in a bucket or repo, it'd already be done. It's just the usual thankless hard work over time grind.
[1] https://xkcd.com/2347/
[2] https://en.wikipedia.org/wiki/Free-rider_problem
I know a thing or two about that, 2,400 commits to the scrapers powering openstates over the past 9 years.
Even with openstates, we have an API but don't "just" dump the bills to git for legacy nerd reasons.
The nice thing about laws is that the host websites (or PDFs) don't change templates _that_ often, so generally you can rescrape quarterly (or in some states, annually) without a ton of maintenance. With administrative codes you need to scrape more often, but the websites are still pretty stable.
The downside is that codes in particular are often big, so a single scrape might need to make 20,000 or more requests, so you have to be very careful about rate limiting and proxies, which goes to my original point that it sucks that accessing this stuff is such a mess.
Fair, I stand corrected. Thanks for your work. All Openness efforts are welcome.
Also the states themselves can’t update their own laws. In Nevada the code (NRS) from Nevada’s website is out of date. Very embarrassing imo and hard to get it to work bc AI can’t have a trusted source of data.
Why can't Nevada update its own laws on its own website?
The answer is probaly embedded within the concept of codification of acts. Legislatures pass acts, which are kind of like diffs for statutory law. But there is no base document, just a series of diffs from the beginning. Somewhere along the way, someone did a lot of work to “codify” the law, and when you go look up 18 USC 1001, and then click “next,” you are taking advantage of the codification process.
But the person who did the codification has some rights thereto, meaning that while NV can post every act that passed the legislature, they can’t publish someone else’s codification of the statutes.
This matters very little because everyone just has Westlaw and no one uses the state legislature’s website to cite statutes.
The codification needs to become part of the process of passing acts. The government should be required to publish the updated code themselves along with any act that changed it. The whole concept that a commercial entity can have rights to the fully assembled text is terribly broken.
If anybody is worried about the jobs those businesses created, then tell them to pivot into publishing commented editions of the codes (add cross-references, references to relevant court decisions, etc.).
The codification happened hundreds of years ago, though.
But you could do it too! The Congressional Record is a thing, and it publishes all the acts of Congress, all the way back to the beginning.
The problem is that after you were done, the first thing someone would ask you is to cross-cite everything into the West Annotated code because no one else has your code and no one cares about it, because we all have Westlaw.
(Which publishes commented editions of the codes, with cross references, references to relevant court decisions, etc.)
It's all a little bit antiquated but it works fine. Someday it will change. I too thought it should work the way people are describing upthread when I was a computer guy but it is what it is.
So are Nevada lawmakers using Westlaw as there reference for the base document to build "edits" off of?
I would imagine you either start with the first acts of your legislature and codify it from the beginning, or you start with some version of the code you figure you have rights to and go from there. It seems like it would be insane to do that job halfway, but that's not my area of expertise.
I have no idea whatsoever what is going on in Nevada.
I would argue it does matter because the public has to know all the law as ignorance of the law is no excuse. If Westlaw is limited or an unreasonable monetary burden on the populace (or possibly, depending on your argument, costs anything at all) then the argument kind of evaporates of how you can prosecute someone, because an essential part of law is that the party under criminal penalty must be put under clear notice of what is illegal.
IMO, this should also extend to opinions -- if there is precedent that guides what the law is, it needs to be publicly published free of charge so that the public is put under notice what the law is. (someone might mention something like PACER is free in small quantities, I would counter it would cost you a gazillion dollars to be fully informed of all the precedent that forms the full common law meaning of the laws.) This is especially important in mala prohibita crimes since there's no way to even guess through moral/ethical deduction.
> argument kind of evaporates of how you can prosecute someone, because an essential part of law is that the party under criminal penalty must be put under clear notice of what is illegal.
I reckon that’s why the sixth amendment exists but if you want to make a free PACER, go for it.
Google scholar tried and gave up
The state of Georgia tried to copyright its public written laws. They lost.
https://www.bfvlaw.com/supreme-court-rules-georgia-cannot-cl...
I'm not sure what point you're trying to make?
First, laws cannot be copyrighted. They're in the public domain.
Second, the case you link to was not about laws, but annotated law. The annotated version (OCGA) was written by LexisNexis, and all that work paid for by LexisNexis. Obviously, normally that would mean LexisNexis has the copyright on the annotations. This is unproblematic.
Another organization (Public.Resource.Org) posted the OCGA online. Georgia sued, arguing the annotations were the work of LexisNexis and therefore obviously copyrighted. But the problem was that Georgia had declared the annotations to be official.
The Supreme Court found this act of making them official made them essentially the law itself, and therefore uncopyrightable. Which meant LexisNexis had done all that expensive work for free, whoops.
So it's a weird edge case where Georgia should either not have accorded any special status to the annotations, or should have paid for annotations themselves if they wanted "official" ones, and made them public domain.
It’s not a weird edge case at all. For many specific areas like building codes, states have long delegated publication and “annotation” (essentially, writing down critical elements of the law that must be known to folks who want to obey the law) to private publishers. This isn’t literally “Georgia tried to copyright its laws” but it’s close enough to be morally the same. I remember being incensed about this in the late 1990s and I’m glad the wheel of justice (finally) turned.
ETA: A critical tell here are the words “The State of Georgia sued.” Georgia essentially tried to copyright its laws by having a private firm do the copyrighting, and they didn’t even make much effort to hide behind this fiction, since they were the plaintiff. Kudos to the courts for seeing past this low-effort obfuscation.
I’d guess Lexis did that to itself. Usually the “deal” is that West or Lexis provides codification and reporting services for zero dollars, if they are named the official printer.
So the Legislature doesn’t have to maintain and oversee their own nest of troublesome legal pedants, and picks up a few contributions from legal publishing “entrepreneurs.”
By making the Annotated code official, it meant that anyone looking to prove what a particular law says in court would have to get it from the expensive, $412 hardback book, not the free version. I’d guess Lexis asked for that provision as part of its deal with Georgia.
How does one decide what should be in the public domain for the good of society and what should be commercialized? These are a couple of examples that I learned recently
1. In the UK, Royal Mail owns the postal addresses data. I was looking at UK's open datasets - apparently lot of datasets that have addresses can't be used without paying Royal Mail. There are some exceptions - but I am no lawyer. It is depressing to learn that Royal Mail is no longer a public institution, it was sold against public will by the UK government to a private entity, and sold again and as of last year it is owned by a Czech billionaire. Similarly, Canadian postal code database is also not free.
2. CPT code descriptions are owned by AMA (apparently they're super litigious?). Sure they took the time to write them, they should be compensated - but imagine how many interesting projects can be built if this data was freely available
On one hand, multi Billion dollar companies like Bloomberg exist, thanks to free and open data. But also things that should be free (dictionaries, postal codes etc) aren't.
It's not a question of "how do you decide which public standards should be freely accessible". That's easy: all of them. The public benefits more from freely accessible standards whether they're building codes, legal codes, ISO standards, or HDMI. The effect of not having them publicly available is that people make-do without having read the standard and the public has no way to validate things against the standard afterwards.
The question we don't have an easy answer for is how to incentivise the people behind these things without locking their work behind paywalls? Compliance marks, homologation regimes, copyright, and other strategies all have their own downsides.
Do you think it’s good for UpToDate that OpenEvidence scrapes and paraphrases UpToDate and sells the same information in a GPT wrapper to make big investor bucks? I don’t know what the answer is. Go for it, tell me.
If you're trying to fix anything relating to public policy, politics, &c., without fixing this you are wasting your time. You will make a bigger difference adopting a local stretch of highway and picking up litter. It is maddening to see generation after generation learn this the hard way.
FLP's CourtListener RECAP hits hard, but it has no counterparts either worldwide or within the US. If it goes offline, democracy and rule of law across the entire world go offline with it. Federal statutes and regulations are a very small part of black letter law.
There is little worse action you can take than discouraging others from doing anything.
Far worse problems than PACER have been solved. Just because you've been unsuccessful doesn't necessarily mean others will be.
PACER is currently a "lesser concern" because of FLP, compared to state court dockets in backwards states like California and New York (affecting some 60 million people alone), though by no means solved.
And if by discouraging you mean encouraging people to adopt freeways and pick up litter, I definitely intended to set expectations about how much might be achieved without addressing fundamental free law movement issues.
We do need help on Wikipedia though, but that is more ancillary and general.
No counterparts worldwide? Here in Australia, all significant case law has been published on AustLII, which is freely accessible and searchable, since about 2000. I think CanLII and NZLII are equally comprehensive. Government law publishers provide free and authoritative access to all statutes. Lexis and Westlaw still make money by publishing commentary and summaries, but paywalled primary sources are an American problem.
AustLII, CanLII and NZLII are all clones of the original Legal Information Institute (LII) from Cornell University. [0]
Most Australian courts require or prefer that case law be cited using exact page numbers from the 'authorised reports', i.e. the ones published by Westlaw or LexisNexis.
[0] https://www.law.cornell.edu
Free Law project is amazing and the best org to counter this duopoly. Please check them out if you haven’t!
https://www.courtlistener.com/ is run by freelaw.
PlainSite is another project worth checking out. I have come across both when exploring in the space.
[1]: https://free.law/
[2]: https://www.plainsite.org/
Here's a great X thread with additional reporting giving an inside look on this: https://x.com/ROSSIntel/status/2006429363034824709?s=20
And this doesn't include the messiness in the decisions themselves. Judges often edit the decisions for some time after they are published.
For example, Souter was editing his published opinion in Lee v. Weisman nine years after the case was decided.
https://balkin.blogspot.com/2025/05/a-justice-souter-anecdot...
> Having received my letter, Justice Souter notified the Reporter of the error, and at 535 U.S. i appears an erratum, directing that “Homer nodded” be inserted—eight or nine years after the opinion was delivered.
I would guess corrections so long after publishing are relatively rare - but if they're also marked as such, it seems a slight stretch to say it's still being "edited" 9 years later
I do a lot of IL criminal courts research and it's really, really fucking bad out there.
Transcripts: Multiple dollars per pages. Want that expedited? Multiply that amount by four. Don't know the court date? Can't get your transcript. Clerks put in the wrong date? Tough luck. Payment for those transcripts? Over Zelle because the court reporters themselves are contractors and get paid independently.
FOIA: IL Judicial system isn't FOIAable. There's a "data request" process that you can go down, but last time I tried by requesting through the chief judge of Cook County, they told me to request the data from the Clerk of Courts. The Clerk of Courts told me I need to speak to the chief judge's office. That went on for 6 months. The docket-level data I've seen from them is... beyond indecipherable. Literally never seen a dataset so indecipherable in my life -- a combination of esoteric codes, hidden encoding schemes, and length-delimited lines.
Court docs: Gotta go downtown if you're not a lawyer or if you're not a "criminal justice partner" to get any, then you have to print them (don't print past 4 or you'll get yelled at). Some journalists have access, but the IL Supreme Court's policies disallow journalists from getting it. So we have a system where only some journalists get access to court documents and fuck everyone else, I guess.
Civil Asset Forfeiture cases: the courts and state's attorneys do everything in their part to separate it from the initiating criminal case. So peoples' cars are being taken by the cops under dubious constitutional grounds and they can't get it back until the criminal case is over. And I've yet to find a consistent way to figure out the initiating criminal case tied to a civil asset forfeiture. I've yet to find anything like a policy doc that says it's intentional to obfuscate the criminal case from the civil case, but I can't think of any reason why it would be like this if it wasn't intentional. FOIA through the State's Attorney? Nah, they don't respond, or they give nonsense denials that require litigation.
I could go on, but. Fuck. It's bad out there and it hides so much suffering. I highly encourage folk to give court watching a try. You'll be shocked how much you'll learn.
Consider reaching out to https://www.reclaimtherecords.org/ if any headway can be made legally on these scenarios. They are aggressive, highly recommend. If the legal scaffolding exists to enable FOIA requests, https://www.muckrock.com/ can then do the rest.
Great recs.
Reclaim the Records and I do a loooot of the same work but it's separate enough that we don't really interact much. But I'm a big, big fan of their work and have had a couple light chats with the founder a couple times. She's great and yeah, wonderfully aggressive. Her work's the sort of foundational work that enables so much.
Muckrock and I go back quite a long way! Sending about 200 requests through them later this month :)
I'll tip my hand a bit, I suggested Reclaim the Records because while I am time poor, if they take up fighting anything you suggested, I can send a directed donation towards that work. If you have someone else in mind who will take on these legal challenges in Illinois, I am interested.
Love the Muckrock love, they are a treasure.
Nice, thank you. It's really appreciated. For Chicagoland:
Lucy Parsons Labs: https://www.lucyparsonslabs.com/donate
Chicago Appleseed: https://www.chicagoappleseed.org/ways-to-give/
Invisible Institute: https://invisible.institute/donate
Unraveled Press: https://unraveledpress.com/support-unraveled/
All four are great. One is my employer. Unraveled Press would benefit the most from donations.
Meth labs of democracy
I've been frustrated by this stuff for many years, and come to the sad conclusion that corruption in American public life is simply endemic. Lawmakers could and should prioritize the public availability of public legislative and judicial proceedings, but habitually choose not to, preferring obfuscation over clarity.
I buy into Plato's concept (sometimes attributed to Tactitus) that 'the more corrupt the state, the more numerous the laws.'
> Transcripts: Multiple dollars per pages. Want that expedited? Multiply that amount by four. Don't know the court date? Can't get your transcript. Clerks put in the wrong date? Tough luck. Payment for those transcripts? Over Zelle because the court reporters themselves are contractors and get paid independently.
Is this because a person has to literally transcribe the stenographer's notes into plain English? Is it expensive because it is labor intensive?
Yes and yes. I'm sure there's historical or possibly ethical reasons for it being this way, but I don't understand why it's not the clerk of court's job. Something to figure out!
Over here in Sweden they justify the fees for printed cases by the paper cost. Cheaper than the number you mentioned but still quite costly if you want all nontrivial verdicts. And quite silly when they deliver pdfs..
Then every single case needs to have all PII redacted before it is made searchable, but that is a different story.
I don't know much about stenography, but that seems like something that would be trivial for a computer to do.
And certainly something that would only have to be done once...
The human should be left in the loop, full stop. Language is hard and court rooms can be chaotic places and you want the transcript to be clean.
(disability accomodations is a different conversation altogether)
I assume there's some reason you can't, but it'd be nice if one could simply declare that the civil forfeiture case is invalid when no criminal case exists, and place the burden of proof back where it belongs.
Friend, I highly highly recommend you do some court watching. The courts.. really, really, really do not work that way.
Pal, I think the immediate disclaimer at the very beginning is enough to show that I'm being wistful.
My point is that -- the criminal case exists. Please try out court watching!
Family court is even worse. You can't even know what's illegal for your kid to do because the civil standards on things like 'neglect' are incredibly nebulous and left for agencies like CPS and family law judges to review, so to find out how they're actually interpreted you would need to see cases and how they played out. Those are ~all wrapped up in secret court hearings.
So basically you have no fucking idea what the law actually is, how it's being enforced, what will happen to you if you don't follow their interpretation, and god forbid you wind up there they can use a lot of methods to keep you from talking about it.
>"Tough luck. Payment for those transcripts? Over Zelle because the court reporters themselves are contractors and get paid independently."
That doesn't sound right, are you sure that wasn't a bribe?
You seem to have faith in the courts still. It was not a bribe. Here's a short exchange between me and the court reporter who did indeed sent me the transcripts after payment:
It sounds unbelievable but it's true! We do it in California too, at least in my county's court.
I wonder if anything would happen from a law stating that government (judges, clerks, legislators, etc.) may not pay less for access than the public.
Could it align incentives for future reform, or would it just blow to the bottom line of these companies?
Its crazy insane that citizens who are expected to follow the law cannot have access to law ? what does that even mean ? Do we at-least know some high level numbers on how many law/opinions/cases texts are not freely available to citizens ?
Tangentially the State of Florida will charge you $75/day for your time in jail, regardless of whether you were found not guilty, the case was dismissed or charges dropped completely.
And not paying those fees is a second degree felony.
If someone wanted to do business research for Mexico or any other LATAM country, while in the US and using English source data what resources would you look to?
Depends on what kind of business research you need. But there are at least collections of all tax treaties that exist available from a single source somewhere. I don't remember what the source is off the top of my head unfortunately.
Thank you. If it ever comes to you please let me know.
"The fees the federal government charges for court documents are significant. PACER charges users $0.10 for every page users search and access, and $30 per name or item searched plus $0.10 per page per document per month, albeit with a max of $3 per individual document."
It's funny how these numbers sound very reasonable on their face, especially if used for human review.
But if you consider programmatic searches, 10 cents per page is a lot. Consider the CPU cost of cycling through each pdf, assume 1000 letters, that's 1Khz which is like a billionth of a cent, in terms of electricity and hardware ammortization.
I'd bet that this incentivizes a 'hoarding' and reuse of information, rather than a "query when necessary" practice, which would in fact give place to monopolies like OP argues.
A newcomer legal analysis company would have to pay, say 200$ for research from their first customer. If they save those documents, then it is unlikely that their second customer will benefit from those documents.
However huge firms would make a business out of the byproduct of searches, they spend 200$ on documents, and they serve a client, but they add those documents to their own internal database, which of course has costs much lower than the nominal fees stipulated by the federal steward (PACER).
An argument could be that you can't eliminate the fees because there's costs that need to be paid. But isn't that what taxes are typically for? Especially corporate income tax. The costs are probably a nominal fee to avoid overbearing systems, similar to franchise taxes and incorporation clerical fees.
I don't think the idea of charging for documents is bad, who said that the system would need to be adopted for huge data hungry systems, I'm all for giving human analysis an advantage instead of adapting systems to play into the data munching AI fad. I'd rather see balancing that rewards human analysis rather than bringing data munching to the masses.
Your point makes sense, but for law-abiding citizens, it’s hard to justify: people are expected to understand and comply with the law, yet the practical record of how it’s enforced (dockets/filings) is paywalled. If the goal is to fund the system or deter abuse, that’s what general taxation and targeted anti-scraping limits are for, metering “public” records mostly punishes ordinary users and small competitors, not heavy institutional users.
On a tangential note, this seems ripe for LLM analysis/summarization to digest the legal code down so mere mortals could better understand it.
It would also be great for pointing out all the loopholes that have been built in by our owners.
There is, but that depends a whole lot on access to content that can be hard to get - both case data and books in law are essential.
Riches are in the niches.
One would need the data.
I'd be interested to see some information on LLMs being used for legal research. I expect it would not go well. I don't imagine it's just a simple matter of summarizing the text that's there - one has to interpret it with legalese, put it in context of the jurisdiction, look up related cases, etc.