When "detectors" first started popping up all over the place, all of them rated the declaration of independence as 100% AI written, so... Yeah, these things just don't work. And what's even more dangerous is that people that don't understand how any of it works use these tools, and accuse people of using AI, sometimes with grave consequences. Students have been through this, at all levels of education.
In fairness, the original crop of detectors were based on perplexity scores and were entirely useless (famously, the declaration of independence often came back as 100% AI generated).
I'm not convinced we'll ever have full-proof detectors, and certainly the false-positive rate will make them irresponsible for accusations of intellectual/academic fraud, I do think that LLMs are easy for folks to sniff out on average so I imagine it's possible to detect many instances.
Pangram's detector is anecdotally very accurate in my tests. This detector appears to be fine-tuned on a very small dataset (200 papers per subject), and suspect the problem might be in part that.
I replied to a comment with a structured informal proof on HN.
A well established user here was adamant that I used AI because apparently humans never ever wrote proofs.
This was a while ago. Any well crafted human output is now being dismissively cast as AI if the reader is challenged by the output intellectually/politically.
I’ve been accused of being a bot myself. I’ve started leaving some incorrect autocorrects in, hopping it would stave off people for whom it seems making accusations of being a bot is soothing or a coping mechanism when they’re confronted with information that does not fit into their world view or perception of self.
They use a "threshold calibrated so pre-ChatGPT papers flag at 0.4%", so these things do work most of the time. It also means that there are known false positives, so for any given paper, scoring above the threshold isn't irrefutable proof of AI usage. But for things like estimating the overall proportion of AI writing, you only need to be correct on average, so individual false positives don't matter much.
It would be interesting to see how that .4% varies across fields on arXiv. I imagine some areas influenced LLM writing much more than others and are more susceptible to false positives. Anyway it's a little strange to me that these detectors do so poorly (at least by what I hear on this site). There is as much labeled data as you would need to train on pre-LLM human vs LLM text. If their out of sample errors are as good as has been claimed, the only methodological flaw I can think of is that post-LLM human text and LLM text have been converging.
"Any attempt to build AI generated content (deep fake) detection systems is flawed, since the outputs of such a system may be used to train an even better fake data generator. This leads to an equilibrium state of digital uncertainty: nothing in the digital realm can be deemed as real anymore - only as digital. I do not care if a digital artifact is human or AI made - I only care if it is useful to me. Useful content is on point, factual and at best surprising (teaches something new)." - https://seanpedersen.github.io/posts/digital-uncertainty/
I scored the full text of 12,750 arXiv papers from 2021 through 2026 to find out how many of these get flagged as machine written and how much it increased since the release of chatGPT. I purposely tuned the detector to avoid false positives. My detection rate pre chatGPT is around .4% for that reason.
The biggest results: in Jan of 2026 about 39% of papers got flagged as AI written. In computer science speicifcally the peak was at 65%. Mathematics barely moved away from 0.7%, though the proof heavy math texts might just not get picked up by the detector properly.
All this is a detector estimate of a statistical signal and not a proof any given author used AI. Machine written can also mean heavy AI-assisted editing.
* pre-chatGPT is not an effective control because language evolves. In the arxiv corpus in particular there are "fashions" in research depending on what gets funded lately, not to mention many new words and topics not invented before a given paper.
* In general, detecting AI from content seems difficult as humans write like they read. To the extent there are unique factors recognizable as AI and to the extent humans read them, they will eventually incorporate them into their writing style. Accordingly, you'd need to model a rolling window of "AI tells" that decay at some rate.
This is a pretty stunning result. The time series looks really convincing. Is the way the detector itself is trained orthogonal to this or could there be some "leakage" in that the pre-chatgpt text is in the (positive) training data?
My honest first impressions, since I think the project is well intended: This writeup is itself AI, and I would venture to call it slop. The Calibration section is borderline uninterpretable, and I challenge any non-author who claims to understand it to answer some basic peer review questions about it.
I’ve heard good things about Pangram but it’s trivially easy to fool it. I generated text using Opus 4.8 and then humanized it using Grammarly. Pangram determined it 100% human-written. Tried it numerous times with the same result. GPTZero claimed with 80% confidence that it was human-written but AI polished. Nothing in that text was human-written.
Could you share some pre LLM false positives? Would be interesting to see what is tripping the detection. Did a tiny fraction of authors write like LLMs, before LLMs?
this is neat! is the model available somewhere for local execution? or even a lookup table with your results for all arXiv pre-print codes? I want to run it on lots of pre-prints and I don't want to kill your server
There some real game theory mechanics at play when it comes to LLM usage in corporations. Devs are cranking out superficially superior code and documentation by just aiming the Claude Code fire hose at everything they can. Leadership encourages this because from what they can tell, there is no downside.
It's hard to say if this code is structurally better or worse than before, but it's certainly voluminous and as far as leadership could ever tell, with their flawed metrics, that is all that matters. It will be years before we figure out if this is a good idea and worth the cognitive atrophy.
Anyone not using LLMs all day is just not going to be as prolific. I can't imagine that the same factors aren't at play in the scientific research community where it's all about how much you can publish.
>I can't imagine that the same factors aren't at play in the scientific research community
I can't speak for the entire scientific research community, but I can say that for basic biomedical research (not clinical research), LLMs are mostly ignored. They simply don't have the ability to parse most raw wet lab data.
This obviously has not stopped the creation of a legion of startups, new Asst Profs, etc, claiming that they are using "AI" to crack new frontiers. In practice, the most effective of these use machine learning, rather than LLMs.
> It's hard to say if this code is structurally better or worse than before, but it's certainly voluminous and as far as leadership could ever tell, with their flawed metrics, that is all that matters.
Hah. I've been working with ChatGPT-5.6 Sol and a lot of guidance to port a particular geometry from 2800 lines of Python-scripted Rhino3D (basically a custom parametric CAD kernel in there, plus use of it) to Python-scripted OCCT/FreeCAD (an existing CAD kernel) and it's up to 36,000 lines and only about half way there. And I've been setting goals and targets for duplication minimization, code size reduction, etc. The results are fine for my purposes, but if I put a positive value on "voluminous" I'd be super impressed; with my actual negative value assignment on lines of code, this is more "absolute crap but still useful to me."
I think "absolute crap but still useful to me" is a pretty high value and not worth neglecting, but I'm definitely scared by leadership who sees the toilet overflowng and assumes that means that value has been maximized.
> I think "absolute crap but still useful to me" is a pretty high value and not worth neglecting
If I extrapolate this example to my professional life, this code now manages millions of dollars, a single mistake can wipe it all out, it has to be maintained by 5 other engineers and understood by 5 other domain experts.
That's a real risk, yes. There's definitely a difference between cases where the code is the artifact being built, and cases where the code is being built to generate an artifact that is independently valuable and assessable. I'm happily in the latter case -- but I definitely share your fear of those who think current LLMs are valuable in the former case.
> It's hard to say if this code is structurally better or worse than before,
Huh?
But that's easy to tell?
It's better then incompetents code, worse then a motivated average dev... But good enough hence the real question is value aka time& money invested/quality.
That's much harder to tell, and I currently think opus/fable generated code is decent enough to be safely in favor - at least on subscription
> It's better then incompetents code, worse then a motivated average dev...
That's well put.
> But good enough hence the real question is value aka time& money invested/quality.
There's time invested SO FAR and time that will have to be invested to maintain it. In my experience, even with Fable, it's not there yet. It's the reason why it's easy to vibe code an app from scratch, but at some point when complexity significantly increases, the codebase becomes a mess.
> Leadership encourages this because from what they can tell, there is no downside.
This was my big fear before we saw price increases. Now I'm pinning all my hopes on AI being too expensive to justify further big corporate pushes. (Sigh.) I love having new tools, but I hate being pushed to use ______ tool to meet some managerial metric.
IMO, cognitive athrophy is the point. If humans become dependent on AI, those corporations will get enough leverage to change laws in their favor. For example, people don't want a huge data center in the middle of their city, but the corporations will say that the same people can't do anything without AI powered by this data center. That's why I think that making humans deficient and dependent on AI is the entire point.
Can you unpack that a bit? It produces measurably, meaningfully inferior code everywhere I see it in use.
> "leadership encourages this because from what they can tell, there is no downside"
As said 'leadership' I find this a bit puzzling. I'm seeing strong, quantifiable evidence of increasing churn, increasing incident count, and length of downtime from the date of our biggest push into GenAI, and I'm organizing efforts on my teams to mitigate those issues and actively reduce GenAI adoption.
If you mean my c-suite, you're mostly correct although they are already rumbling about seeing zero or negative ROI on GenAI investments.
> Anyone not using LLMs all day is just not going to be as prolific
Not sure if its just me, lately I have started feeling pretty offensive about the increased usage of the word. Its management not leadership by any means.
> "leadership encourages this because from what they can tell, there is no downside"
For most people in management its easier to pick the current set of slangs/abbreviation's, general trend and go with it. Understanding the details would take time, raise questions and no one in management has time or political capital to spend on it.
The code written by agents generally seems to reflect what you (the human driver) asked for. If you discuss the approach and architecture first, asking the right questions in the process, and then let it implement - the result is quite on point with the frontier models. Might need some minor touch-ups if agent missed some common conventions or guessed the expectations wrong (but again, salvageable with follow-up prompts). And when it comes to line-by-line logic within functions, I would argue that today's models are LESS likely to make a mistake in there than humans - especially if you cross-review with another model (e.g. "write with Claude, review with GPT").
As with all text-only AI detection schemes, I am concerned about the accuracy of the detection. I'm skeptical of the methodology, specifically the final join of the three detector scores---how can you be sure that that final step does not introduce any biases? There's no source available, so it's difficult to tell exactly how this works or reproduce the research.
I've also uploaded text samples from my own (unreleased) research from pre-LLM era, and it's seemingly scoring pretty high on the LLM-detection scores. On other papers, nearly every sentence is highlighted as red "machine-leaning," but that does not impact the score? Additionally, there are dramatic differences between the scores for identical text with and without LaTeX formatting, despite the fact that it should not matter.
The takeaway from this should be "it is difficult to detect generated text and we should be careful about accepting results simply because they confirm a hypothesis."
--
Relatedly, the text above scores as highly machine-written, despite the fact that I just wrote it with my human hands, I promise :)
I've looked at this same problem in an academic environment and have come to conclude that there is no way to reliably detect AI writing using only text. The reason for this is that no detector can take two identical inputs and classify one as synthetic (LLM) and one as organic (Human) and this situation can easily happen at the sentence or even paragraph level. Posed as a question, if a human writes a paragraph that happens to be exactly the same as a paragraph written by an LLM how do you classify that paragraph? There are a finite number of words and a finite way of combining them within an academic setting/field which means that we can't build a perfect classifier with just text. In practice, there are obvious tells and LLM-isms but these also change with time and each model has different tells so that even if a paper is full of LLM like writing we have no way to disambiguate between organic text that appears synthetic or synthetic text that appears organic.
A more interesting question, to me, is looking at a corpus of essays and analyzing how writing has changed with the introduction of LLMs. We can look at changes in vocabulary, linguistic features, style embeddings, regular embeddings, typos, errors, and references over time. When looked at in this way it is clear that academic writing has changed at the population level but what has led the change is harder to track down.
> There are a finite number of words and a finite way of combining them within an academic setting/field which means that we can't build a perfect classifier with just text.
Maybe for very short phrases, but otherwise I disagree. Phrasing very quickly runs into a combinatorial explosion. In the words of Noam Chomsky, "Virtually every sentence that a person utters or understands is a brand-new combination of words, appearing for the first time in the history of the universe."
In my opinion, the difficulty in LLM/human text discrimination isn't that a person might coincidentally write exactly the same text as an LLM would, but rather that 1) LLMs aren't hard locked to a single phrasing (so this is a tougher problem than matching to a single static document, e.g. plagiarism detection) and 2) text has relatively low information density, so you need quite a bit of it to gather enough data to run a statistical test with a reasonably narrow confidence interval.
> If a tool marks 40% of new papers as machine-written but also marks 20% of papers written before ChatGPT existed, the real story is the 20% nobody mentioned.
When 65% of the papers you read have the characteristics of being AI written, whether or not you use AI to write, your writing will be influenced by the AI style. I imagine this must be particularly the case for newbie researchers who are still developing their writing style
even if your writing is influenced by "the AI style," it would likely only show up in patches, a word or rhetorical flourish maybe, but AI-generated text doesn't "sound like AI" in patches, it does so consistently across the entirety of the output. it's very hard for a person to maintain the syntactical patterns of AI writing over an extended period of time unless they know exactly what those are -- probably more knowledge than any person currently has, really -- and are applying a level of detail to each clause comparable to forging a painting.
there's also the fact that "the AI style" has changed over time. for instance, the word "delve" is notorious as an "AI sign," which it was up until mid-2024, at which point it dropped off sharply and has now basically disappeared from LLM output. so if someone happened to pick that up due to reading it everywhere, their writing is now less characteristic of AI, not more.
I might be missing something but what’s the real story of the 20%.
To me it sounds like 1. Either your tool is just not that good and reliable as you thought, 2. AI is trained on human written articles, so some of that human written content informed the now established “AI slop”.
The funny thing is that "slop" was defined by the writing habits of AI model, which we have learned to pick upon and recognize.
The "It's not X, it's Y", the rhetorical questions and other patterns would have been the tools of a skilled writer, and those people writing "like AI" before AI most likely would have been recognized as such.
I am not a native English speaker. This is not surprising to me. I think most of the papers we write would be flagged by AI detectors.
It is not because we ask LLM to write us a paper about X. It is because we are bad at writing in a scientific style, and american editors expect us to do it. With LLMs, we can write in basic sentences and tell the LLM the idea and it converts that to nice writing.
If you write each paragraph and have an LLM make that paragraph more scientific, it is entirely your paper, but it is flagged as LLM generated. If you have an LLM write the paper but speak good enough English, you can make it look human even though it is not human.
I don't think this is surprising. Good technical writing is very precise. If you're starting from non-technical writing, I suspect that in most cases you can't make it "sound scientific" without adding new claims or changing the meaning. (Maybe you are being more careful, but this is something that worries me in general.)
most papers like this are flagged by AI detectors because, by your own admission, AI was used to generate the final output. what exactly did you expect here?
My English is fine, and I did hand-write my MSc thesis in English, but scientific style writing is very tedious. Personally I don't think I'd even consider writing a paper "by hand" if I were to write another one.
There really is no point, as long as you verify the content matches your intent and edit out anything poorly written.
Frankly, I've read plenty of papers by native English-speakers over the years that'd strongly benefit from being rewritten by an LLM too...
> It is because we are bad at writing in a scientific style, and american editors expect us to do it. With LLMs, we can write in basic sentences and tell the LLM the idea and it converts that to nice writing.
Which part is a sin? Using LLMs to deal with a lack of English-language fluency? I am a scientist (actually a mathematician, if it matters), and, if that's the way to deal with the practical hegemony of English in the scientific literature, then I have no problem with it. Rather that than people with important ideas can't get them before the scientific community. As long as the authors personally check and stand behind the scientific content of their papers, what do I care how the exposition was produced, especially if the role of LLMs is properly disclosed?
> If you write each paragraph and have an LLM make that paragraph more scientific, it is entirely your paper
"More scientific" is not some merely stylistic thing that faithfully preserves the original meaning of what you wrote. The precise details of each paragraph matters a lot in terms of what and how it communicates. The fact that these details do matter means that, according to my accounting, it is not entirely your paper.
Also, I find striving for "scientific" to be a pretty undesirable thing. Why should papers read like that? What is the benefit? The best papers (in terms of their writing and communication) are unpretentious and conversational. I'm pretty sure I'd prefer your "basic sentences", especially if they were wholly yours. (I understand that there are also external forces at play here as you mentioned.)
Being charitable, "more scientific" means more rigorous, reproducible, reliable based on the consensus of that scientific community (if any consensus exists, which it usually does in math and hard sciences), transparent in methodology, and candid about authorities cited and limitations to the experimental approach. Science is not a style guide; the purpose of a research publication is to impart a result to experts in the field and make claims that can be rigorously scrutinized. The best way to make a claim credibly is to demonstrate full fluency with the tools and techniques of the academic community in applying them thoughtfully to the chosen problem. But the result of doing that often looks downright impenetrable to the general public; see, e.g., Niven 1947 (1-page proof that pi is irrational). It's not the point of science to make it accessible to everyone. The point is to convince experts in the field of the correctness of the claim. The audiences are often totally different, and the tone and qualities of the writings likewise must vary.
I've never had any trouble publishing in a more conversational style and have read plenty of papers that do the same. Of course, this varies by field and venue and format (journal vs. conference) and even reviewer and it's difficult to understand what the true constraints are and what the state of the publishing system is in this regard.
But I suspect that a lot of academic's feelings about it are informed by what others have told them and how they've been trained, rather than by what's actually permissible in the publishing system.
Just to play devils advocate. These kind of papers are very verbose and boilerplate. I can imagine using AI to write 90% but then the actual novel content and explaining what’s important could be handwritten.
I tried the detector on several of my own Stack Overflow answers, and the fourth or fifth one I tried was flagged as "85 % machine / likely machine-written" with the default settings, well above the default sensitivity threshold of 42%. When I turn off the option to strip LaTeX commands (the source is Markdown), that jumps to 96% machine.
Admittedly, a Stack Overflow answer is somewhat outside the realm of scientific writing, so it's still possible that the detector may be accurate within that domain. That said, it's a cautionary tale on the hazards of applying classifiers like this outside of the domain that they were trained on.
One challenge with this approach is: could it be possible that the detector is simply learning to recognize words and jargon used more in the literature post 2022 as 'AI'?
For example, LLMs love to talk about LLMs (and the people who write with LLMs love to write about LLMs). Could "large language model" itself therefore be flagged as an AI-like phrase by this approach? It didn't exist much in the literature before 2022, does now, and certainly does more in AI-generated text: but, it is not actually a great way to distinguish modern AI generated text from human written text.
A helpful control would be to show that on some cohort of papers that can be declared reasonably clean of LLM generated text post 2023 there are very low rates compared to the arxiv.
For example, while papers in the journals Nature and Science are unlikely to be entirely LLM free at this point, if those were tested through 2026, we should see a line significantly lower than the arxiv's growth.
You raise a valid point. Just by intuition I'd say if this were true, it would probably just be a small fraction of the actual flagged articles. I will still look into how I can mitigate this when I update the detector.
The difficulty with this is then: How do you get a clean post 2023 dataset? I have no straightforward idea for this. You can't use other AI detectors to build it because then you'd never outperform them.
I work with several researchers (primarily working on consensus protocols etc).
We were discussing research in general and I asked them: "Do you prefer the writing of the papers or the research?"
They, almost unanimously, agreed that they preferred the research. This makes sense as if they preferred writing they probably would have chosen another profession.
I say this b/c having LLMs available to turn research diagrams, code etc into a paper (or at least the starting point of a paper) will probably lead to MORE quality research papers. This is b/c I'm sure there was some friction in a researcher's mind of "I would love to do the research on this but don't want the trouble of writing the paper".
Put another way: on a 2D plot with one axis being the skills as a researcher and the other being hatred of writing, LLMs may "unlock" the people high on both axes to get more papers out.
Post Script: I agree that this could also lead to more BAD papers but the net may turn out to be positive in the long run.
Something I've thought about a lot is that there is having someone with some domain knowledge or reason to care a lot about a particular issue spend a bunch of tokens and cycles on it until something useful comes out the other end. The most obvious ones are the math problems that have been coming out and help push the frontier of various areas of math. Another example is taking all of the public NYC open data ecosystem and crunching it to get some value which I have spent a lot of time and tokens on but not found a great medium to share.
The question is just how to organize these outputs and conclusions in a way that is consistently reproducible and also how to correct errors or remove LLM nonsense where it refuses to take a position on something.
Before it made sense to do this in papers but it feels like we need something like a paper format.. that is fully reproducible ideally and optimized for aggregating knowledge in a better way. I.e. before a person spent months on one of these and there was just more filtering, and the output itself was a clear signal of time spent and effort that no longer exists.
Yes, I’ve thought about this too. The strength of these models is that there is a lot more knowledge encoded in them than the average scientist has in mind at any given time. That means they can explore many more possible combinations of concepts.
If we imagine a set of all human ideas that these models have access to, then the set of possible discoveries would be something like the superset of all possible combinations of those ideas. I think all LLM discoveries are bounded by that space.
Looking at the recent OpenAI math discoveries, that seems to be pretty much what happened. Existing ideas were used as building blocks, the model found a valuable combination, and the result was something new that had real value.
Yeah and I think even what might be the most useful is extracting how Codex got to a particular solution into a skill or into even more customized software so it can be applied in lots more places.
I know people are working on these things I just haven’t seen the right way yet. Like in manufacturing right now people are trying to encode what skilled machinists do into software and scale it up, we need to go further on that for Math/ data analysis etc.
I get 0% (accurately) on my latest paper. Not super surprised, as I intentionally avoid some LLM-isms that I used to use because I don't want reviewers to have even the slightest indication that text is LLM-generated (even if in principle I'm not opposed to polishing or even wholesale generating academic text if it can convey the original research well, especially for non-native speakers).
I don't think the problem is as bad as a naive reading of this article suggests. I'm highly skeptical that anywhere near 65% of recent CS papers that I've read (mostly systems papers) are substantially AI-written. I threw some recent papers I've read into the system and they come back as 0-7%.
To corroborate this - I found pretty similar rates of AI-flagged papers over time from running pangram on ArXiv (abstracts) in a physics subfield, which is currently at about 25% as of April (write up here: https://peterse.github.io/2026/06/15/The-rising-tide-pt1.htm...).
Its not clear from your writeup what threshold needs to be reached to be classified as "machine written". A preprint where half the text is human and half is 100% AI should be a different category than a preprint where 100% of the text is AI-assisted.
Also its cool that you're making the detector available. When you say "cheap to run", do you know how this compares to pricing for a commercial detector pangram or GPTZero?
Is it just me, or does this article itself read as AI generated?.. Ironically, I tried running the text of this article through their own classifier, and pretty much all of it was highlighted in red. I'm not sure what to make of this. Is this some kind of intentional irony?
It obviously is! Human pattern-matching abilities are rather amazing, and unfortunately I've seen so much AI text that it instantly flagged in my brain within 3 seconds of opening the page.
I run most professional statements and articles through LLMs before submission. This helps correct grammar and improves accessibility through better syntax (because I write exactly how I think).
The article doesn't seem to mention consideration of AI for polishing human work.
AI doesn't polish human work. It is more like an extruder that squeezes anything you put into it into a generic, formulaic shape, indistinguishable from writing that was lazily prompted because the writer couldn't be bothered to put forth any more effort than that.
"AI does not polish human work. It acts more like an extruder, forcing anything fed into it into the same generic, formulaic shape—indistinguishable from writing produced by a lazy prompt from someone unwilling to put in any more effort."
> I run most professional statements and articles through LLMs before submission. This helps correct grammar and improves accessibility through better syntax (because I write exactly how I think).
> The article doesn't seem to mention consideration of AI for polishing human work.
Because it isn't a consideration. You are what they are looking for.
I’m very skeptical of these results. I run a pretty large group paper website on top of arXiv and we run pangram on papers. The numbers are not nearly this high.
One thing I see a lot is papers flagged as AI because they include llm rollouts in the paper as examples.
I ran two papers and three blog posts of mine through here, and all but one (correctly) flagged as 0-1% machine written. The other (human written) blog post was 20%. Pretty good afaict!
On my knees begging that the developers of AI detectors use them on their own writing and that at least pretended to mask the obvious LLM cliches in their blog posts.
Not convinced that this slop measurement is useful.
I asked Codex to generate an article with a high score and then asked Codex to (reverse?) hill climb that score. The original generation scored 97% and then the optimized one scored 1%. Both are pretty bad and read like slop.
the funniest part of these AI detectors is that if I were to upload any of einstin's paper's they will all be flagged as AI-written. it makes sense because it's part of their training data.
but this post makes me wonder, if more papers' are written with AI, or the shape of knowledge of converging?
the measurement breaks because the detector is probably also AI, and at that point you are just watching two language models argue about who wrote what
People are saying this is a bad thing but is it really a problem? The compelling aspect of research is the data and/or description of work, not the writing. Papers probably should be written by AI so that they're clear and well presented, while the researchers should focus on generating good data. If there is no data or work behind the paper, we should question whether the research group needs funding.
but the assumption you are making is that the underlying ideas are compelling. In practice, people often decide to publish incremental and/or mediocre work for various reasons, and then dress these ideas up to seem as compelling as possible to get past peer review or make a press release.
at the least, this is problematic for peer-review because the absolute number of submissions outpaces the time availability of a finite number of expert reviewers. We cannot quickly generate expert human reviewers, and so the community might converge towards half-baked solutions (AI-generated reviews or rejection systems, vastly expanded referee pools, etc.) that tend to erode trust and and make scientific communities more adversarial.
What's the point of looking when no one even cares?
consider the fraudster that went around suing people on the basis of his absurd claims of being bitcoin's creator. He's now transitioned to using AI to gather graduate degrees and is obtaining masters and doctoral degrees at a regular place and writing multiple 'papers' per day that are all quite obviously AI slop.
People report his cheating and publications and simply no one cares... and this is someone court adjudicated to have fabricated evidence in court on a massive scale, including through the use of AI.
But when it comes to the degrees and publication everyone involved that wanted paid got paid, and apparently that's all that matters.
As an academic who is forced to review AI-generated conference/journal submissions, I can assure you this is not the case. LLM-heavy papers are terrible.
Even if they manage to avoid straight-up factual incorrectness, the writing is full of ambiguities and vagueness when you look closely. Meanwhile, space is wasted repeating the same claim in multiple ways, or explaining something simple.
They also seem unable to resist the hype/advertising tone, overselling the contribution while exaggerating the limitations of related work.
I would much rather read grammatically incorrect or awkward sentences.
While academic writing does have a few pointless historical conventions, the huge majority of "status quo writing style" is a logical consequence of 1) minimizing ambiguity, 2) organizing ideas coherently, 3) distinguishing opinion/interpretation from fact, and 4) providing enough detail to be reproducible.
Genuinely. I get that there may be some visceral reaction against this, but when I break it down, I mostly fail to see the problem. Seems like what is actually important is:
Compared to before, when a human reads it, do they -- or society -- get something good out of it? Is it worth it to add this to the "pantheon?"
If that's not what's happening enough, and if this doesn't describe the process -- then the problem lies elsewhere, no?
I recently desk-rejected a paper where every single citation in its Introduction was hallucinated. That means that the entire connection between what the author(s) did and how it relates to existing research was simply made up. I've never seen this happening before AI but now there's at least one paper in every cycle pulling something similar.
My problem therefore is: we are seeing more and more papers written with tools that are known to make up facts, citations, and even entire papers. And the number of papers has increased, too. I therefore see it less as "people are being more productive" and more "people are releasing bad science much faster than we can keep up with".
Yeah but that's an issue with the researcher putting out a bad paper, and it suggests you'll have to reject more papers. We wouldn't ban email because many of the emails are spam, it just means we need new tools to filter out junk. AI will allow researchers to be more productive all together and take less time to publish a paper, which is good.
The problem with spam is that it's not a technology like AI is. So I suggest taking cars instead.
Cars have plenty of advantages, and yet no one would say "the number of pedestrians killed by cars is rising, but that's an issue with the drivers". In fact, the opposite is true: from fines and school zones to speed bumps and bollards, we have accepted that cars bring structural problems with them that cannot be solved at the driver level alone.
> we need new tools to filter out junk
Agreed, but if my office suddenly was flooded with garbage my first thought wouldn't be "I need more, bigger trash cans" but rather "who brought all this junk here and why?". To simply assume that the garbage is a sudden natural phenomena that I have to live with seems, at the very least, unfair.
> Yeah but that's an issue with the researcher putting out a bad paper, and it suggests you'll have to reject more papers. We wouldn't ban email because many of the emails are spam, it just means we need new tools to filter out junk. AI will allow researchers to be more productive all together and take less time to publish a paper, which is good.
It's a signal:noise ratio thing. If 1 out of every 1000 AI-written papers are bad, it makes sense to put in a filter that auto-rejects any paper that has AI tells.
After all, if that 1 researcher was any good, he wouldn't have used AI to write the thing in the first place.
Publishing was always about getting past the filters. There's one more filter - "AI-generated content" - so do what you have to to get past it. IOW, write your own paper.
While working on my PhD, there was one guy in the field that would publish new, better results to Springer every time someone improved upon benchmarks. No code, no data, no conference publications, paywall restricted, unverifiable, always the best.
I am almost certain he was "hallucinating" the results. This was in the 2010s
There are well known issues in academic publishing, though I imagine it has become much noisier like open source
Surprisingly I agree with you. My opinion is: if it makes communicating research more effective, while not reducing the quality of the output substantially, I see no issue.
A possible conclusion for this could be: If the majority of CS papers is AI written, let's just accept this reality universally and stop worrying about it altogether.
I think the key question is more effective at what?
I see plenty of anecdotal evidence that models have been trained fantastically well—and getting better—at writing to trigger the right neurons in the human population to produce “This is interesting/informative/correct” responses in bulk.
Could their ability to produce those responses run far ahead of their ability to actually achieve the last in reality? Sure seems plausible, and then where are we?
Many people are bad at writing, including scientists. Improvements could be articulating a concept in a way that the reader will understand it clearly.
My hypothesis is that an LLM’s ability to concoct prose that will convince even an expert of the validity of an idea is largely independent of the LLM’s ability to validate the idea itself, or whether the idea is correct in the first place. Especially if it’s being prompted to do the former, not the latter.
My anecdotal evidence is the LLM-generated, inchoate technical dross that is routinely upvoted onto the hn front page. Much of it isn’t even coherent enough to be wrong, but the readership here finds it interesting!
> if it makes communicating research more effective, while not reducing the quality of the output substantially, I see no issue.
That's a big "If".
If a research is good, the author still has to clear all the hurdles in publishing. "Writing your own paper" is just one more hurdle.
> A possible conclusion for this could be: If the majority of CS papers is AI written, let's just accept this reality universally and stop worrying about it altogether.
That's just a different way of saying "if the majority of CS papers are crap, lets just accept this reality".
So, go on, publish away all your AI-induced "research", but the bar is slowly going to be raised anyway to reject that. That's how science always worked - when a bar is not sufficient to exclude the crap, it is raised.
I reviewed for ACL (big NLP conf) and another similar conference recently. Lots of crap (though there always has been). One superficially well-written paper was likely AI-plagiarized (i.e. the AI sampled an idea from prior work and rewrote it) and got desk rejected for it.
Reviewers were also totally unengaged. Of 20 reviews I read (from my reviewers or from reviewers on the same papers), maybe 2 were mediocre, and the rest were crap (though likely not AI).
The notion that science will somehow benefit from this is about as stupid an idea as you can have. Science relies on skepticism. AIs are not skeptical, and many folks are submitting papers because they stand to gain something, not because they are motivated to do good research or develop new understanding. Fields are being inundated with garbage that is maximally indistinguishable from real work (that's the training objective for LLMs). This in turn maximizes the cost of identifying bad work.
This is the same enshittification process that we see everywhere else. You get spam phone calls because there is no reason for a spammer not to call you. "Researchers" are submitting spam papers because there is no cost to doing so with some possible gain. Absent intervention, this eventually drives the community value of the network to zero (or potentially negative, if friction costs to switching are high).
And yet the quality of work at ACL is still higher than NeurIPS, and that's with ACL this year having over 40% of all posters looking nearly identical due to everyone claude coding their posters...
I mean, sure. But what an absolutely insane predicate. "Not reducing the quality of the output substantially" is (as an AI might say...) load bearing there.
Other problems include: Signal to Noise Ratio going through the roof.
> My opinion is: if it makes communicating research more effective, while not reducing the quality of the output substantially, I see no issue.
That's a big if. ArXiv is not peer reviewed and LLMs basically interpolate and extrapolate text, which makes them essentially fluff generators. Even in the most charitable interpretation, LLMs enable those with nothing to say to say nothing while meeting surface-level style guides.
We can't know if real science is happening in the background but I'd wager that the majority of these papers is not complete slop but real findings with AI generated text used to communicate it. If it was just straight slop I would be really worried.
> We can't know if real science is happening in the background but I'd wager that the majority of these papers is not complete slop but real findings with AI generated text used to communicate it. If it was just straight slop I would be really worried.
Why don't you read them and see? The ones I looked at were clear slop.
If a human writes a given paragraph, I know it has attained a minimum level of awareness in some human mind at some point. If an AI writes a paper, I have no guarantee that it has reached that minimum. We also fear, with some reason, a correlation between AI writing and possibly being hallucinated or non-existent. Not just because AI is bad and people should feel bad for using them, but because given that a paper is somehow fake, it is probably more likely to be made with AI, and running that backwards it is reasonable to take closer looks at AI papers.
If fake papers weren't already a big problem before AI and the fields had already been policing themselves adequately, if this was already a functioning high-trust domain, maybe we could ignore this a bit more, but the fields already manifestly had problems. People taking advantage of that are reasonably more likely to use AI. The pressures to publish or perish provide the voltage and the AIs are a rather convenient path-to-ground.
I agree in some sense that if a truth is published, it doesn't matter if the AI or a human published it. However there are perfectly reasonable reasons to be concerned that AI usage is correlated to not publishing truths, especially in a world where merely being human-generated was already not an adequate check against that.
Even if you can reject the aesthetic argument for non-fiction works (although read some of Dijkstra's papers for a good counterargument), it is still a problem because it breaks an important quality signalling mechanism.
Pre-LLMs, a paper with no spelling or grammar errors showed that somebody had put effort into writing and editing it. If they cared about the presentation, they probably also cared about the content. LLMs routinely produce nonsense that looks superficially like high-quality work.
There are far too many papers to read all of them. LLM slop is evidence that something is probably low quality. As the saying goes, "if you can't be bothered writing it, I can't be bothered reading." The rare outliers will get enough citations and recommendations to overcome this filter.
That's not the point though, the point being made ist that an LLM written paper very likely doesn't actually find something despite looking like it does on the surface
if 5% of the work that I'm reading to keep up is slop, that's a shame. that's time I spent puzzling over connections that weren't real, trying to impose some kind of logic on the arguments, and wondering why the data doesn't really support the conclusion.
if 50% of the work is nonsense, then there's a serious concern that we can't move forward at all.
> Arxiv is full of pre-prints that anyone can upload.
You now (at least for some categories) have to receive endorsement from someone who has multiple recent papers on arxiv in the same (or adjacent) category.
The problem is that humans start with credulity. AI hallucinates and makes up stuff some percentage of the time. Humans are not "default deny" when given information. Unleashing that was a disservice to mankind and has created a future filled with lies and people who are confident in them.
LLMs are in essence an attack on the concept of written language, harvesting and dissolving the social contract that underpins it; it is no longer safe to assume that text has intent, let alone content, simply because it is grammatically well-formed.
Perhaps the most darkly amusing consequence of this particular mania is that by poisoning the majority of our information environment with hallucinated slop, we have likely crippled the next several generations of machine-learning techniques before they're even invented! Small, locally-hostable LLMs will rattle along spewing spam long after the broader "genai bubble" pops, and building clean training datasets will permanently be more difficult and expensive.
AI hallucinates and makes up stuff 100% percent of the time. Never been a fan of that word for this.
Again, I fail to see the problem here that isn't solved by careful reading WHICH IS WHAT PEOPLE SHOULD BE DOING ANYWAY. I would like to see room for AI disclosure, maybe a statement of "this is how much AI I used."
But this blanket X% of this looks like AI? Again, so what?
If as a primary school student you had needed to reverify every fact /experiment that was presented in your textbooks, you would not have gotten very far.
I find the fact that 39% of papers use a style that signals lack of effort somewhat worrisome. Even if that’s 100% wrong, and they’re all high effort papers, the fact that they give off the same aura as low effort papers is a problem for the authors.
LLM-written text tends to have the property that it gives the impression of expertise and knowledge in excess of what the text actually contains.
In other words, it "sounds smart" without necessarily having anything to back it up. In even more critical terms, it's very good at bullshitting.
Unfortunately for us, the scientific community current relies on a certain amount of trust. (To do otherwise is very expensive! see: bitcoin). When you introduce a known-bullshitter to write your papers, every human in the loop effectively has to defend against an adversarial attack. Not just the readers at home, or the peer reviewers, but even the author needs to be wary that the facts and arguments coming out of the LLM are true and meaningful.
Personally, I've been a minor contributor to several high-profile papers. I don't know how every field does it, but in my experience, the corresponding author (generally the PI or other senior scientist), is responsible for the accuracy of the paper. They ultimately have to trust the people who did the work that the facts are true. Introducing LLMs into the mix make it more difficult for them to identify and review sections they are unsure of. (An honest person will typically write at a confidence level reflecting their certainty. LLMs do not do this in any reliable way.)
I've also found that LLMs frequently use metaphors that are unhelpful, or used out of context in a field that isn't familiar with them. This makes understanding the text more work, for no good reason. Introducing terms or definitions with low relevance reads as impressive at first glance, but avoiding the standard terminology in the field just adds confusion. (As an analogy, imagine if you were reading a CS paper that, for no particular reason, devoted a section to a new data structure called an "akimbo tree," which after much untangling, you realized was a reinvention of a randomized splay tree.)
Agreed. I know nothing about nuclear physics. I still doubt you could pick a random person off the street and have them convincingly pose as a nuclear physicist to explain a "nuclear physics" concept to me. I doubt you could do it with a random PhD from a non-physics field. An LLM could probably convince me even if 90% of the content of the explanation is subtly or blatantly incorrect.
> Here is the method, the results, and an honest account of the limitations.
This paper was written using AI, to be honest
I'm not sure what to think... I uploaded a PyHPC workshop paper I wrote in 2011 and it said 27% machine.
I also uploaded my PhD dissertation from 2012 and got back 40% machine, which is just barely below the 42% threshold.
I don't publish anymore... but does this mean I wrote like a LLM or did LLMs learn from me? :p
Update: I also uploaded a IEEE CLUSTERS paper I wrote in 2015 and it came back 74% machine written :|
When "detectors" first started popping up all over the place, all of them rated the declaration of independence as 100% AI written, so... Yeah, these things just don't work. And what's even more dangerous is that people that don't understand how any of it works use these tools, and accuse people of using AI, sometimes with grave consequences. Students have been through this, at all levels of education.
In fairness, the original crop of detectors were based on perplexity scores and were entirely useless (famously, the declaration of independence often came back as 100% AI generated).
I'm not convinced we'll ever have full-proof detectors, and certainly the false-positive rate will make them irresponsible for accusations of intellectual/academic fraud, I do think that LLMs are easy for folks to sniff out on average so I imagine it's possible to detect many instances.
Pangram's detector is anecdotally very accurate in my tests. This detector appears to be fine-tuned on a very small dataset (200 papers per subject), and suspect the problem might be in part that.
I replied to a comment with a structured informal proof on HN.
A well established user here was adamant that I used AI because apparently humans never ever wrote proofs.
This was a while ago. Any well crafted human output is now being dismissively cast as AI if the reader is challenged by the output intellectually/politically.
It's the irl version of being accused of hacking in Counterstrike.
I’ve been accused of being a bot myself. I’ve started leaving some incorrect autocorrects in, hopping it would stave off people for whom it seems making accusations of being a bot is soothing or a coping mechanism when they’re confronted with information that does not fit into their world view or perception of self.
They use a "threshold calibrated so pre-ChatGPT papers flag at 0.4%", so these things do work most of the time. It also means that there are known false positives, so for any given paper, scoring above the threshold isn't irrefutable proof of AI usage. But for things like estimating the overall proportion of AI writing, you only need to be correct on average, so individual false positives don't matter much.
It would be interesting to see how that .4% varies across fields on arXiv. I imagine some areas influenced LLM writing much more than others and are more susceptible to false positives. Anyway it's a little strange to me that these detectors do so poorly (at least by what I hear on this site). There is as much labeled data as you would need to train on pre-LLM human vs LLM text. If their out of sample errors are as good as has been claimed, the only methodological flaw I can think of is that post-LLM human text and LLM text have been converging.
There's a table in the article breaking down the rate by field.
It means machine text and human text cannot be distinguished from each other.
It's just text.
"Any attempt to build AI generated content (deep fake) detection systems is flawed, since the outputs of such a system may be used to train an even better fake data generator. This leads to an equilibrium state of digital uncertainty: nothing in the digital realm can be deemed as real anymore - only as digital. I do not care if a digital artifact is human or AI made - I only care if it is useful to me. Useful content is on point, factual and at best surprising (teaches something new)." - https://seanpedersen.github.io/posts/digital-uncertainty/
Could it be that your papers are literally in the training set?
It's been trained on a work and then distilled, the false positive noise has to be absurd.
I scored the full text of 12,750 arXiv papers from 2021 through 2026 to find out how many of these get flagged as machine written and how much it increased since the release of chatGPT. I purposely tuned the detector to avoid false positives. My detection rate pre chatGPT is around .4% for that reason.
The biggest results: in Jan of 2026 about 39% of papers got flagged as AI written. In computer science speicifcally the peak was at 65%. Mathematics barely moved away from 0.7%, though the proof heavy math texts might just not get picked up by the detector properly.
All this is a detector estimate of a statistical signal and not a proof any given author used AI. Machine written can also mean heavy AI-assisted editing.
Couple of methodological notes
* pre-chatGPT is not an effective control because language evolves. In the arxiv corpus in particular there are "fashions" in research depending on what gets funded lately, not to mention many new words and topics not invented before a given paper.
* In general, detecting AI from content seems difficult as humans write like they read. To the extent there are unique factors recognizable as AI and to the extent humans read them, they will eventually incorporate them into their writing style. Accordingly, you'd need to model a rolling window of "AI tells" that decay at some rate.
This is a pretty stunning result. The time series looks really convincing. Is the way the detector itself is trained orthogonal to this or could there be some "leakage" in that the pre-chatgpt text is in the (positive) training data?
I tried my best to avoid leakage. If you're curious about how I trained the detector I have a writeup on it: https://unslop.run/blog/how-our-ai-text-detector-works
FYI this is all relatively new so there might be lots of issues and iterations coming.
My honest first impressions, since I think the project is well intended: This writeup is itself AI, and I would venture to call it slop. The Calibration section is borderline uninterpretable, and I challenge any non-author who claims to understand it to answer some basic peer review questions about it.
I have to concur, unfortunately. This isn't reliable, reproducible, or interpretable.
What detector are you using? How can you be sure of its accuracy given that every commercial AI detector has been debunked?
Almost every. Pangram is pretty accurate on longer texts. I haven't seen any glaring examples of false positives.
I’ve heard good things about Pangram but it’s trivially easy to fool it. I generated text using Opus 4.8 and then humanized it using Grammarly. Pangram determined it 100% human-written. Tried it numerous times with the same result. GPTZero claimed with 80% confidence that it was human-written but AI polished. Nothing in that text was human-written.
This! Pangram is very good. They claim 1 in 10,000 fp rate.
I had to change my mind on AI detectors after playing around with it.
It would be interesting to hear how this detector compares. It also seems to be aiming for low fp rate.
A signal, I put an AI generated of mine (with heavy human guidance on aesthetics mostly and some minor human editing) and got 5% only.
Could you share some pre LLM false positives? Would be interesting to see what is tripping the detection. Did a tiny fraction of authors write like LLMs, before LLMs?
Extend it to scan papers from pre-2020. That should give you a better baseline accuracy for your detection system.
this is neat! is the model available somewhere for local execution? or even a lookup table with your results for all arXiv pre-print codes? I want to run it on lots of pre-prints and I don't want to kill your server
Thank you, I plan to release the arxiv preprint codes. Also don't worry about killing my server, let me know if you succeed :D
Wouldn't that provide an excellent signal to tune models to be less like AI? I suppose that's a good thing ultimately.
Thank you for this work.
There some real game theory mechanics at play when it comes to LLM usage in corporations. Devs are cranking out superficially superior code and documentation by just aiming the Claude Code fire hose at everything they can. Leadership encourages this because from what they can tell, there is no downside.
It's hard to say if this code is structurally better or worse than before, but it's certainly voluminous and as far as leadership could ever tell, with their flawed metrics, that is all that matters. It will be years before we figure out if this is a good idea and worth the cognitive atrophy.
Anyone not using LLMs all day is just not going to be as prolific. I can't imagine that the same factors aren't at play in the scientific research community where it's all about how much you can publish.
>I can't imagine that the same factors aren't at play in the scientific research community
I can't speak for the entire scientific research community, but I can say that for basic biomedical research (not clinical research), LLMs are mostly ignored. They simply don't have the ability to parse most raw wet lab data.
This obviously has not stopped the creation of a legion of startups, new Asst Profs, etc, claiming that they are using "AI" to crack new frontiers. In practice, the most effective of these use machine learning, rather than LLMs.
> It's hard to say if this code is structurally better or worse than before, but it's certainly voluminous and as far as leadership could ever tell, with their flawed metrics, that is all that matters.
Hah. I've been working with ChatGPT-5.6 Sol and a lot of guidance to port a particular geometry from 2800 lines of Python-scripted Rhino3D (basically a custom parametric CAD kernel in there, plus use of it) to Python-scripted OCCT/FreeCAD (an existing CAD kernel) and it's up to 36,000 lines and only about half way there. And I've been setting goals and targets for duplication minimization, code size reduction, etc. The results are fine for my purposes, but if I put a positive value on "voluminous" I'd be super impressed; with my actual negative value assignment on lines of code, this is more "absolute crap but still useful to me."
I think "absolute crap but still useful to me" is a pretty high value and not worth neglecting, but I'm definitely scared by leadership who sees the toilet overflowng and assumes that means that value has been maximized.
> I think "absolute crap but still useful to me" is a pretty high value and not worth neglecting
If I extrapolate this example to my professional life, this code now manages millions of dollars, a single mistake can wipe it all out, it has to be maintained by 5 other engineers and understood by 5 other domain experts.
That's a real risk, yes. There's definitely a difference between cases where the code is the artifact being built, and cases where the code is being built to generate an artifact that is independently valuable and assessable. I'm happily in the latter case -- but I definitely share your fear of those who think current LLMs are valuable in the former case.
> It's hard to say if this code is structurally better or worse than before,
Huh?
But that's easy to tell?
It's better then incompetents code, worse then a motivated average dev... But good enough hence the real question is value aka time& money invested/quality.
That's much harder to tell, and I currently think opus/fable generated code is decent enough to be safely in favor - at least on subscription
> It's better then incompetents code, worse then a motivated average dev...
That's well put.
> But good enough hence the real question is value aka time& money invested/quality.
There's time invested SO FAR and time that will have to be invested to maintain it. In my experience, even with Fable, it's not there yet. It's the reason why it's easy to vibe code an app from scratch, but at some point when complexity significantly increases, the codebase becomes a mess.
> Leadership encourages this because from what they can tell, there is no downside.
This was my big fear before we saw price increases. Now I'm pinning all my hopes on AI being too expensive to justify further big corporate pushes. (Sigh.) I love having new tools, but I hate being pushed to use ______ tool to meet some managerial metric.
IMO, cognitive athrophy is the point. If humans become dependent on AI, those corporations will get enough leverage to change laws in their favor. For example, people don't want a huge data center in the middle of their city, but the corporations will say that the same people can't do anything without AI powered by this data center. That's why I think that making humans deficient and dependent on AI is the entire point.
This is just conspiracy nonsense.
> "superficially superior code"
Can you unpack that a bit? It produces measurably, meaningfully inferior code everywhere I see it in use.
> "leadership encourages this because from what they can tell, there is no downside"
As said 'leadership' I find this a bit puzzling. I'm seeing strong, quantifiable evidence of increasing churn, increasing incident count, and length of downtime from the date of our biggest push into GenAI, and I'm organizing efforts on my teams to mitigate those issues and actively reduce GenAI adoption.
If you mean my c-suite, you're mostly correct although they are already rumbling about seeing zero or negative ROI on GenAI investments.
> Anyone not using LLMs all day is just not going to be as prolific
Agreed, but prolific != productive.
> leadership
Not sure if its just me, lately I have started feeling pretty offensive about the increased usage of the word. Its management not leadership by any means.
> "leadership encourages this because from what they can tell, there is no downside"
For most people in management its easier to pick the current set of slangs/abbreviation's, general trend and go with it. Understanding the details would take time, raise questions and no one in management has time or political capital to spend on it.
What size company are you talking about?
The code written by agents generally seems to reflect what you (the human driver) asked for. If you discuss the approach and architecture first, asking the right questions in the process, and then let it implement - the result is quite on point with the frontier models. Might need some minor touch-ups if agent missed some common conventions or guessed the expectations wrong (but again, salvageable with follow-up prompts). And when it comes to line-by-line logic within functions, I would argue that today's models are LESS likely to make a mistake in there than humans - especially if you cross-review with another model (e.g. "write with Claude, review with GPT").
Humans write slop too, you know. Just saying.
As with all text-only AI detection schemes, I am concerned about the accuracy of the detection. I'm skeptical of the methodology, specifically the final join of the three detector scores---how can you be sure that that final step does not introduce any biases? There's no source available, so it's difficult to tell exactly how this works or reproduce the research.
I've also uploaded text samples from my own (unreleased) research from pre-LLM era, and it's seemingly scoring pretty high on the LLM-detection scores. On other papers, nearly every sentence is highlighted as red "machine-leaning," but that does not impact the score? Additionally, there are dramatic differences between the scores for identical text with and without LaTeX formatting, despite the fact that it should not matter.
The takeaway from this should be "it is difficult to detect generated text and we should be careful about accepting results simply because they confirm a hypothesis."
--
Relatedly, the text above scores as highly machine-written, despite the fact that I just wrote it with my human hands, I promise :)
The most concerning is the use of these tools to detect cheaters in schools.
I've looked at this same problem in an academic environment and have come to conclude that there is no way to reliably detect AI writing using only text. The reason for this is that no detector can take two identical inputs and classify one as synthetic (LLM) and one as organic (Human) and this situation can easily happen at the sentence or even paragraph level. Posed as a question, if a human writes a paragraph that happens to be exactly the same as a paragraph written by an LLM how do you classify that paragraph? There are a finite number of words and a finite way of combining them within an academic setting/field which means that we can't build a perfect classifier with just text. In practice, there are obvious tells and LLM-isms but these also change with time and each model has different tells so that even if a paper is full of LLM like writing we have no way to disambiguate between organic text that appears synthetic or synthetic text that appears organic.
A more interesting question, to me, is looking at a corpus of essays and analyzing how writing has changed with the introduction of LLMs. We can look at changes in vocabulary, linguistic features, style embeddings, regular embeddings, typos, errors, and references over time. When looked at in this way it is clear that academic writing has changed at the population level but what has led the change is harder to track down.
> There are a finite number of words and a finite way of combining them within an academic setting/field which means that we can't build a perfect classifier with just text.
Maybe for very short phrases, but otherwise I disagree. Phrasing very quickly runs into a combinatorial explosion. In the words of Noam Chomsky, "Virtually every sentence that a person utters or understands is a brand-new combination of words, appearing for the first time in the history of the universe."
In my opinion, the difficulty in LLM/human text discrimination isn't that a person might coincidentally write exactly the same text as an LLM would, but rather that 1) LLMs aren't hard locked to a single phrasing (so this is a tougher problem than matching to a single static document, e.g. plagiarism detection) and 2) text has relatively low information density, so you need quite a bit of it to gather enough data to run a statistical test with a reasonably narrow confidence interval.
> If a tool marks 40% of new papers as machine-written but also marks 20% of papers written before ChatGPT existed, the real story is the 20% nobody mentioned.
When 65% of the papers you read have the characteristics of being AI written, whether or not you use AI to write, your writing will be influenced by the AI style. I imagine this must be particularly the case for newbie researchers who are still developing their writing style
even if your writing is influenced by "the AI style," it would likely only show up in patches, a word or rhetorical flourish maybe, but AI-generated text doesn't "sound like AI" in patches, it does so consistently across the entirety of the output. it's very hard for a person to maintain the syntactical patterns of AI writing over an extended period of time unless they know exactly what those are -- probably more knowledge than any person currently has, really -- and are applying a level of detail to each clause comparable to forging a painting.
there's also the fact that "the AI style" has changed over time. for instance, the word "delve" is notorious as an "AI sign," which it was up until mid-2024, at which point it dropped off sharply and has now basically disappeared from LLM output. so if someone happened to pick that up due to reading it everywhere, their writing is now less characteristic of AI, not more.
I might be missing something but what’s the real story of the 20%.
To me it sounds like 1. Either your tool is just not that good and reliable as you thought, 2. AI is trained on human written articles, so some of that human written content informed the now established “AI slop”.
There are people who shipped “slop” before AI.
> There are people who shipped “slop” before AI.
The funny thing is that "slop" was defined by the writing habits of AI model, which we have learned to pick upon and recognize.
The "It's not X, it's Y", the rhetorical questions and other patterns would have been the tools of a skilled writer, and those people writing "like AI" before AI most likely would have been recognized as such.
I am not a native English speaker. This is not surprising to me. I think most of the papers we write would be flagged by AI detectors.
It is not because we ask LLM to write us a paper about X. It is because we are bad at writing in a scientific style, and american editors expect us to do it. With LLMs, we can write in basic sentences and tell the LLM the idea and it converts that to nice writing.
If you write each paragraph and have an LLM make that paragraph more scientific, it is entirely your paper, but it is flagged as LLM generated. If you have an LLM write the paper but speak good enough English, you can make it look human even though it is not human.
It seems that LLMs often insert new ideas even when asked only to polish writing without changing it: https://bsky.app/profile/tomerullman.bsky.social/post/3mq33c...
I don't think this is surprising. Good technical writing is very precise. If you're starting from non-technical writing, I suspect that in most cases you can't make it "sound scientific" without adding new claims or changing the meaning. (Maybe you are being more careful, but this is something that worries me in general.)
most papers like this are flagged by AI detectors because, by your own admission, AI was used to generate the final output. what exactly did you expect here?
Yes that is exactly what I suspected. And I see absolutely nothing wrong with this.
My English is fine, and I did hand-write my MSc thesis in English, but scientific style writing is very tedious. Personally I don't think I'd even consider writing a paper "by hand" if I were to write another one.
There really is no point, as long as you verify the content matches your intent and edit out anything poorly written.
Frankly, I've read plenty of papers by native English-speakers over the years that'd strongly benefit from being rewritten by an LLM too...
> It is because we are bad at writing in a scientific style, and american editors expect us to do it. With LLMs, we can write in basic sentences and tell the LLM the idea and it converts that to nice writing.
This is a sin.
Which part is a sin? Using LLMs to deal with a lack of English-language fluency? I am a scientist (actually a mathematician, if it matters), and, if that's the way to deal with the practical hegemony of English in the scientific literature, then I have no problem with it. Rather that than people with important ideas can't get them before the scientific community. As long as the authors personally check and stand behind the scientific content of their papers, what do I care how the exposition was produced, especially if the role of LLMs is properly disclosed?
https://lists.isocpp.org/std-proposals/att-0486/Reply_to_Zer...
Here's a paper by a non native English speaker. The lack of the formal style doesn't cause any issue. Preciseness is what matters.
I feel like they should release the original before the llm chewed on it.
> If you write each paragraph and have an LLM make that paragraph more scientific, it is entirely your paper
"More scientific" is not some merely stylistic thing that faithfully preserves the original meaning of what you wrote. The precise details of each paragraph matters a lot in terms of what and how it communicates. The fact that these details do matter means that, according to my accounting, it is not entirely your paper.
Also, I find striving for "scientific" to be a pretty undesirable thing. Why should papers read like that? What is the benefit? The best papers (in terms of their writing and communication) are unpretentious and conversational. I'm pretty sure I'd prefer your "basic sentences", especially if they were wholly yours. (I understand that there are also external forces at play here as you mentioned.)
Being charitable, "more scientific" means more rigorous, reproducible, reliable based on the consensus of that scientific community (if any consensus exists, which it usually does in math and hard sciences), transparent in methodology, and candid about authorities cited and limitations to the experimental approach. Science is not a style guide; the purpose of a research publication is to impart a result to experts in the field and make claims that can be rigorously scrutinized. The best way to make a claim credibly is to demonstrate full fluency with the tools and techniques of the academic community in applying them thoughtfully to the chosen problem. But the result of doing that often looks downright impenetrable to the general public; see, e.g., Niven 1947 (1-page proof that pi is irrational). It's not the point of science to make it accessible to everyone. The point is to convince experts in the field of the correctness of the claim. The audiences are often totally different, and the tone and qualities of the writings likewise must vary.
You might prefer that, but journal editors might not.
I've never had any trouble publishing in a more conversational style and have read plenty of papers that do the same. Of course, this varies by field and venue and format (journal vs. conference) and even reviewer and it's difficult to understand what the true constraints are and what the state of the publishing system is in this regard.
But I suspect that a lot of academic's feelings about it are informed by what others have told them and how they've been trained, rather than by what's actually permissible in the publishing system.
There is a fine line between using an LLM to clean up grammar and spelling and using it for text generation.
Just to play devils advocate. These kind of papers are very verbose and boilerplate. I can imagine using AI to write 90% but then the actual novel content and explaining what’s important could be handwritten.
Perhaps that’s what’s happening.
If 90% of it is boilerplate, then you really need to question if you have something worth publishing as an academic paper.
It's generally always better for an academic's career to publish something, rather than publish nothing.
Yes, this is the thing. In the scientific enterprise, the writing is mostly wasted time. Why would you write it up if machines can do it competently?
Sure, some people are artists - but most aren't.
I tried the detector on several of my own Stack Overflow answers, and the fourth or fifth one I tried was flagged as "85 % machine / likely machine-written" with the default settings, well above the default sensitivity threshold of 42%. When I turn off the option to strip LaTeX commands (the source is Markdown), that jumps to 96% machine.
Admittedly, a Stack Overflow answer is somewhat outside the realm of scientific writing, so it's still possible that the detector may be accurate within that domain. That said, it's a cautionary tale on the hazards of applying classifiers like this outside of the domain that they were trained on.
One challenge with this approach is: could it be possible that the detector is simply learning to recognize words and jargon used more in the literature post 2022 as 'AI'?
For example, LLMs love to talk about LLMs (and the people who write with LLMs love to write about LLMs). Could "large language model" itself therefore be flagged as an AI-like phrase by this approach? It didn't exist much in the literature before 2022, does now, and certainly does more in AI-generated text: but, it is not actually a great way to distinguish modern AI generated text from human written text.
A helpful control would be to show that on some cohort of papers that can be declared reasonably clean of LLM generated text post 2023 there are very low rates compared to the arxiv.
For example, while papers in the journals Nature and Science are unlikely to be entirely LLM free at this point, if those were tested through 2026, we should see a line significantly lower than the arxiv's growth.
You raise a valid point. Just by intuition I'd say if this were true, it would probably just be a small fraction of the actual flagged articles. I will still look into how I can mitigate this when I update the detector.
The difficulty with this is then: How do you get a clean post 2023 dataset? I have no straightforward idea for this. You can't use other AI detectors to build it because then you'd never outperform them.
I work with several researchers (primarily working on consensus protocols etc).
We were discussing research in general and I asked them: "Do you prefer the writing of the papers or the research?"
They, almost unanimously, agreed that they preferred the research. This makes sense as if they preferred writing they probably would have chosen another profession.
I say this b/c having LLMs available to turn research diagrams, code etc into a paper (or at least the starting point of a paper) will probably lead to MORE quality research papers. This is b/c I'm sure there was some friction in a researcher's mind of "I would love to do the research on this but don't want the trouble of writing the paper".
Put another way: on a 2D plot with one axis being the skills as a researcher and the other being hatred of writing, LLMs may "unlock" the people high on both axes to get more papers out.
Post Script: I agree that this could also lead to more BAD papers but the net may turn out to be positive in the long run.
Something I've thought about a lot is that there is having someone with some domain knowledge or reason to care a lot about a particular issue spend a bunch of tokens and cycles on it until something useful comes out the other end. The most obvious ones are the math problems that have been coming out and help push the frontier of various areas of math. Another example is taking all of the public NYC open data ecosystem and crunching it to get some value which I have spent a lot of time and tokens on but not found a great medium to share.
The question is just how to organize these outputs and conclusions in a way that is consistently reproducible and also how to correct errors or remove LLM nonsense where it refuses to take a position on something.
Before it made sense to do this in papers but it feels like we need something like a paper format.. that is fully reproducible ideally and optimized for aggregating knowledge in a better way. I.e. before a person spent months on one of these and there was just more filtering, and the output itself was a clear signal of time spent and effort that no longer exists.
Yes, I’ve thought about this too. The strength of these models is that there is a lot more knowledge encoded in them than the average scientist has in mind at any given time. That means they can explore many more possible combinations of concepts.
If we imagine a set of all human ideas that these models have access to, then the set of possible discoveries would be something like the superset of all possible combinations of those ideas. I think all LLM discoveries are bounded by that space.
Looking at the recent OpenAI math discoveries, that seems to be pretty much what happened. Existing ideas were used as building blocks, the model found a valuable combination, and the result was something new that had real value.
Yeah and I think even what might be the most useful is extracting how Codex got to a particular solution into a skill or into even more customized software so it can be applied in lots more places.
I know people are working on these things I just haven’t seen the right way yet. Like in manufacturing right now people are trying to encode what skilled machinists do into software and scale it up, we need to go further on that for Math/ data analysis etc.
I get 0% (accurately) on my latest paper. Not super surprised, as I intentionally avoid some LLM-isms that I used to use because I don't want reviewers to have even the slightest indication that text is LLM-generated (even if in principle I'm not opposed to polishing or even wholesale generating academic text if it can convey the original research well, especially for non-native speakers).
I don't think the problem is as bad as a naive reading of this article suggests. I'm highly skeptical that anywhere near 65% of recent CS papers that I've read (mostly systems papers) are substantially AI-written. I threw some recent papers I've read into the system and they come back as 0-7%.
To corroborate this - I found pretty similar rates of AI-flagged papers over time from running pangram on ArXiv (abstracts) in a physics subfield, which is currently at about 25% as of April (write up here: https://peterse.github.io/2026/06/15/The-rising-tide-pt1.htm...).
Its not clear from your writeup what threshold needs to be reached to be classified as "machine written". A preprint where half the text is human and half is 100% AI should be a different category than a preprint where 100% of the text is AI-assisted.
Also its cool that you're making the detector available. When you say "cheap to run", do you know how this compares to pricing for a commercial detector pangram or GPTZero?
Is it just me, or does this article itself read as AI generated?.. Ironically, I tried running the text of this article through their own classifier, and pretty much all of it was highlighted in red. I'm not sure what to make of this. Is this some kind of intentional irony?
Yes, 100%. "where the measurement breaks" and "honest account of the limitations" are all I needed to see.
It obviously is! Human pattern-matching abilities are rather amazing, and unfortunately I've seen so much AI text that it instantly flagged in my brain within 3 seconds of opening the page.
Pangram agrees: https://www.pangram.com/history/3de33376-94e3-404d-bbb0-751a...
I run most professional statements and articles through LLMs before submission. This helps correct grammar and improves accessibility through better syntax (because I write exactly how I think).
The article doesn't seem to mention consideration of AI for polishing human work.
AI doesn't polish human work. It is more like an extruder that squeezes anything you put into it into a generic, formulaic shape, indistinguishable from writing that was lazily prompted because the writer couldn't be bothered to put forth any more effort than that.
"AI does not polish human work. It acts more like an extruder, forcing anything fed into it into the same generic, formulaic shape—indistinguishable from writing produced by a lazy prompt from someone unwilling to put in any more effort."
There. AI-polished sentence.
> I run most professional statements and articles through LLMs before submission. This helps correct grammar and improves accessibility through better syntax (because I write exactly how I think).
> The article doesn't seem to mention consideration of AI for polishing human work.
Because it isn't a consideration. You are what they are looking for.
What was ever wrong with just producing a second draft of your work?
I had an LLM that could do it for me. Cheaper, faster, better. And yes, I re-edit parts I want afterwards.
I just generated some docs for a lib I'm writing and it says: > 9 % machine likely human-written
I’m very skeptical of these results. I run a pretty large group paper website on top of arXiv and we run pangram on papers. The numbers are not nearly this high.
One thing I see a lot is papers flagged as AI because they include llm rollouts in the paper as examples.
I ran two papers and three blog posts of mine through here, and all but one (correctly) flagged as 0-1% machine written. The other (human written) blog post was 20%. Pretty good afaict!
On my knees begging that the developers of AI detectors use them on their own writing and that at least pretended to mask the obvious LLM cliches in their blog posts.
Not convinced that this slop measurement is useful.
I asked Codex to generate an article with a high score and then asked Codex to (reverse?) hill climb that score. The original generation scored 97% and then the optimized one scored 1%. Both are pretty bad and read like slop.
https://gist.github.com/wbew/8a2bd6686bf875210f2244ac8ea65bf...
Awesome work, the detector seems accurate on a bunch of texts I tried, surprisingly even human/ai hybrid texts
That's honestly so good to hear, thank you.
the funniest part of these AI detectors is that if I were to upload any of einstin's paper's they will all be flagged as AI-written. it makes sense because it's part of their training data.
but this post makes me wonder, if more papers' are written with AI, or the shape of knowledge of converging?
> the funniest part of these AI detectors is that if I were to upload any of einstin's paper's they will all be flagged as AI-written.
Have you tried doing that or even read the article?
The article says that their detector flags 0.4% of pre-AI papers as AI-written.
If I paste the first page from this paper (https://www.fourmilab.ch/etexts/einstein/specrel/specrel.pdf) in https://unslop.run/app, I get a 0% chance that it was AI-written.
Even the translated version of the field equations paper has a 1% match only. Which I expected to be a tiny bit higher but still near 0.
the measurement breaks because the detector is probably also AI, and at that point you are just watching two language models argue about who wrote what
Good contents should always thrive, with or without AI
> and an honest account of the limitations.
Gotta be trolling :-D
> Here is the method, the results, and an honest account of the limitations.
Pot meet kettle?
People are saying this is a bad thing but is it really a problem? The compelling aspect of research is the data and/or description of work, not the writing. Papers probably should be written by AI so that they're clear and well presented, while the researchers should focus on generating good data. If there is no data or work behind the paper, we should question whether the research group needs funding.
but the assumption you are making is that the underlying ideas are compelling. In practice, people often decide to publish incremental and/or mediocre work for various reasons, and then dress these ideas up to seem as compelling as possible to get past peer review or make a press release.
at the least, this is problematic for peer-review because the absolute number of submissions outpaces the time availability of a finite number of expert reviewers. We cannot quickly generate expert human reviewers, and so the community might converge towards half-baked solutions (AI-generated reviews or rejection systems, vastly expanded referee pools, etc.) that tend to erode trust and and make scientific communities more adversarial.
No wonder reading all these paper is tiring these days.
I don't want to read slop generated by AI. AI written articles are generally low effort.
What's the point of looking when no one even cares?
consider the fraudster that went around suing people on the basis of his absurd claims of being bitcoin's creator. He's now transitioned to using AI to gather graduate degrees and is obtaining masters and doctoral degrees at a regular place and writing multiple 'papers' per day that are all quite obviously AI slop.
People report his cheating and publications and simply no one cares... and this is someone court adjudicated to have fabricated evidence in court on a massive scale, including through the use of AI.
But when it comes to the degrees and publication everyone involved that wanted paid got paid, and apparently that's all that matters.
Honestly academic writing is the only place where I think AI slop might be an improvement over the status quo writing style.
As an academic who is forced to review AI-generated conference/journal submissions, I can assure you this is not the case. LLM-heavy papers are terrible.
Even if they manage to avoid straight-up factual incorrectness, the writing is full of ambiguities and vagueness when you look closely. Meanwhile, space is wasted repeating the same claim in multiple ways, or explaining something simple.
They also seem unable to resist the hype/advertising tone, overselling the contribution while exaggerating the limitations of related work.
I would much rather read grammatically incorrect or awkward sentences.
While academic writing does have a few pointless historical conventions, the huge majority of "status quo writing style" is a logical consequence of 1) minimizing ambiguity, 2) organizing ideas coherently, 3) distinguishing opinion/interpretation from fact, and 4) providing enough detail to be reproducible.
The important question is:
So what?
Genuinely. I get that there may be some visceral reaction against this, but when I break it down, I mostly fail to see the problem. Seems like what is actually important is:
Compared to before, when a human reads it, do they -- or society -- get something good out of it? Is it worth it to add this to the "pantheon?"
If that's not what's happening enough, and if this doesn't describe the process -- then the problem lies elsewhere, no?
I recently desk-rejected a paper where every single citation in its Introduction was hallucinated. That means that the entire connection between what the author(s) did and how it relates to existing research was simply made up. I've never seen this happening before AI but now there's at least one paper in every cycle pulling something similar.
My problem therefore is: we are seeing more and more papers written with tools that are known to make up facts, citations, and even entire papers. And the number of papers has increased, too. I therefore see it less as "people are being more productive" and more "people are releasing bad science much faster than we can keep up with".
Yeah but that's an issue with the researcher putting out a bad paper, and it suggests you'll have to reject more papers. We wouldn't ban email because many of the emails are spam, it just means we need new tools to filter out junk. AI will allow researchers to be more productive all together and take less time to publish a paper, which is good.
The problem with spam is that it's not a technology like AI is. So I suggest taking cars instead.
Cars have plenty of advantages, and yet no one would say "the number of pedestrians killed by cars is rising, but that's an issue with the drivers". In fact, the opposite is true: from fines and school zones to speed bumps and bollards, we have accepted that cars bring structural problems with them that cannot be solved at the driver level alone.
> we need new tools to filter out junk
Agreed, but if my office suddenly was flooded with garbage my first thought wouldn't be "I need more, bigger trash cans" but rather "who brought all this junk here and why?". To simply assume that the garbage is a sudden natural phenomena that I have to live with seems, at the very least, unfair.
> Yeah but that's an issue with the researcher putting out a bad paper, and it suggests you'll have to reject more papers. We wouldn't ban email because many of the emails are spam, it just means we need new tools to filter out junk. AI will allow researchers to be more productive all together and take less time to publish a paper, which is good.
It's a signal:noise ratio thing. If 1 out of every 1000 AI-written papers are bad, it makes sense to put in a filter that auto-rejects any paper that has AI tells.
After all, if that 1 researcher was any good, he wouldn't have used AI to write the thing in the first place.
Publishing was always about getting past the filters. There's one more filter - "AI-generated content" - so do what you have to to get past it. IOW, write your own paper.
While working on my PhD, there was one guy in the field that would publish new, better results to Springer every time someone improved upon benchmarks. No code, no data, no conference publications, paywall restricted, unverifiable, always the best.
I am almost certain he was "hallucinating" the results. This was in the 2010s
There are well known issues in academic publishing, though I imagine it has become much noisier like open source
Surprisingly I agree with you. My opinion is: if it makes communicating research more effective, while not reducing the quality of the output substantially, I see no issue.
A possible conclusion for this could be: If the majority of CS papers is AI written, let's just accept this reality universally and stop worrying about it altogether.
I think the key question is more effective at what?
I see plenty of anecdotal evidence that models have been trained fantastically well—and getting better—at writing to trigger the right neurons in the human population to produce “This is interesting/informative/correct” responses in bulk.
Could their ability to produce those responses run far ahead of their ability to actually achieve the last in reality? Sure seems plausible, and then where are we?
Many people are bad at writing, including scientists. Improvements could be articulating a concept in a way that the reader will understand it clearly.
My hypothesis is that an LLM’s ability to concoct prose that will convince even an expert of the validity of an idea is largely independent of the LLM’s ability to validate the idea itself, or whether the idea is correct in the first place. Especially if it’s being prompted to do the former, not the latter.
My anecdotal evidence is the LLM-generated, inchoate technical dross that is routinely upvoted onto the hn front page. Much of it isn’t even coherent enough to be wrong, but the readership here finds it interesting!
I agree prose and validation are different. I dont know anyone who contests this.
> if it makes communicating research more effective, while not reducing the quality of the output substantially, I see no issue.
That's a big "If".
If a research is good, the author still has to clear all the hurdles in publishing. "Writing your own paper" is just one more hurdle.
> A possible conclusion for this could be: If the majority of CS papers is AI written, let's just accept this reality universally and stop worrying about it altogether.
That's just a different way of saying "if the majority of CS papers are crap, lets just accept this reality".
So, go on, publish away all your AI-induced "research", but the bar is slowly going to be raised anyway to reject that. That's how science always worked - when a bar is not sufficient to exclude the crap, it is raised.
I reviewed for ACL (big NLP conf) and another similar conference recently. Lots of crap (though there always has been). One superficially well-written paper was likely AI-plagiarized (i.e. the AI sampled an idea from prior work and rewrote it) and got desk rejected for it.
Reviewers were also totally unengaged. Of 20 reviews I read (from my reviewers or from reviewers on the same papers), maybe 2 were mediocre, and the rest were crap (though likely not AI).
The notion that science will somehow benefit from this is about as stupid an idea as you can have. Science relies on skepticism. AIs are not skeptical, and many folks are submitting papers because they stand to gain something, not because they are motivated to do good research or develop new understanding. Fields are being inundated with garbage that is maximally indistinguishable from real work (that's the training objective for LLMs). This in turn maximizes the cost of identifying bad work.
This is the same enshittification process that we see everywhere else. You get spam phone calls because there is no reason for a spammer not to call you. "Researchers" are submitting spam papers because there is no cost to doing so with some possible gain. Absent intervention, this eventually drives the community value of the network to zero (or potentially negative, if friction costs to switching are high).
And yet the quality of work at ACL is still higher than NeurIPS, and that's with ACL this year having over 40% of all posters looking nearly identical due to everyone claude coding their posters...
> if it makes communicating research more effective, while not reducing the quality of the output substantially
That’s a big if. We all know that’s not what’s happening.
I mean, sure. But what an absolutely insane predicate. "Not reducing the quality of the output substantially" is (as an AI might say...) load bearing there.
Other problems include: Signal to Noise Ratio going through the roof.
Yeah I was careful on purpose with my statement. :D
Is it? I guess look -- I'm an academic, I've read piles of articles such as these. Signal to noise, already not great.
Yes, I feel like there's room to improve things, I just strongly doubt that "using AI to detect AI" is a particularly useful thing to do here.
> My opinion is: if it makes communicating research more effective, while not reducing the quality of the output substantially, I see no issue.
That's a big if. ArXiv is not peer reviewed and LLMs basically interpolate and extrapolate text, which makes them essentially fluff generators. Even in the most charitable interpretation, LLMs enable those with nothing to say to say nothing while meeting surface-level style guides.
We can't know if real science is happening in the background but I'd wager that the majority of these papers is not complete slop but real findings with AI generated text used to communicate it. If it was just straight slop I would be really worried.
> We can't know if real science is happening in the background but I'd wager that the majority of these papers is not complete slop but real findings with AI generated text used to communicate it. If it was just straight slop I would be really worried.
Why don't you read them and see? The ones I looked at were clear slop.
> (...) but I'd wager that the majority of these papers is not complete slop (...)
That's a huge assumption, and one that goes against the whole notion of using LLMs to generate text. AI slop is by far the norm.
If a human writes a given paragraph, I know it has attained a minimum level of awareness in some human mind at some point. If an AI writes a paper, I have no guarantee that it has reached that minimum. We also fear, with some reason, a correlation between AI writing and possibly being hallucinated or non-existent. Not just because AI is bad and people should feel bad for using them, but because given that a paper is somehow fake, it is probably more likely to be made with AI, and running that backwards it is reasonable to take closer looks at AI papers.
If fake papers weren't already a big problem before AI and the fields had already been policing themselves adequately, if this was already a functioning high-trust domain, maybe we could ignore this a bit more, but the fields already manifestly had problems. People taking advantage of that are reasonably more likely to use AI. The pressures to publish or perish provide the voltage and the AIs are a rather convenient path-to-ground.
I agree in some sense that if a truth is published, it doesn't matter if the AI or a human published it. However there are perfectly reasonable reasons to be concerned that AI usage is correlated to not publishing truths, especially in a world where merely being human-generated was already not an adequate check against that.
Even if you can reject the aesthetic argument for non-fiction works (although read some of Dijkstra's papers for a good counterargument), it is still a problem because it breaks an important quality signalling mechanism.
Pre-LLMs, a paper with no spelling or grammar errors showed that somebody had put effort into writing and editing it. If they cared about the presentation, they probably also cared about the content. LLMs routinely produce nonsense that looks superficially like high-quality work.
There are far too many papers to read all of them. LLM slop is evidence that something is probably low quality. As the saying goes, "if you can't be bothered writing it, I can't be bothered reading." The rare outliers will get enough citations and recommendations to overcome this filter.
This is fallacious. The fact that someone used LLM to write their paper does not negate its findings.
It certainly casts suspicion on the ‘findings’ which may well be hallucinated or subtly wrong.
That's not the point being made.
That's not the point though, the point being made ist that an LLM written paper very likely doesn't actually find something despite looking like it does on the surface
Indeed. We'd likely find a similar pattern if we count spelling mistakes before and after autocorrect.
you just dont respect your readers, thats all
if 5% of the work that I'm reading to keep up is slop, that's a shame. that's time I spent puzzling over connections that weren't real, trying to impose some kind of logic on the arguments, and wondering why the data doesn't really support the conclusion.
if 50% of the work is nonsense, then there's a serious concern that we can't move forward at all.
Another important questions is:
What about the papers that graduate to proper publication?
Arxiv is full of pre-prints that anyone can upload.
> Arxiv is full of pre-prints that anyone can upload.
You now (at least for some categories) have to receive endorsement from someone who has multiple recent papers on arxiv in the same (or adjacent) category.
The problem is that humans start with credulity. AI hallucinates and makes up stuff some percentage of the time. Humans are not "default deny" when given information. Unleashing that was a disservice to mankind and has created a future filled with lies and people who are confident in them.
LLMs are in essence an attack on the concept of written language, harvesting and dissolving the social contract that underpins it; it is no longer safe to assume that text has intent, let alone content, simply because it is grammatically well-formed.
Perhaps the most darkly amusing consequence of this particular mania is that by poisoning the majority of our information environment with hallucinated slop, we have likely crippled the next several generations of machine-learning techniques before they're even invented! Small, locally-hostable LLMs will rattle along spewing spam long after the broader "genai bubble" pops, and building clean training datasets will permanently be more difficult and expensive.
Oh, to be technically correct:
AI hallucinates and makes up stuff 100% percent of the time. Never been a fan of that word for this.
Again, I fail to see the problem here that isn't solved by careful reading WHICH IS WHAT PEOPLE SHOULD BE DOING ANYWAY. I would like to see room for AI disclosure, maybe a statement of "this is how much AI I used."
But this blanket X% of this looks like AI? Again, so what?
If as a primary school student you had needed to reverify every fact /experiment that was presented in your textbooks, you would not have gotten very far.
Trust is very important to human progress.
I find the fact that 39% of papers use a style that signals lack of effort somewhat worrisome. Even if that’s 100% wrong, and they’re all high effort papers, the fact that they give off the same aura as low effort papers is a problem for the authors.
LLM-written text tends to have the property that it gives the impression of expertise and knowledge in excess of what the text actually contains.
In other words, it "sounds smart" without necessarily having anything to back it up. In even more critical terms, it's very good at bullshitting.
Unfortunately for us, the scientific community current relies on a certain amount of trust. (To do otherwise is very expensive! see: bitcoin). When you introduce a known-bullshitter to write your papers, every human in the loop effectively has to defend against an adversarial attack. Not just the readers at home, or the peer reviewers, but even the author needs to be wary that the facts and arguments coming out of the LLM are true and meaningful.
Personally, I've been a minor contributor to several high-profile papers. I don't know how every field does it, but in my experience, the corresponding author (generally the PI or other senior scientist), is responsible for the accuracy of the paper. They ultimately have to trust the people who did the work that the facts are true. Introducing LLMs into the mix make it more difficult for them to identify and review sections they are unsure of. (An honest person will typically write at a confidence level reflecting their certainty. LLMs do not do this in any reliable way.)
I've also found that LLMs frequently use metaphors that are unhelpful, or used out of context in a field that isn't familiar with them. This makes understanding the text more work, for no good reason. Introducing terms or definitions with low relevance reads as impressive at first glance, but avoiding the standard terminology in the field just adds confusion. (As an analogy, imagine if you were reading a CS paper that, for no particular reason, devoted a section to a new data structure called an "akimbo tree," which after much untangling, you realized was a reinvention of a randomized splay tree.)
Agreed. I know nothing about nuclear physics. I still doubt you could pick a random person off the street and have them convincingly pose as a nuclear physicist to explain a "nuclear physics" concept to me. I doubt you could do it with a random PhD from a non-physics field. An LLM could probably convince me even if 90% of the content of the explanation is subtly or blatantly incorrect.
Weimar moment of science
More like the death of the public Internet before our eyes
Good contents will always thrive, with or without AI