The difference I see between a company dealing with this as opposed to an open source community dealing with this is that the company can fire employees as a reactive punishment. Drive-by open source contributions cost very little to lob over and can come from a wide variety of people you don't have much leverage over, so maintainers end up making these specific policies to prevent them from having to react to the thousandth person who used "The AI did it" as an excuse.
When you shout "use AI or else!" from a megaphone, don't expect everyone to interpret it perfectly. Especially when you didn't actually understand what you were saying in the first place.
Shouldn't this go without saying though? At some point someone has to review the code and they see a human name as the sender of the PR. If that person sees the work is bad, isn't it just completely unambiguous that the person whose name is on the PR is responsible for that? If someone responded "but this is AI generated" I would feel justified just responding "it doesn't matter" and passing the review back again.
And the rest (what's in the LLVM policy) should also fall out pretty naturally from this? If someone sends me code for review, and have the feeling they haven't read it themselves, I'll say "I'm not reviewing this and I won't review any more of your PRs unless you promise you reviewed them yourself first".
The fact that people seem to need to establish these things as an explicit policy is a little concerning to me. (Not that it's a bad idea at all. Just worried that there was a need).
You would think it's common sense but I've received PRs that the author didn't understand and when questioned told me that the AI knows more about X than they do so they trust its judgement.
A terrifying number of people seem to think that the damn thing is magic and infallible.
>I neither tell people to use AI, nor tell them not to use it, and in practice people have not been using AI much for whatever that is worth.
I find this bit confusing. Do you provide enterprise contracts for AI tools? Or do you let employees use their personal accounts with company data? It seems all companies have to be managing this somehow at this point.
Some people who just want to polish their resume will feed any questions/feedback back into the AI that generated their slop. That goes back and forth a few times until the reviewing side learns that the code authors have no idea what they're doing. An LLM can easily pretend to "stand behind its work" if you tell it to.
A company can just fire someone who doesn't know what they're doing, or at least take some kind of measure against their efforts. On a public project, these people can be a death by a thousand cuts.
The best example of this is the automated "CVE" reports you find on bug bounty websites these days.
What good does it really do me if they "stand behind their work"? Does that save me any time drudging through the code? No, it just gives me a script for reprimanding. I don't want to reprimand. I want to review code that was given to me in good faith.
At work once I had to review some code that, in the same file, declared a "FooBar" struct and a "BarFoo" struct, both with identical field names/types, and complete with boilerplate to convert between them. This split served no purpose whatsoever, it was probably just the result of telling an agent to iterate until the code compiled then shipping it off without actually reading what it had done. Yelling at them that they should "stand behind their work" doesn't give me back the time I lost trying to figure out why on earth the code was written this way. It just makes me into an asshole.
It adds accountability, which is unfortunately something that ends up lacking in practice.
If you write bad code that creates a bug, I expect you to own it when possible. If you can't and the root cause is bad code, then we probably need to have a chat about that.
Of course the goal isn't to be a jerk. Lots of normal bugs make it through in reality. But if the root cause is true negligence, then there's a problem there.
If you asked Claude to review the code it would probably have pointed out the duplication pretty quickly. And I think this is the thing - if we are going to manage programmers who are using LLM's to write code, and have to do reviews for their code, reviewers aren't going to be able to do it for much longer without resorting to LLM assistance themselves to get the job done.
It's not going to be enough to say - "I don't use LLM's".
Yelling at incompetent or lazy co-workers isn't your responsibility, it's your manager's. Escalate the issue and let them be the asshole. And if they don't handle it, well it's time to look for a new job.
> At my company I just tell people “You have to stand behind your work”
Since when has that not been the bare minimum. Even before AI existed, and even if you did not work in programming at all, you sort of have to do that as a bare minimum. Even if you use a toaster and your company guidelines suggest you toast every sandwich for 20 seconds, if following every step as per training results in a lump of charcoal for bread, you can’t serve it up to the customer. At the end of the day, you make the sandwich, you’re responsible for making it correctly.
Using AI as a scapegoat for sloppy and lazy work needs to be unacceptable.
Of course it’s the minimum standard, and it’s obvious if you view AI as a tool that a human uses.
But some people view it as a seperate entity that writes code for you. And if you view AI like that, then “The AI did it” becomes an excuse that they use.
Bad example. If the toaster carbonized bread in 20 seconds it's defective, likely unsafe, possibly violates physics, certainly above the pay grade of a sandwich-pusher.
Taking responsibility for outcomes is a powerful paradigm but I refuse to be held responsible for things that are genuinely beyond my power to change.
> If the toaster carbonized bread in 20 seconds it's defective, likely unsafe, possibly violates physics, certainly above the pay grade of a sandwich-pusher.
If the toaster is defective, not using it, identifying how to use it if it’s still usable or getting it replaced by reporting it as defective are all well within the pay grade of a sandwich pusher as well as part of their responsibilities.
And you’re still responsible for the sandwich. You can’t throw up your arms and say “the toaster did it”. And that’s where it’s not tangential to the AI discussion.
Toaster malfunctioning is beyond your control, but whether you serve up the burnt sandwich is absolutely within your control, which you will be and should be held responsible for.
No it’s not. If you burn a sandwich, you make new sandwich. Sandwiches don’t abide by the laws of physics. If you call a physicist and tell them you burnt your sandwich, they won’t care.
I think it depends on the pay. You pay below the living wage? Better live with your sla.. ah employees.. serving charcoal. You pay them well above the living wage? Now we start to get into they should care-territory.
But "AI did it" is not immediate you are out thing? If you cannot explain why something is made the way you committed to git, we can just replace you with AI right?
One of the reasons I left a senior management position at my previous 500-person shop was that this was being done, but not even accurately. Copilot usage via the IDE wasn't being tracked; just the various other usage paths.
It doesn't take long for shitty small companies to copy the shitty policies and procedures of successful big companies. It seems even intelligent executives can't get correlation and causation right.
The title should be changed to "LLVM AI tool policy: human in the loop".
At the moment it's "We don't need more contributors who aren't programmers to contribute code," which is from a reply and isn't representative of the original post.
The HN guidelines say: please use the original title, unless it is misleading or linkbait; don't editorialize.
Seconded, the original title ("We don't need more contributors who aren't programmers to contribute code") caricatured what seems like an eminently measured and sensible policy change.
I'll say it as it is: if you can't read code, but have a friend write it for you, you are in fact the wrong person to submit it, since you are the most exhausting person to deal with.
A human in the loop that doesn't understand what is going on but still pushes isn't only useless, but actively harmful.
It’s happening a lot with me at work. I am a programmer working largely with a hardware team and now they’re contributing large changes that I’m supposed to just roll with the punches. Management loves it
People who are smart enough to think that far ahead are also smart enough not to fall into the “ai can do all jobs perfectly all the time and just need my divine guidance” trap.
I 100% think that every programmer is responsible for the code they submit for PR, whether they used AI or not. Whether they used Google/StackExchange/etc or not.
They are responsible for it.
However, here's a different situation:
If the company you're working for requires you to use LLMs to code, I think it's 100% defensible to say "Oh, the AI did that" when there's a problem because the company required its usage. You would have done it better if the company hadn't forced you to cut corners.
I like this, especially because it focuses on the actual problem these contributioms cause, not the AI tools themselves.
I especially like the term "extractive contribution." That captures the issue very well and covers even non-AI instances of the problem which were already present before LLMs.
Making reviewer friendly contributions is a skill on its own and makes a big difference.
We use plenty of models to calculate credit risk, but we never let the model sign the contract.
An algorithm can't go to court, and it can't apologize to a bankrupt family.
"Human in the Loop" isn't just about code quality. It's about liability.
If production breaks, we need to know exactly which human put their reputation on the line to merge it.
Accountability is still the one thing you can't automate.
As a a developer, you're not only responsible for contributing code. But verifying that it works. I've seen this practice be put in place on other teams, not just with LLM's, but with devs who contribute bugfixes without understanding the problem
Their copyright clause reflects my own quandry about LLM usage:
I am responsible for ensuring copyright has not been violated with LLM generated code I publish. However, proving the negative, i.e. the code is not copyrighted is almost impossible.
I have experienced this - Claude came up with an almost perfect solution to a tricky problem, ten lines to do what I've seen done in multiple KLOC, and I later found the almost identical solution in copyrighted material.
Personally I have not experienced it, but I have heard of people scanning for LGPL library usage in iOS apps, then essentially extorting the developers for their source code.
I can't find the specific article now, but I am extremely careful to avoid anything GPL or LGPL.
How is that extortion? If you use a library/project with a licence you need to abide by its terms. If you don't want to do that, then either write it yourself or find an alternative. People asking for the source code is not extortion, they're fully within their rights to do so.
Copyright is for creative work so if it really is the best way to do it, you should be safe even if the AI copied the idea from somebody else. You can't use copyright to restrict access to useful technology like a patent.
That's very useful feedback. I suspect if the solution is irreducible it's OK, which in this case is close to true (this is for bit-unpacking integers):
// bytes are added to the buffer until a value of bitwidth is available
for _ in 0..<bitWidth {
let byte = try UInt8(parsing: &input)
buffer |= UInt64(byte) << bitsInBuffer
bitsInBuffer += 8
// Values of bitwidth are right-shifted off the buffer.
while bitsInBuffer >= bitWidth && outPos < numValues {
let value = Int(buffer & mask)
out[outPos + outOffset] = value
outPos += 1
buffer >>= bitWidth
bitsInBuffer -= bitWidth
}
}
If you don't speak Hungarian and you have a Hungarian friend write a text for you that you submit to a collection of worthy Hungarian poetry, do you really think you are the correct person to answer questions about the text that was written?
You know what is supposed to be in it, but that's it. You can't judge the quality of the text and how it is fitting the rest at all. You can only trust your friend. And it is okay if you do, just don't pull others into it.
IMO it is extremely rude to even try to pull this off and if you do, shame on you for wasting peoples time.
Its everywhere. I worked with a micro-manager CTO who farmed code review out to claude, which of course, when instructed to find issues with my code, did so.
One thing I didn't like was the copy/paste response for violations.
It makes sense to have one. Just the text they propose uses what I'd call insider terms, and also terms that sort of put down the contributor.
And while that might be appropriate at the next level of escalation, the first level stock text should be easier for the outside contributor to understand, and should better explain the next steps for the contributor to take.
One (narrow) circumstance to make the process of reviewing a large contribution — with significant aid from LLM — easier to review is to jump on a call with the reviewer, explain what the change is, and answer their questions on why is it necessary and what it brings to the table. This first pass is useful for a few reasons:
1. It shifts the cognitive load from the reviewer to the author because now the author has to do an elevator pitch and this can work sort of like a "rubber duck" where one would likely have to think about these questions up front.
2. In my experience this is a much faster to do this than a lonesome review with no live input from the author on the many choices they made.
First pass and have a reviewer give a go/no-go with optional comments on design/code quality etc.
Have you ever done that with new contributors to open source projects? Typically things tend to be asynchronous but maybe it's a practice I've just not encountered in such context.
I've done that in contributions to unknown people's repo but not necessarily open source ones. I believe that this is quite under valued for the reasons I listed.
In addition, 1:1 contact can speed up things immensely in such situations because most activity on a change happens very soon after the first change is made and that initial voluminous back and forth can be faster than typing to have a back and forth on github for a PR.
Oh wow. That something like this is necessary is kind of sad. At first (while reading the title), I thought they just didn’t want AI-generated contributions at all (which would be understandable as well). But all they are actually asking for is that one understands (and label) the contributions they submit, regardless of whether those are AI-generated, their own work, or maybe even written by a cat (okay, that last one was added by me ;).
Reading through the (first few) comments and seeing people defending the use of pure AI tools is really disheartening. I mean, they’re not asking for much just that one reviews and understands what the AI produced for them.
If it made it into your pull request, YOU wrote it, and it it’ll be part of your performance review. Cursor doesn’t have a performance review. Simple as
I’ve seen a bunch of my colleagues say this when I ask about the code they’ve submitted for review. Incredibly frustrating, but likely to become more common
This AI usage is like a turbo-charger for the Dunning–Kruger effect, and we will see these policies crop up more and more, as technical people become more and more harassed and burnt out by AI slop.
I also recently wrote a similar policy[0] for my fork of a codebase. I had to write this because the original developer took the AI pill, and starting committing totally broken code that was fulled of bugs, and doubled down when asked about it [1].
On an analysis level, I recently commented[2] that "Non-coders using AI to program are effectively non-technical people, equipped with the over-confidence of technical people. Proper training would turn those people into coders that are technical people. Traditional training techniques and material cannot work, as they are targeted and created with technical people in mind."
But what's more, we're also seeing programmers use AI creating slop. They're effectively technical people equipped with their initial over-confidence, highly inflated by a sense of effortless capability. Before AI, developers were once (sometimes) forced to pause, investigate, and understand, and now it's just easier and more natural to simply assume they grasp far more than they actually do, because @grok told them this is true.
It is insane how pro-industry people argue with corpspeak and LLMspeak and are still getting serious responses. This is where open source discussions have arrived now, and no one dares to tell them to STFU because of their corporate sponsors:
"To critique the fortress is not enough. We must offer a blueprint for a better structure: a harbor. A harbor, unlike a fortress, does not have a simple binary function of letting things in or keeping them out. It is an active, intelligent system with channels, docks, workshops, and expert pilots, all designed to guide valuable cargo safely to shore, no matter the state of the vessel that carries it. This is the model we must adopt for open source in a post-AI world. The anxiety over “extractive” contributions is real, but the solution is not a higher wall; it is a smarter intake process."
It is insane that this is happening in one of the most essential piece of software. This is a much needed step to decrease the increase of slop contribution. It's more work for the maintainer to review all this mess.
At least that's much better than not being able to answer them. If LLMs are truly intelligent enough to justify their contributions (including copyrights!), why should we treat them differently from human contributions?
For what it's worth, this is essentially the policy my current and most recent previous workplace followed. (My employers before that were pre-LLMs.)
If you are the one with your name on the PR, it's your code and you have to understand it. If you don't understand it at least well enough to speak intelligently about it, you aren't ready to submit it for review. If you ask Copilot, Cursor, or whatever to generate a PR for you, it still must be reviewed and approved by you and another engineer who acts as your reviewer.
I haven't heard a lot of pushback on this; it feels like common sense to me. It's effectively the same rules we'd use if somebody who wasn't an engineer wanted to submit code; they'd need to go through an engineer to do it.
LLM usage has increased our throughput and the quality of our code thus far, but without these rules (and people following the spirit of them, being bought in to their importance), I really don't think it would.
I encourage you to raise this policy with your management, if you think you can get them to listen, and demonstrate how it might help. I would be very frustrated if my colleagues were submitting AI-generated code without thinking it through.
Open source projects like LLVM need to do this as it is one of those projects that is widely used in the software supply chain, on the level that needs protection from contributors who do not understand the code they are writing or cannot defend their changes.
There needs to be a label which designates such open source projects that is so important and adopted throughout the industry that not anyone can throw patches to it without understanding what it does, and why they need it.
I am so exhausted by reviewing the AI slop from other “developers”. For a while I was trying to be a good sport and point out where it was just wrong or doing things that were unnecessary or inefficient. I’m at the point of telling people to not bother using an AI. I don’t have the time or energy to deal with it. It’s like a missile defense system that costs a million dollars to intercept but the incoming projectile cost $10 for your adversary. It’s not sustainable.
"Vibe coding" (i.e. the kind of code that is statistically 'plausible' that sometimes works and the user doesn't look at the code but tries it to see if it works to their liking (with no tests) )
Was the worst thing to happen to programming, computer science I have seen, good for prototypes but not production software, and especially for important projects like LLVM.
It is good to gatekeep this slop from LLVM before it gets out of control.
I think it's great that this AI slop fatigue happened so quickly.
Now we can still identify them easily, and I am maintaining bookmarks of non slop codebases, so that I know which software to avoid.
I encourage everyone to do the same because the slopcode fallout will be inevitable, and likely be the most harmful thing that ever happened to open source as a philosophy.
We need to change our methodology of how to develop verifiable and specdriven software, because TDD isn't good enough to catch this. Something that is able to verify logical conclusions and implications (aka branches) and not only methods and their types/signatures.
Not just codebases, but developers too. Keep your eyes open: if someone visibly embraces slop, you know they're a clown. Don't let clowns touch your repos, and do your best to cut out dependencies on anything maintained by clowns.
Personally, I’ve got fatigue at the phrase “AI slop”. It’s used as a catch all to dismiss the content due to the source, regardless of the quality or suitability when taken in context.
Just like everything else these days the responses skew towards both extremes on the spectrum and people hand waving away the advancements is just as annoying as those who are zealots on the other end.
> automated review tools that publish comments without human review are not allowed
This seems like a curious choice. At my company we have both Gemini and cursor (I’m not sure which model under the hood on that) review agents available. Both frequently raise legitimate points. Im sure they’re abusable, I just haven’t seen it
one reason i can imagine for this choice is that human review distributes new knowledge among human maintainers. Automated review might discourage that valuable interaction.
The comment is an artefact of an inherently valuable process, not the sole objective. So I'd prefer code is reviewed by a human who maybe initiates a discussion with an agent. I'd hope this minor workflow detail encourages the human to take a more active lead, rather than risk mere ceremonial approval.
The policy isn’t about whether those tools raise good points. It’s about not letting agents act autonomously in project spaces. Human reviewed, opt in use is explicitly allowed.
At my company I just tell people “You have to stand behind your work”
And in practice that means that I won’t take “The AI did it” as an excuse. You have to stand behind the work you did even if you used AI to help.
I neither tell people to use AI, nor tell them not to use it, and in practice people have not been using AI much for whatever that is worth.
The difference I see between a company dealing with this as opposed to an open source community dealing with this is that the company can fire employees as a reactive punishment. Drive-by open source contributions cost very little to lob over and can come from a wide variety of people you don't have much leverage over, so maintainers end up making these specific policies to prevent them from having to react to the thousandth person who used "The AI did it" as an excuse.
When you shout "use AI or else!" from a megaphone, don't expect everyone to interpret it perfectly. Especially when you didn't actually understand what you were saying in the first place.
Shouldn't this go without saying though? At some point someone has to review the code and they see a human name as the sender of the PR. If that person sees the work is bad, isn't it just completely unambiguous that the person whose name is on the PR is responsible for that? If someone responded "but this is AI generated" I would feel justified just responding "it doesn't matter" and passing the review back again.
And the rest (what's in the LLVM policy) should also fall out pretty naturally from this? If someone sends me code for review, and have the feeling they haven't read it themselves, I'll say "I'm not reviewing this and I won't review any more of your PRs unless you promise you reviewed them yourself first".
The fact that people seem to need to establish these things as an explicit policy is a little concerning to me. (Not that it's a bad idea at all. Just worried that there was a need).
You would think it's common sense but I've received PRs that the author didn't understand and when questioned told me that the AI knows more about X than they do so they trust its judgement.
A terrifying number of people seem to think that the damn thing is magic and infallible.
>I neither tell people to use AI, nor tell them not to use it, and in practice people have not been using AI much for whatever that is worth.
I find this bit confusing. Do you provide enterprise contracts for AI tools? Or do you let employees use their personal accounts with company data? It seems all companies have to be managing this somehow at this point.
Some people who just want to polish their resume will feed any questions/feedback back into the AI that generated their slop. That goes back and forth a few times until the reviewing side learns that the code authors have no idea what they're doing. An LLM can easily pretend to "stand behind its work" if you tell it to.
A company can just fire someone who doesn't know what they're doing, or at least take some kind of measure against their efforts. On a public project, these people can be a death by a thousand cuts.
The best example of this is the automated "CVE" reports you find on bug bounty websites these days.
What good does it really do me if they "stand behind their work"? Does that save me any time drudging through the code? No, it just gives me a script for reprimanding. I don't want to reprimand. I want to review code that was given to me in good faith.
At work once I had to review some code that, in the same file, declared a "FooBar" struct and a "BarFoo" struct, both with identical field names/types, and complete with boilerplate to convert between them. This split served no purpose whatsoever, it was probably just the result of telling an agent to iterate until the code compiled then shipping it off without actually reading what it had done. Yelling at them that they should "stand behind their work" doesn't give me back the time I lost trying to figure out why on earth the code was written this way. It just makes me into an asshole.
It adds accountability, which is unfortunately something that ends up lacking in practice.
If you write bad code that creates a bug, I expect you to own it when possible. If you can't and the root cause is bad code, then we probably need to have a chat about that.
Of course the goal isn't to be a jerk. Lots of normal bugs make it through in reality. But if the root cause is true negligence, then there's a problem there.
AI makes negligence much easier to achieve.
If you asked Claude to review the code it would probably have pointed out the duplication pretty quickly. And I think this is the thing - if we are going to manage programmers who are using LLM's to write code, and have to do reviews for their code, reviewers aren't going to be able to do it for much longer without resorting to LLM assistance themselves to get the job done.
It's not going to be enough to say - "I don't use LLM's".
Yelling at incompetent or lazy co-workers isn't your responsibility, it's your manager's. Escalate the issue and let them be the asshole. And if they don't handle it, well it's time to look for a new job.
>> Yelling at incompetent or lazy co-workers isn't your responsibility, it's your manager's
First: Somebody hired these people, so are they really "lazy and incompetent"?
Second: There is no one who's "job" is to yell at incompetent or lazy workers.
> At my company I just tell people “You have to stand behind your work”
Since when has that not been the bare minimum. Even before AI existed, and even if you did not work in programming at all, you sort of have to do that as a bare minimum. Even if you use a toaster and your company guidelines suggest you toast every sandwich for 20 seconds, if following every step as per training results in a lump of charcoal for bread, you can’t serve it up to the customer. At the end of the day, you make the sandwich, you’re responsible for making it correctly.
Using AI as a scapegoat for sloppy and lazy work needs to be unacceptable.
Of course it’s the minimum standard, and it’s obvious if you view AI as a tool that a human uses.
But some people view it as a seperate entity that writes code for you. And if you view AI like that, then “The AI did it” becomes an excuse that they use.
"Yes, but you submitted it to us."
If you're illiterate and can't read maybe don't submit the text someone has written for you if you can't even parse the letters.
The policy in TFA is a nicer way of saying that.
Bad example. If the toaster carbonized bread in 20 seconds it's defective, likely unsafe, possibly violates physics, certainly above the pay grade of a sandwich-pusher.
Taking responsibility for outcomes is a powerful paradigm but I refuse to be held responsible for things that are genuinely beyond my power to change.
This is tangential to the AI discussion though.
> If the toaster carbonized bread in 20 seconds it's defective, likely unsafe, possibly violates physics, certainly above the pay grade of a sandwich-pusher.
If the toaster is defective, not using it, identifying how to use it if it’s still usable or getting it replaced by reporting it as defective are all well within the pay grade of a sandwich pusher as well as part of their responsibilities.
And you’re still responsible for the sandwich. You can’t throw up your arms and say “the toaster did it”. And that’s where it’s not tangential to the AI discussion.
Toaster malfunctioning is beyond your control, but whether you serve up the burnt sandwich is absolutely within your control, which you will be and should be held responsible for.
No it’s not. If you burn a sandwich, you make new sandwich. Sandwiches don’t abide by the laws of physics. If you call a physicist and tell them you burnt your sandwich, they won’t care.
I think it depends on the pay. You pay below the living wage? Better live with your sla.. ah employees.. serving charcoal. You pay them well above the living wage? Now we start to get into they should care-territory.
But "AI did it" is not immediate you are out thing? If you cannot explain why something is made the way you committed to git, we can just replace you with AI right?
> we can just replace you with AI right?
Accountability and IP protection is probably the only thing saving someone in that situation.
Why stop there? We can replace git with AI too!
If you generate the code each time you need it, all version control becomes obsolete.
They'll version control the prompts because the requirements change.
Not if we AI-generate the requirements!
The smartest and most sensible response.
I'm dreading the day the hammer falls and there will be AI-use metrics implemented for all developers at my job.
It's already happened at some very big tech companies
One of the reasons I left a senior management position at my previous 500-person shop was that this was being done, but not even accurately. Copilot usage via the IDE wasn't being tracked; just the various other usage paths.
It doesn't take long for shitty small companies to copy the shitty policies and procedures of successful big companies. It seems even intelligent executives can't get correlation and causation right.
The title should be changed to "LLVM AI tool policy: human in the loop".
At the moment it's "We don't need more contributors who aren't programmers to contribute code," which is from a reply and isn't representative of the original post.
The HN guidelines say: please use the original title, unless it is misleading or linkbait; don't editorialize.
Seconded, the original title ("We don't need more contributors who aren't programmers to contribute code") caricatured what seems like an eminently measured and sensible policy change.
Additionally, it comes across as pretty hostile toward new contributors, which isn’t the intent of the article at all.
I'll say it as it is: if you can't read code, but have a friend write it for you, you are in fact the wrong person to submit it, since you are the most exhausting person to deal with.
A human in the loop that doesn't understand what is going on but still pushes isn't only useless, but actively harmful.
My original link was to a comment in this thread which I quoted that from. The link's now been changed to the main thread.
I'd like to know why the title hasn't been fixed.
It's depressing this has to be spelled out. You'd think people would be smart enough not to harass maintainers with shit they don't understand.
It’s happening a lot with me at work. I am a programmer working largely with a hardware team and now they’re contributing large changes that I’m supposed to just roll with the punches. Management loves it
People who are smart enough to think that far ahead are also smart enough not to fall into the “ai can do all jobs perfectly all the time and just need my divine guidance” trap.
Not if they're not programmers!
I think it's part of the "AI replacing software developers" hype.
Many beginners or aspiring developers swallow this whole and simply point an AI to a problem and submit the generated slop to maintainers.
People who rely on a computer algorithm to literally think for them are not going to be very smart.
I 100% think that every programmer is responsible for the code they submit for PR, whether they used AI or not. Whether they used Google/StackExchange/etc or not.
They are responsible for it.
However, here's a different situation:
If the company you're working for requires you to use LLMs to code, I think it's 100% defensible to say "Oh, the AI did that" when there's a problem because the company required its usage. You would have done it better if the company hadn't forced you to cut corners.
I like this, especially because it focuses on the actual problem these contributioms cause, not the AI tools themselves.
I especially like the term "extractive contribution." That captures the issue very well and covers even non-AI instances of the problem which were already present before LLMs.
Making reviewer friendly contributions is a skill on its own and makes a big difference.
As a loan officer, I like this policy.
We use plenty of models to calculate credit risk, but we never let the model sign the contract. An algorithm can't go to court, and it can't apologize to a bankrupt family.
"Human in the Loop" isn't just about code quality. It's about liability. If production breaks, we need to know exactly which human put their reputation on the line to merge it.
Accountability is still the one thing you can't automate.
This account seems like LLM slop looking at the post history. Who starts every post ‘from Japan’?
I don’t want to interact with hidden chat bots on HN. The irony of this comment about accountability is also frustrating.
The code writers increased exponentially overnight. The number of reviewers is constant (slightly reduced due to layoffs).
And so did the slop.
As a a developer, you're not only responsible for contributing code. But verifying that it works. I've seen this practice be put in place on other teams, not just with LLM's, but with devs who contribute bugfixes without understanding the problem
You also have a second responsibility with LLM generated code you publish: it must not be copyrighted.
Their copyright clause reflects my own quandry about LLM usage:
I am responsible for ensuring copyright has not been violated with LLM generated code I publish. However, proving the negative, i.e. the code is not copyrighted is almost impossible.
I have experienced this - Claude came up with an almost perfect solution to a tricky problem, ten lines to do what I've seen done in multiple KLOC, and I later found the almost identical solution in copyrighted material.
I’m curious have you ever been burnt or seen anyone burnt by copyright infringements in code?
Sometimes it’s super obvious because a game company steals code from a previous employer, but I have never seen this play out in entreprise software.
Personally I have not experienced it, but I have heard of people scanning for LGPL library usage in iOS apps, then essentially extorting the developers for their source code.
I can't find the specific article now, but I am extremely careful to avoid anything GPL or LGPL.
It's unlikely to be a problem until an app is highly successful, but once that happens people will grasp at whatever straws they can, e.g. https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America%2....
How is that extortion? If you use a library/project with a licence you need to abide by its terms. If you don't want to do that, then either write it yourself or find an alternative. People asking for the source code is not extortion, they're fully within their rights to do so.
Copyright is for creative work so if it really is the best way to do it, you should be safe even if the AI copied the idea from somebody else. You can't use copyright to restrict access to useful technology like a patent.
That's very useful feedback. I suspect if the solution is irreducible it's OK, which in this case is close to true (this is for bit-unpacking integers):
> However, proving the negative, i.e. the code is not copyrighted is almost impossible.
Nonsense, it's trivially easy if you wrote the code yourself. You hold the copyright.
If you had some LLM puke out the code for you, well, then you have no idea. So you can't contribute that puke.
Did you read my comment? I was specifically talking about publishing LLM generated code:
> I am responsible for ensuring copyright has not been violated with LLM generated code I publish
Your comment:
> If you had some LLM puke out the code for you, well, then you have no idea. So you can't contribute that puke.
That's not true. You can if it's not violating copyright. That "puke" as you put it comes in many flavours.
I feel like the title should definitely be changed.
Requiring people who contribute to "able to answer questions about their work during review." is definitely reasonable.
The current title of "We don't need more contributors who aren't programmers to contribute code" is an entirely different discussion.
Is it tho?
If you don't speak Hungarian and you have a Hungarian friend write a text for you that you submit to a collection of worthy Hungarian poetry, do you really think you are the correct person to answer questions about the text that was written?
You know what is supposed to be in it, but that's it. You can't judge the quality of the text and how it is fitting the rest at all. You can only trust your friend. And it is okay if you do, just don't pull others into it.
IMO it is extremely rude to even try to pull this off and if you do, shame on you for wasting peoples time.
Its everywhere. I worked with a micro-manager CTO who farmed code review out to claude, which of course, when instructed to find issues with my code, did so.
With little icons of rocket ships and such.
Good idea. I have just proposed that Raku adopts this policy for core and module submissions https://github.com/Raku/problem-solving/issues/510
The new policy looks very reasonable and fair. Unfortunately I'd be surprised if the bad apples will read the policy before spamming their "help".
Looks like a good policy to me.
One thing I didn't like was the copy/paste response for violations.
It makes sense to have one. Just the text they propose uses what I'd call insider terms, and also terms that sort of put down the contributor.
And while that might be appropriate at the next level of escalation, the first level stock text should be easier for the outside contributor to understand, and should better explain the next steps for the contributor to take.
One (narrow) circumstance to make the process of reviewing a large contribution — with significant aid from LLM — easier to review is to jump on a call with the reviewer, explain what the change is, and answer their questions on why is it necessary and what it brings to the table. This first pass is useful for a few reasons:
1. It shifts the cognitive load from the reviewer to the author because now the author has to do an elevator pitch and this can work sort of like a "rubber duck" where one would likely have to think about these questions up front.
2. In my experience this is a much faster to do this than a lonesome review with no live input from the author on the many choices they made.
First pass and have a reviewer give a go/no-go with optional comments on design/code quality etc.
> jump on a call with the reviewer
Have you ever done that with new contributors to open source projects? Typically things tend to be asynchronous but maybe it's a practice I've just not encountered in such context.
I've done that in contributions to unknown people's repo but not necessarily open source ones. I believe that this is quite under valued for the reasons I listed.
In addition, 1:1 contact can speed up things immensely in such situations because most activity on a change happens very soon after the first change is made and that initial voluminous back and forth can be faster than typing to have a back and forth on github for a PR.
Oh wow. That something like this is necessary is kind of sad. At first (while reading the title), I thought they just didn’t want AI-generated contributions at all (which would be understandable as well). But all they are actually asking for is that one understands (and label) the contributions they submit, regardless of whether those are AI-generated, their own work, or maybe even written by a cat (okay, that last one was added by me ;).
Reading through the (first few) comments and seeing people defending the use of pure AI tools is really disheartening. I mean, they’re not asking for much just that one reviews and understands what the AI produced for them.
Contributors should never find themselves in the position of saying “I don’t know, an LLM did it”
I would never have thought that someone could actually write this.
I get this at work, frequently.
“Oh, cursor wrote that.”
If it made it into your pull request, YOU wrote it, and it it’ll be part of your performance review. Cursor doesn’t have a performance review. Simple as
I could see this coming when I quit. I would not have been able to resist insisting people doing that be fired.
Yea, this is just lazy. If you don't know what it does and how then you shouldn't submit it at all.
I’ve seen a bunch of my colleagues say this when I ask about the code they’ve submitted for review. Incredibly frustrating, but likely to become more common
See this thread from a while back: https://news.ycombinator.com/item?id=46039274
Good policy.
This AI usage is like a turbo-charger for the Dunning–Kruger effect, and we will see these policies crop up more and more, as technical people become more and more harassed and burnt out by AI slop.
I also recently wrote a similar policy[0] for my fork of a codebase. I had to write this because the original developer took the AI pill, and starting committing totally broken code that was fulled of bugs, and doubled down when asked about it [1].
On an analysis level, I recently commented[2] that "Non-coders using AI to program are effectively non-technical people, equipped with the over-confidence of technical people. Proper training would turn those people into coders that are technical people. Traditional training techniques and material cannot work, as they are targeted and created with technical people in mind."
But what's more, we're also seeing programmers use AI creating slop. They're effectively technical people equipped with their initial over-confidence, highly inflated by a sense of effortless capability. Before AI, developers were once (sometimes) forced to pause, investigate, and understand, and now it's just easier and more natural to simply assume they grasp far more than they actually do, because @grok told them this is true.
[0]: https://gixy.io/contributing/#ai-llm-tooling-usage-policy
[1]: https://joshua.hu/gixy-ng-new-version-gixy-updated-checks#qu...
[2]: https://joshua.hu/ai-slop-story-nginx-leaking-dns-chatgpt#fi...
It is insane how pro-industry people argue with corpspeak and LLMspeak and are still getting serious responses. This is where open source discussions have arrived now, and no one dares to tell them to STFU because of their corporate sponsors:
"To critique the fortress is not enough. We must offer a blueprint for a better structure: a harbor. A harbor, unlike a fortress, does not have a simple binary function of letting things in or keeping them out. It is an active, intelligent system with channels, docks, workshops, and expert pilots, all designed to guide valuable cargo safely to shore, no matter the state of the vessel that carries it. This is the model we must adopt for open source in a post-AI world. The anxiety over “extractive” contributions is real, but the solution is not a higher wall; it is a smarter intake process."
It is insane that this is happening in one of the most essential piece of software. This is a much needed step to decrease the increase of slop contribution. It's more work for the maintainer to review all this mess.
Then the vibe coder will ask an LLM to answer questions about the contribution.
I don't know how it is for you, but I find it rather easy to tell when someone doesn't actually understand what they're talking about.
At least that's much better than not being able to answer them. If LLMs are truly intelligent enough to justify their contributions (including copyrights!), why should we treat them differently from human contributions?
That would also include humans not taking credit for AI's work.
I only wish my workplace had the same policy. I’m so tired of reviewing slop where the submitter has no idea what it’s even for.
For what it's worth, this is essentially the policy my current and most recent previous workplace followed. (My employers before that were pre-LLMs.)
If you are the one with your name on the PR, it's your code and you have to understand it. If you don't understand it at least well enough to speak intelligently about it, you aren't ready to submit it for review. If you ask Copilot, Cursor, or whatever to generate a PR for you, it still must be reviewed and approved by you and another engineer who acts as your reviewer.
I haven't heard a lot of pushback on this; it feels like common sense to me. It's effectively the same rules we'd use if somebody who wasn't an engineer wanted to submit code; they'd need to go through an engineer to do it.
LLM usage has increased our throughput and the quality of our code thus far, but without these rules (and people following the spirit of them, being bought in to their importance), I really don't think it would.
I encourage you to raise this policy with your management, if you think you can get them to listen, and demonstrate how it might help. I would be very frustrated if my colleagues were submitting AI-generated code without thinking it through.
Open source projects like LLVM need to do this as it is one of those projects that is widely used in the software supply chain, on the level that needs protection from contributors who do not understand the code they are writing or cannot defend their changes.
There needs to be a label which designates such open source projects that is so important and adopted throughout the industry that not anyone can throw patches to it without understanding what it does, and why they need it.
This is why gcc and linux use mailing lists instead of github pull requests.
In the gcc/linux flow, convincing the rest of the mailing list to not ignore their message is the submitter's job.
In the github flow, keeping the "unresolved PR count" (displayed right beneath your project's title) low is the developers' job.
I am so exhausted by reviewing the AI slop from other “developers”. For a while I was trying to be a good sport and point out where it was just wrong or doing things that were unnecessary or inefficient. I’m at the point of telling people to not bother using an AI. I don’t have the time or energy to deal with it. It’s like a missile defense system that costs a million dollars to intercept but the incoming projectile cost $10 for your adversary. It’s not sustainable.
"Vibe coding" (i.e. the kind of code that is statistically 'plausible' that sometimes works and the user doesn't look at the code but tries it to see if it works to their liking (with no tests) )
Was the worst thing to happen to programming, computer science I have seen, good for prototypes but not production software, and especially for important projects like LLVM.
It is good to gatekeep this slop from LLVM before it gets out of control.
I think it's great that this AI slop fatigue happened so quickly.
Now we can still identify them easily, and I am maintaining bookmarks of non slop codebases, so that I know which software to avoid.
I encourage everyone to do the same because the slopcode fallout will be inevitable, and likely be the most harmful thing that ever happened to open source as a philosophy.
We need to change our methodology of how to develop verifiable and specdriven software, because TDD isn't good enough to catch this. Something that is able to verify logical conclusions and implications (aka branches) and not only methods and their types/signatures.
Not just codebases, but developers too. Keep your eyes open: if someone visibly embraces slop, you know they're a clown. Don't let clowns touch your repos, and do your best to cut out dependencies on anything maintained by clowns.
I don't have enough snark in me (yet) to make a "proud to be AI" list to make such lookups easier.
Personally, I’ve got fatigue at the phrase “AI slop”. It’s used as a catch all to dismiss the content due to the source, regardless of the quality or suitability when taken in context.
Just like everything else these days the responses skew towards both extremes on the spectrum and people hand waving away the advancements is just as annoying as those who are zealots on the other end.
> automated review tools that publish comments without human review are not allowed
This seems like a curious choice. At my company we have both Gemini and cursor (I’m not sure which model under the hood on that) review agents available. Both frequently raise legitimate points. Im sure they’re abusable, I just haven’t seen it
one reason i can imagine for this choice is that human review distributes new knowledge among human maintainers. Automated review might discourage that valuable interaction.
The comment is an artefact of an inherently valuable process, not the sole objective. So I'd prefer code is reviewed by a human who maybe initiates a discussion with an agent. I'd hope this minor workflow detail encourages the human to take a more active lead, rather than risk mere ceremonial approval.
The policy isn’t about whether those tools raise good points. It’s about not letting agents act autonomously in project spaces. Human reviewed, opt in use is explicitly allowed.
An LLM is a plausibility engine. That can't be the final step of any workflow.