Ironically this seems like a perfect use for AI from the maintaner side. "Find PR's without associated issues from new contributors. Discuss with the contributors in the PR about how contributions should be made, then reject their PR's. If any bad behavior is detected in this interaction, block the accounts for 24 hours and add them to a list of accounts to review for permanent banning from the project".
Automated PR's can have automated responses. You make a human effort, you get a human response.
Seems like a contributor reputation score (shared across all projects) that one can build up over time would be a clean solution to the low quality pull requests.
I also maintain a popular open source repo and have seen similar. If I receive a low effort single-pass obviously-Clauded PR (many people (agents) don’t even bother using the AGENTS.md file I provide right in the repo?!) with no associated issue I also have no qualms closing. I do generally leave a short note about why I closed (volume of these is around 5/week) showing how far off they (Claude) was and encourage opening an issue where we can start a discussion which rarely happens for the drive-by folks. So I totally feel this but I find myself more disheartened by the practice than angry. It feels like slowly watching open-source go the way of email. Open and free until no-cost spam ruined the inbox for everyone...
> If I receive a low effort single-pass obviously-Clauded PR (many people (agents) don’t even bother using the AGENTS.md file I provide right in the repo?!)
Claude doesn't read AGENTS.md automatically, it only looks for CLAUDE.md.
Hi Neil, fun to see you on HN. I agree with your points and I you summarized it very well as "Ultimately, open source is built on trust".
AI is destroying trust in open source and many other areas and I think this will discourage teams from publishing their source code in the future.
On the other hand, personal connections are becoming even more important, which is unfair to the younger generation and people who don't live near tech hubs.
Today's conception of open source is different from the original anyway. It used to be "we made this thing, here's the code" but now it's turned into some kind of weird parasocial thing more akin to Facebook.
Do recruiters actually care about your open source contributions? Especially now when only LLMs read CVs and match them against strict criteria, I doubt there are many companies that actually care about your work outside of work, in fact they might care in opposite direction - thinking that you'll be distracted from work
Just speculating (I use zero automation when recruiting), but automated candidate ranking systems could well still look for signals that were valuable years ago, but no longer are. It's not a given that people using this stuff even understand the criteria under the hood.
Same story as SEO, I presume. Google surely understood their ranking criteria deeply, yet couldn't exactly win that fight.
So the fixes are still fixes, but we (I am also a OSS maintainer) are unwilling to accept them as they boost the contributor’s status where we think the merit is very or extremely limited.
Why not have these PRs counted differently (by the platform), and/or colored differently in the timeline(s) thus made less visible or more clear?
Unless the change is an obvious improvement, it has to be worth the time for the maintainers to spend attention reviewing it (and supporting the code forever, and all the rest).
Even if these particular changes are "harmless" and easy to review, accepting them sets a precedent that encourages an unsustainable floodgate of AI-generated changes that will overwhelm the project.
These fixes, regularly destroy the architecture, accrue bloat for little gain, refuse to rewrite while demanding to rewrite- and many other such funny noises. Most code contribution by LLMs is garbage if you long-term care about the project. Look at closed source projects that ingest all this madness - windows with its seconds to open the explorer and other catastrophes.
I think what you're saying is fair, tbh. Imagine you put in months or years into a project to make it a quality piece of work. It develops into something notable and you took all the risk. Then someone comes along to fix a spelling error with a pull request so their name effectively appears on the repo as a "contributor." And you just know right after its going on their resume as "contributed to [...]" or maybe if they're bold "software engineer working on [...]" which implies substantial investment. Then you're effectively sharing credit for YOUR work with someone who did nothing. That is rage inducing. ((Of course: it probably is just juniors trying their best in this horrible industry.))
On the other hand: lets be careful not to dismiss valid but inexperienced attempts to contribute. Having someone want to genuinely contribute to your software is incredibly generous. If someone seems like they're trying its better to give advice than act like a snob because its not good enough. Often pull requests only need small fixes to get in, anyway.
Because every merge request and every merge comes with cost and (potential) more cost in the future.
Linux lately received a lot of fixes for drivers nobody has cared about for ages. They decided to remove them. Maybe they should have done this earlier, I don't know, but that happened because every code change comes with cost down the line. For real contributers, this cost is much more limited and they have a higher probability of burden the cost down the line.
Pure AI MRs are just rude, because it's just "here take this, don't really care what this is, but AI said it being good, I won't be around, so you will have to deal with this code after the merge and you better don't overlook anything, because I personally don't really know what I am doing, I only know how to have AI making something that looks convincing. Good luck fixing the bugs this change introduces".
My solution is to ask them for proof of work. Many of these disappear after opening a PR that they never tested. How can you claim to have fixed a visual issue without a screenshot? Insta-close.
Both are fine, but I also prefer 'furnish', as in put in the main parts of your CV. In a way, 'burnish' is more like 'polish', suggesting an extra layer on top, rather than the fundamental parts.
This makes sense. Apart from the original thing, I no longer upstream anything. It just takes comparatively more effort than maintaining a private fork with my own idiosyncratic fix. This must mean that even small fixes that non-contributors would make in the past are just happening off the books, so to speak. e.g. I made a tiny fix to Thrift codegen because some function could be much faster. These days I wouldn't bother. I'd just fork and leave upstream to be upstream.
I suspect many people are like me since it feels like a very normie position to take. That means that contributors are even more likely to be useful because both the drive-by genuine contributors have chosen otherwise and these contributors have increased.
It must be quite painful to be an open source maintainer right now. One wonders what to make of such projects in a future where a feature-list is a sufficient prompt.
I ran through exactly this thought process & LLM based resolution just 8 hours ago with a segfault compiling a project which likely doesn't care about the given platform combo much.
Vibe slop or put on your C.V "contributor to <wellknownproject>"? In terms of social currency B wins easily, unless you can guerilla market as well as Openclaw.
that won't work because all those millions of 'projects' are unmaintainable vaporware which is exactly why people try to piggyback on the reputation of software that is actually being used and developed by people
I very much can relate, as I've also been receiving many of these drive-by PRs lately.
One big problem I have with them is that they take away time from project maintainers for reviewing and helping to get the PRs into shape, which then can't be spent on other, more important things. I feel like the "good first issue" GitHub label is specifically attracting these kinds of contributions.
It's not a black-or-white thing though, and you need to tell apart folks who produce slop PRs against any arbitrary repo, from folks using AI to contribute in a sensible way. We've tried to codify some rules in our contribution guide [1]:
- PRs from apparent bot accounts are closed
- PRs from users who file large numbers against random repos are closed
- You're welcome to use AI, but you need to stand behind your PR and be able to explain it
I'm sure we'll adjust those rules over time, but since we have instantiated them, it definitely has become easier to deal with AI PRs and handle them in an a relatively objective way. It absolutely means that sometimes a PR will be closed which could have been an improvement, but I think this is the right thing to do given the circumstances.
GitHub flavored FOSS I believe works best for "corporate FOSS" projects and terrible for "FOSS as how it was conceived decades ago".
Or rather I believe that it is engineered exactly for the former.
What it does well is coordination between corps, working in public (while on a corp payroll) and extracting some drive-by-PR value out of random third-parties.
The closer your project is to that shape, the better it works for you. The further it is away, the more pain you will feel.
This has nothing to do with AI slop.
AI slop just turned up a few knobs that were already there.
I help maintain a big project as well and I can totally relates what the author is describing.
The amount of security advisories and PRs opened is getting out of hands.
This is no longer just spell check PRs that you can merge in less than 1 minute of review. Often that's going to be supposed perf improvement, attempts at fixing things that are not broken, or submitting completely new features.
These relatively low effort PRs takes a lot of effort to review or even just to filter out.
> The changes were harmless and correct, but that did not make me feel better about accepting or merging them.
So do you have the project’s best interest at heart or not? If you’re more concerned about the intent of a valid contribution than the content, why don’t you ask yourself where your intent is? You rejected a valid contribution based on unverified vibes about the person’s intent, instead of assuming they were just being helpful.
Why not assume someone does have the best interest of their own project at heart? How do you get to question that out of the gate, while pretending automated grammar and spelling fixes to rack up PR counts are benevolent unless proven otherwise? Simply assume the person who made the thing you didn't make, who was the only person (or persons) on the planet to come up with that exact thing, knows what's best for that person (or persons). Just like you would handle your own stuff.
I think it’s implied that accepting these kinds of PRs will take time away from working on actual improvements to the codebase. So, on the long term it’s better to be strict with these kinds of silly contributions, and focus on changes that matter more (e.g. features or fixes).
Ironically this seems like a perfect use for AI from the maintaner side. "Find PR's without associated issues from new contributors. Discuss with the contributors in the PR about how contributions should be made, then reject their PR's. If any bad behavior is detected in this interaction, block the accounts for 24 hours and add them to a list of accounts to review for permanent banning from the project".
Automated PR's can have automated responses. You make a human effort, you get a human response.
Seems like a contributor reputation score (shared across all projects) that one can build up over time would be a clean solution to the low quality pull requests.
I also maintain a popular open source repo and have seen similar. If I receive a low effort single-pass obviously-Clauded PR (many people (agents) don’t even bother using the AGENTS.md file I provide right in the repo?!) with no associated issue I also have no qualms closing. I do generally leave a short note about why I closed (volume of these is around 5/week) showing how far off they (Claude) was and encourage opening an issue where we can start a discussion which rarely happens for the drive-by folks. So I totally feel this but I find myself more disheartened by the practice than angry. It feels like slowly watching open-source go the way of email. Open and free until no-cost spam ruined the inbox for everyone...
> If I receive a low effort single-pass obviously-Clauded PR (many people (agents) don’t even bother using the AGENTS.md file I provide right in the repo?!)
Claude doesn't read AGENTS.md automatically, it only looks for CLAUDE.md.
Very intelligent
Interesting analogy.
Sounds like an opportunity for GitHub/others to better filter these.
Hi Neil, fun to see you on HN. I agree with your points and I you summarized it very well as "Ultimately, open source is built on trust".
AI is destroying trust in open source and many other areas and I think this will discourage teams from publishing their source code in the future.
On the other hand, personal connections are becoming even more important, which is unfair to the younger generation and people who don't live near tech hubs.
Today's conception of open source is different from the original anyway. It used to be "we made this thing, here's the code" but now it's turned into some kind of weird parasocial thing more akin to Facebook.
Fortunately they can’t do duck face with a project for their instagram feed /s
Do recruiters actually care about your open source contributions? Especially now when only LLMs read CVs and match them against strict criteria, I doubt there are many companies that actually care about your work outside of work, in fact they might care in opposite direction - thinking that you'll be distracted from work
Just speculating (I use zero automation when recruiting), but automated candidate ranking systems could well still look for signals that were valuable years ago, but no longer are. It's not a given that people using this stuff even understand the criteria under the hood.
Same story as SEO, I presume. Google surely understood their ranking criteria deeply, yet couldn't exactly win that fight.
“The spirits that I summoned, I cannot now dismiss.”
https://en.wikipedia.org/wiki/The_Sorcerer%27s_Apprentice
So the fixes are still fixes, but we (I am also a OSS maintainer) are unwilling to accept them as they boost the contributor’s status where we think the merit is very or extremely limited.
Why not have these PRs counted differently (by the platform), and/or colored differently in the timeline(s) thus made less visible or more clear?
Change is bad unless it's great.
Unless the change is an obvious improvement, it has to be worth the time for the maintainers to spend attention reviewing it (and supporting the code forever, and all the rest).
Even if these particular changes are "harmless" and easy to review, accepting them sets a precedent that encourages an unsustainable floodgate of AI-generated changes that will overwhelm the project.
These fixes, regularly destroy the architecture, accrue bloat for little gain, refuse to rewrite while demanding to rewrite- and many other such funny noises. Most code contribution by LLMs is garbage if you long-term care about the project. Look at closed source projects that ingest all this madness - windows with its seconds to open the explorer and other catastrophes.
I think what you're saying is fair, tbh. Imagine you put in months or years into a project to make it a quality piece of work. It develops into something notable and you took all the risk. Then someone comes along to fix a spelling error with a pull request so their name effectively appears on the repo as a "contributor." And you just know right after its going on their resume as "contributed to [...]" or maybe if they're bold "software engineer working on [...]" which implies substantial investment. Then you're effectively sharing credit for YOUR work with someone who did nothing. That is rage inducing. ((Of course: it probably is just juniors trying their best in this horrible industry.))
On the other hand: lets be careful not to dismiss valid but inexperienced attempts to contribute. Having someone want to genuinely contribute to your software is incredibly generous. If someone seems like they're trying its better to give advice than act like a snob because its not good enough. Often pull requests only need small fixes to get in, anyway.
Why not let them have the status boost? This isn't zero sum.
> This isn't zero sum.
True, it is actually negative sum, because fake merit erodes trust in real merit, as it makes it a lot harder to spot the latter.
Because every merge request and every merge comes with cost and (potential) more cost in the future.
Linux lately received a lot of fixes for drivers nobody has cared about for ages. They decided to remove them. Maybe they should have done this earlier, I don't know, but that happened because every code change comes with cost down the line. For real contributers, this cost is much more limited and they have a higher probability of burden the cost down the line.
Pure AI MRs are just rude, because it's just "here take this, don't really care what this is, but AI said it being good, I won't be around, so you will have to deal with this code after the merge and you better don't overlook anything, because I personally don't really know what I am doing, I only know how to have AI making something that looks convincing. Good luck fixing the bugs this change introduces".
Boost for what? That they can type a prompt?
How do you distinguish those who know what they are doing from those who only can write a prompt and copy the result?
Making the haystack bigger is a bad idea if you search the needle
The whole idea of "counting PRs" as a vanity metric is flawed because vanity metrics are flawed.
I don't think that there is a technical solution to be found here, as the problem is anything but technical.
__
A hack/trap:
Comment "Ah yes thanks a lot for the hint :)", then make the changes yourself.
Then see how the person reacts to that.
Hack the grifters. Hack the planet.
My solution is to ask them for proof of work. Many of these disappear after opening a PR that they never tested. How can you claim to have fixed a visual issue without a screenshot? Insta-close.
Or just pull the pr into your local git, git amend --reset-author, and then push it to master yourself.
Merging commits/PRs without reading them is how you get a 2024 xz situation.
I think the author meant 'burnish' there, which is a clever way of showing they didn't use AI :-)
this is the real unlock, we found the load-bearing footgun
Furnish makes more sense.
Both are fine, but I also prefer 'furnish', as in put in the main parts of your CV. In a way, 'burnish' is more like 'polish', suggesting an extra layer on top, rather than the fundamental parts.
This makes sense. Apart from the original thing, I no longer upstream anything. It just takes comparatively more effort than maintaining a private fork with my own idiosyncratic fix. This must mean that even small fixes that non-contributors would make in the past are just happening off the books, so to speak. e.g. I made a tiny fix to Thrift codegen because some function could be much faster. These days I wouldn't bother. I'd just fork and leave upstream to be upstream.
I suspect many people are like me since it feels like a very normie position to take. That means that contributors are even more likely to be useful because both the drive-by genuine contributors have chosen otherwise and these contributors have increased.
It must be quite painful to be an open source maintainer right now. One wonders what to make of such projects in a future where a feature-list is a sufficient prompt.
I ran through exactly this thought process & LLM based resolution just 8 hours ago with a segfault compiling a project which likely doesn't care about the given platform combo much.
How could it take less effort to maintain a fork? You have to merge the upstream in every week and fight endlessly on conflicts with your patches.
I’ve heard about similar issues with Hacktoberfest’s T-shirts back in 2020, but it was not as automated back then: https://news.ycombinator.com/item?id=24643894
These maintainer who’ve held the sacred fire for years…
These idea guys who longed for the fire for years- and still don't have it.. full of envy..
This is weird though.
Obviously the best way to use AI to furnish your CV is to build your own project using AI.
Vibe slop or put on your C.V "contributor to <wellknownproject>"? In terms of social currency B wins easily, unless you can guerilla market as well as Openclaw.
that won't work because all those millions of 'projects' are unmaintainable vaporware which is exactly why people try to piggyback on the reputation of software that is actually being used and developed by people
I very much can relate, as I've also been receiving many of these drive-by PRs lately.
One big problem I have with them is that they take away time from project maintainers for reviewing and helping to get the PRs into shape, which then can't be spent on other, more important things. I feel like the "good first issue" GitHub label is specifically attracting these kinds of contributions.
It's not a black-or-white thing though, and you need to tell apart folks who produce slop PRs against any arbitrary repo, from folks using AI to contribute in a sensible way. We've tried to codify some rules in our contribution guide [1]:
- PRs from apparent bot accounts are closed - PRs from users who file large numbers against random repos are closed - You're welcome to use AI, but you need to stand behind your PR and be able to explain it
I'm sure we'll adjust those rules over time, but since we have instantiated them, it definitely has become easier to deal with AI PRs and handle them in an a relatively objective way. It absolutely means that sometimes a PR will be closed which could have been an improvement, but I think this is the right thing to do given the circumstances.
[1] https://github.com/hardwood-hq/hardwood/blob/main/CONTRIBUTI...
GitHub flavored FOSS I believe works best for "corporate FOSS" projects and terrible for "FOSS as how it was conceived decades ago". Or rather I believe that it is engineered exactly for the former.
What it does well is coordination between corps, working in public (while on a corp payroll) and extracting some drive-by-PR value out of random third-parties.
The closer your project is to that shape, the better it works for you. The further it is away, the more pain you will feel.
This has nothing to do with AI slop. AI slop just turned up a few knobs that were already there.
This is nothing new, people have been making spellcheck commits all over github for years... and that was before ai slop even existed.
I help maintain a big project as well and I can totally relates what the author is describing.
The amount of security advisories and PRs opened is getting out of hands. This is no longer just spell check PRs that you can merge in less than 1 minute of review. Often that's going to be supposed perf improvement, attempts at fixing things that are not broken, or submitting completely new features.
These relatively low effort PRs takes a lot of effort to review or even just to filter out.
Create AI to review the pull requests and auto ban people for low SNR.
> The changes were harmless and correct, but that did not make me feel better about accepting or merging them.
So do you have the project’s best interest at heart or not? If you’re more concerned about the intent of a valid contribution than the content, why don’t you ask yourself where your intent is? You rejected a valid contribution based on unverified vibes about the person’s intent, instead of assuming they were just being helpful.
Why not assume someone does have the best interest of their own project at heart? How do you get to question that out of the gate, while pretending automated grammar and spelling fixes to rack up PR counts are benevolent unless proven otherwise? Simply assume the person who made the thing you didn't make, who was the only person (or persons) on the planet to come up with that exact thing, knows what's best for that person (or persons). Just like you would handle your own stuff.
I think it’s implied that accepting these kinds of PRs will take time away from working on actual improvements to the codebase. So, on the long term it’s better to be strict with these kinds of silly contributions, and focus on changes that matter more (e.g. features or fixes).
The changes were grammar and spelling fixes in comments. It is not sufficient for someone to intend to be helpful.
If people can spam low-effort PRs they will. Unverified vibes is you closing your eyes to a very real issue.