Valid position from the Void side AND valid crashout. I'm impressed. A distro absolutely can use policy to enforce a vision to be AI-free on the contributions, and a maintainer can quit if they don't agree with the distro vision
A lot of it is very explicit definitions so there’s no ambiguity, and reasoning to explain why. None of the reasoning is quality of the code, which is what most people assume is the reason.
The SciActive Human Contribution Policy is a free, general contribution policy meant to safeguard projects from the inherent legal and security risks of AI generated contributions.
Boy it's amazing how people ignore the security track record of human authored code over the last 30 years. One need only look at the recent explosion of discovered vulnerabilities by AI.
AI tends to “discover” previously discovered vulnerabilities, or obvious vulnerabilities. It also tends to hallucinate vulnerabilities. I’m not saying it’s useless at discovering vulnerabilities, just that a human audit is better.
The policy also doesn’t say you can’t use AI to audit a code base, or even a single diff, so that point is moot.
AI tends to “discover” previously discovered vulnerabilities,
Friend - your information is very out-of-date. It's accelerating the discovery of vulnerabilities like crazy. I'm not saying we've hit singularity (or even that we will) but to brush it all off as "it only finds things we already knew about" is copium at this point.
I think you’re also missing the bigger point that the policy does not prevent you from auditing your code base with AI. That is completely, 100% allowed under this policy.
As someone who operates an internal AI code review platform for a large engineering org. Trust me, you want this.
There are countless defects that regularly slip by human review. It's one of the few places that AI adds immediate, low-risk value.
That aside, though, this is unfortunate. And the author and maintainer of these packages is entirely within their rights to do this as a form of protest.
Sure, maybe it's fantastic at reviewing PRs. I could imagine that. But
Trust me, you want this.
is pretty condescending, implying those against it haven't thought it through. I personally could list about twenty reasons for not using AI, many of them infinitely more important than code being well reviewed.
Having used these at my day job I can understand the policy against them. Ours is fairly well tuned by the people who operate it, adapted to our codebase, and its comments are usually on point, but its false positive rate for my own PRs is still in the low double digit percent range (feedback that's factually wrong, not relevant to the PR, not scoped appropriately, etc). Before all that tuning, back when we first started using it, I probably went days or weeks without seeing a well-founded suggestion from the tool. Incorrect comments still create work for the PR author (investigating, double checking, etc), they create noise for human reviewers, and ambiguity about whether a change is ready to be merged. I read that policy as being against random people pointing their own review tools at PRs for this project, which I assume would be more on the high false positive/high noise end of the spectrum (i.e., not being tuned for this specific project). I can totally see why already busy maintainers would want to spare themselves from a bunch of poorly calibrated noise.
One nice thing about them compared to human reviewers is that you can just tell them to (professionally) fuck off if they're wrong without hurting their feelings or having an HR conversation.
I hate AI and wish it would disappear but you're right. The automatic PR reviewing LLM at my job genuinely catches a lot of issues that would fly under the radar normally. Obviously a human review is still needed but those downvoting clearly do not work in tech because it's basically just a fancy SonarQube* CircleCI (well, apparently they're AI now but before they did that). If it's wrong, resolve the issue with one click; if it's right, then its beneficial?
The blanket AI hate is stupid because it drowns out the valid criticisms like energy use, knowledge loss, unmaintainability, etc
edit: I said CircleCI but was thinking of SonarQube
What if the goals of reviewing a PR in Void Linux are different from the goals of reviewing a PR in your org? Are there any possible goals you can think of, where introducing an AI tool would actually be detrimental?
Do you work in tech? How often do you submit PRs? AI reviews are one of the best applications for the technology and non-LLM CI tools have existed for ages to serve a similar but less efficient purpose
I don't like what AI has done to the software industry but we need to give credit where its due
One canwork in tech and not use LLMs. Your comment sounds like it should be the norm, or already is.
Also, the discussion is about using LLMs in open source projects, which put a lot more focus on understandable and maintainable code than the software industry typically happens to do.
I work in tech and I don't use LLMs. From my experience, they're catastrophic especially in those positions where they review code/actions. Reviews are the last line of defense against bad code, bugs, vulnerabilities or destructive actions. Giving that role to an RNG parrot that is often confidently wrong is almost suicidal.
From my experience, they’re catastrophic especially in those positions where they review code/actions.
That's interesting. I am currently searching for ways that I can use them a bit to chill-pill our CTO without sinking the company (which is very deep in the water with technical debt).... at least until I get another job.
Can you expand why they are that bad in this use?
My current idea is to pin up new architecture by hand, define new interfaces by hand, and then let the LLM occasionally fill in a function body.
And maybe, maybe, an "ai refactored" branch which has just the purpose to showcase what happens if one is not careful - and will never be merged, at least not by me.
if i have to trust you then doesn't that kinda show why the technology isn't particularly useful? Surely it should be able to speak for itself, rather than need you to vehemently insist that "actually it's really good you guys just don't get it".
28 Comments
swicano@programming.dev · 63 pts · 3d
Valid position from the Void side AND valid crashout. I'm impressed. A distro absolutely can use policy to enforce a vision to be AI-free on the contributions, and a maintainer can quit if they don't agree with the distro vision
talkingpumpkin@lemmy.world · 29 pts · 3d
I agree.
Leaving so abruptly may not be the classiest move but it's not like one can't leave if he so decides.
The dev will be fine and so will void. Nothing to see here for now.
hperrin@lemmy.ca · 25 pts · 3d
It sounds like their policy is basically the same as Human Contribution Policy 2-TR. That’s awesome.
anzo@programming.dev · 5 pts · 2d
I just skimmed over the text. That's just a lot of words for saying "no ai".
Personally, I like debian's approach more. They put some granularity of levels as to when and how much ai assistance can you take.
But I can totally see that void, being the profile of users and devs they got, would prefer to be way more radical.
hperrin@lemmy.ca · 4 pts · 2d
A lot of it is very explicit definitions so there’s no ambiguity, and reasoning to explain why. None of the reasoning is quality of the code, which is what most people assume is the reason.
atzanteol@sh.itjust.works · -2 pts · 1d
Boy it's amazing how people ignore the security track record of human authored code over the last 30 years. One need only look at the recent explosion of discovered vulnerabilities by AI.
This license is fud.
hperrin@lemmy.ca · 2 pts · 1d
Nobody is ignoring anything.
It’s not a license, it’s a policy.
AI tends to “discover” previously discovered vulnerabilities, or obvious vulnerabilities. It also tends to hallucinate vulnerabilities. I’m not saying it’s useless at discovering vulnerabilities, just that a human audit is better.
The policy also doesn’t say you can’t use AI to audit a code base, or even a single diff, so that point is moot.
atzanteol@sh.itjust.works · 1 pts · 15h
Friend - your information is very out-of-date. It's accelerating the discovery of vulnerabilities like crazy. I'm not saying we've hit singularity (or even that we will) but to brush it all off as "it only finds things we already knew about" is copium at this point.
hperrin@lemmy.ca · 0 pts · 15h
Well it’s a good thing I didn’t say that then. xD
I think you’re also missing the bigger point that the policy does not prevent you from auditing your code base with AI. That is completely, 100% allowed under this policy.
Reading comprehension is important.
architect@thelemmy.club · 14 pts · 3d
You know what has never benefited any of us here? Private jets and yachts.
TootSweet@lemmy.world · 4 pts · 3d
Good riddance.
douglasg14b@lemmy.world · -13 pts · 3d
As someone who operates an internal AI code review platform for a large engineering org. Trust me, you want this.
There are countless defects that regularly slip by human review. It's one of the few places that AI adds immediate, low-risk value.
That aside, though, this is unfortunate. And the author and maintainer of these packages is entirely within their rights to do this as a form of protest.
Absolutionist policies aren't helpful.
staircase@programming.dev · 22 pts · 3d
Sure, maybe it's fantastic at reviewing PRs. I could imagine that. But
is pretty condescending, implying those against it haven't thought it through. I personally could list about twenty reasons for not using AI, many of them infinitely more important than code being well reviewed.
annoying_cyclist@programming.dev · 21 pts · 3d
Having used these at my day job I can understand the policy against them. Ours is fairly well tuned by the people who operate it, adapted to our codebase, and its comments are usually on point, but its false positive rate for my own PRs is still in the low double digit percent range (feedback that's factually wrong, not relevant to the PR, not scoped appropriately, etc). Before all that tuning, back when we first started using it, I probably went days or weeks without seeing a well-founded suggestion from the tool. Incorrect comments still create work for the PR author (investigating, double checking, etc), they create noise for human reviewers, and ambiguity about whether a change is ready to be merged. I read that policy as being against random people pointing their own review tools at PRs for this project, which I assume would be more on the high false positive/high noise end of the spectrum (i.e., not being tuned for this specific project). I can totally see why already busy maintainers would want to spare themselves from a bunch of poorly calibrated noise.
One nice thing about them compared to human reviewers is that you can just tell them to (professionally) fuck off if they're wrong without hurting their feelings or having an HR conversation.
talkingpumpkin@lemmy.world · 15 pts · 3d
I'm not saying you are wrong (not saying you are right either), but... your comment sounds a lot like "trust me, I have a conflict of interest" :)
NotSteve_@lemmy.ca · 13 pts · 3d
I hate AI and wish it would disappear but you're right. The automatic PR reviewing LLM at my job genuinely catches a lot of issues that would fly under the radar normally. Obviously a human review is still needed but those downvoting clearly do not work in tech because it's basically just a fancy SonarQube*
CircleCI (well, apparently they're AI now but before they did that). If it's wrong, resolve the issue with one click; if it's right, then its beneficial?The blanket AI hate is stupid because it drowns out the valid criticisms like energy use, knowledge loss, unmaintainability, etc
edit: I said CircleCI but was thinking of SonarQube
Feathercrown@lemmy.world · 2 pts · 1d
Props for objectively evaluating things regardless of your feelings for them
HaraldvonBlauzahn@feddit.org · 11 pts · 3d
Here the pull request and discussion:
https://github.com/void-linux/void-packages/pull/62351#issuecomment-5641804457
Changes not tested.
kibiz0r@midwest.social · 8 pts · 3d
What if the goals of reviewing a PR in Void Linux are different from the goals of reviewing a PR in your org? Are there any possible goals you can think of, where introducing an AI tool would actually be detrimental?
chloroken@lemmy.ml · 8 pts · 3d
Trust me, nobody trusts you.
douglasg14b@lemmy.world · 4 pts · 1d
That's a great way to alienate people who are trying to start discussions. Good job.
Do you have the expertise and experience to state what is or isn't important for PR change review? The data to back up defect rates and outages?
If not, then you're just speaking out of your ass and with your emotions instead of with logic.
NotSteve_@lemmy.ca · 0 pts · 3d
Do you work in tech? How often do you submit PRs? AI reviews are one of the best applications for the technology and non-LLM CI tools have existed for ages to serve a similar but less efficient purpose
I don't like what AI has done to the software industry but we need to give credit where its due
HaraldvonBlauzahn@feddit.org · 14 pts · 3d
One canwork in tech and not use LLMs. Your comment sounds like it should be the norm, or already is.
Also, the discussion is about using LLMs in open source projects, which put a lot more focus on understandable and maintainable code than the software industry typically happens to do.
black0ut@pawb.social · 5 pts · 2d
I work in tech and I don't use LLMs. From my experience, they're catastrophic especially in those positions where they review code/actions. Reviews are the last line of defense against bad code, bugs, vulnerabilities or destructive actions. Giving that role to an RNG parrot that is often confidently wrong is almost suicidal.
HaraldvonBlauzahn@feddit.org · 2 pts · 2d
That's interesting. I am currently searching for ways that I can use them a bit to chill-pill our CTO without sinking the company (which is very deep in the water with technical debt).... at least until I get another job.
Can you expand why they are that bad in this use?
My current idea is to pin up new architecture by hand, define new interfaces by hand, and then let the LLM occasionally fill in a function body. And maybe, maybe, an "ai refactored" branch which has just the purpose to showcase what happens if one is not careful - and will never be merged, at least not by me.
staircase@programming.dev · 1 pts · 3d
a_non_monotonic_function@lemmy.world · -5 pts · 3d
Swedneck@discuss.tchncs.de · 5 pts · 2d
if i have to trust you then doesn't that kinda show why the technology isn't particularly useful? Surely it should be able to speak for itself, rather than need you to vehemently insist that "actually it's really good you guys just don't get it".
douglasg14b@lemmy.world · 0 pts · 1d
I'm referring to PR review specifically. Do you not know what PR reviews are?
CrypticCoffee@lemmy.ml · 5 pts · 3d
Sorry, but what does AI achieve that static code analysis tools don't? Apart from maybe ridiculous environmental impact?