A simple tool to figure out just how much LLM generated code has been used in a project. Obviously not perfect, but it automates the task of going through commits for "co-authored by" statements and other signs. In the off chance you don't want to use the CLI, there's also a web app!
Slopscore: An automated tool to find AI use in FOSS projects!
https://codeberg.org/polyphony/slopscore
16 Comments
Kangae_Hishiryo@scribe.disroot.org · 6 pts · 10h
I already use it. It's GLORIOUS. I'm free from a lot of stress, although not totally free of it. It's better, at least.
the_wizard_of_0Z@lemmy.ca · 3 pts · 17h
neat.
eremophila@lemmy.zip · 1 pts · 23h
artwork@lemmy.world · -5 pts · 23h
Holy smokes... Freaking awesome!
Of course... Programming is normally a human art...
Programming is of human for human to automate machines...
It is as any of people to be left in the infinite history, to learn from and improve accountably, too...
Thank you very much for the marvel... art you do...
There's the following book I recall on the subject that is an essential for every accountable, astute, and respectful developer, artist, I believe.
solrize@lemmy.ml · -7 pts · 1d
Nobody thinks the output of that thing isn't itself slop?
chickenf622@sh.itjust.works · 20 pts · 1d
Looks like a very simple heuristic analysis looking for certain files/commiters. In fact it states in the README that it doesn't go through all the files so it is probably more reliable than those AI "detectors" that also rely on AI.
CameronDev@programming.dev · 13 pts · 1d
Its just reading the commit history. It doesnt read the files at all. So it only finds overt slop, anyone who does their commits manually would pass completely.
Edit: https://slopscore.ava.pet/repo/https%3A%2F%2Fgithub%2Ecom%2Fcameroncros%2Fedgy%2Dtable
This repo is 100% slop (its mine, I know what I did), and it gets marked clean.
bugbear@lemmy.sdf.org · 8 pts · 15h
I mean, realistically speaking that's what I would do manually. So it's just making my life easier by automating it.
I don't expect it to analyse individual files.
FedX@quokk.au · 5 pts · 13h
Exactly. It can produce false negatives, but by nature of Git's blaming everyone for every change, it makes it (probably) won't produce false positives. It's a give and take in that instance. Do you do a witch hunt and try and track down every LLM generated repo and get some legit repos caught up in an imperfect process. Or do you use actionable evidence to make an informed decision?
In my opinion, witch hunts are really quite unhealthy for society. When possible, I choose projects with blanket no-AI statements, but when not, this expedites the process of choosing between alternatives. Rustic has less AI generated code than Restic, let's do that. Kitty has hundreds of AI generated commits, let's delete that immediately.
Kangae_Hishiryo@scribe.disroot.org · 1 pts · 10h
Yeah, and at least the site shows you the content of the files.
I think there can be a reasonable way for it to get false positives, and it's when a repo has an
AGENTS.md/<insert-llm-here>.mdfile, or a.skills/,cursor/ortabby/folder, for example, but they're adversarial (for example, theres awful.system's AAA-NO-SLOP.md).chickenf622@sh.itjust.works · 4 pts · 23h
It also looks for certain files like AGENTS.md which your repo doesn't appear to have.
eremophila@lemmy.zip · 0 pts · 23h
*CameronSlop
CameronDev@programming.dev · 3 pts · 23h
Ha, I still hand write most of my code, i used it for that repo to try out local LLMs. Know thy enemy.
I did hand-polish that code as well, just the initial pass was slopped together.
eremophila@lemmy.zip · 1 pts · 23h
I do know my enemy ;)
FedX@quokk.au · 1 pts · 9h
They're the teachers who taught me to fight me. Compromise, conformity, Assimilation, submission, Ignorance, hypocrisy, Brutality, the elite. All of which are American dreams.
eremophila@lemmy.zip · 1 pts · 5h
https://www.youtube.com/watch?v=qKpik3zS_-o
"She cared for me and put me on this earth Oh, the pain of just a simple birth Right now, I find that she has left me dumb and blind Poisoned, twisted, and destroyed my mind She taught me things that simply were not true She taught me hate for race, that's why I hate you!"
also crypto bros, ai users, duopoly voters, amazon shoppers....
artwork@lemmy.world · 1 pts · 16h
There's a self-scan:
- https://slopscore.ava.pet/repo/https://U:T@codeberg.org/polyphony/repo-slopscore
From the first glance, it works most of the times better than nothing.