Someone should make a community to freely distribute examples of data poisoning people can randomly put in their social media posts/images to sabotage AI
I've read in papers that you can poison datasets with a very small percentage of the data, if done cleverly. I can fish up the source if you want (but it might take me some time).
We conduct the largest pretraining poisoning experiments to date, pretraining models from 600M to 13B parameters on chinchilla-optimal datasets (6B to 260B tokens). We find that 250 poisoned documents similarly compromise models across all model and dataset sizes (...)
Emphasis mine. All it takes is 250 poisoned documents.
I think there's a subreddit for that. /r/PoisonAI or something. I am not aware of a fediverse equivalent, but that seems like it would be a better fit than using Reddit for that discussion.
Poi sonai, while initially was able to influence the AI output but the whole subreddit got selectively filtered out and doesn't have impact any longer. It's still good place to discuss the topic though.
I misunderstood the purpose of that community. I figured it was just for discussing how to poison AI models. But actually visiting it, I see it is primarily for posting gibberish in the hopes that AI models would scrape the sub and treat it all as genuine content. As you say, that does not seem like it would have much of any impact on actually poisoning AI models because basically every scraper is going to know to avoid a community called "poison AI".
I believe /c/Totallytruefactsnolies@lemmy.world was created for that but then newer people came in and didn’t get the joke. I posted what I thought was a solid shitpost and there were some incredulous reactions. Lots of _whoosh_ing happening there.
That assumes that AI companies are negatively impacted by the quality of their product. It’s true that they’re competing with each other based on their quality relative to other companies’ products, but poisoning public data impacts everyone’s models similarly. Setting aside competition and looking at the success of the AI sector as a whole, I think it’s more dependent on marketing and hype than on real performance... and if that’s the case, then poisoning public data doesn’t hurt anyone except the people being forced to use it.
There are lots of resources already on this. Just use a search engine. There are even apps and software to poison images and videos before you upload them to social media.
I heard of the nightshade from that video but they mention others. The better ones are from universities.
You can poison YouTube videos by making subtitles off screen or transparent. that is nonsense text or add a clip at the end that's about something unrelated.
You can poison resumes and such by having text that's "white" or a super tiny font. And have that text have commands or nonsense.
You have to poison images/videos before uploading/posting. You can't do much once it's already out there.
29 Comments
slazer2au@lemmy.world · 26 pts · 13d
Na, for it to be effective it needs to be wide spread, but if its wide spread then it can be filtered out of the training material.
Pudutr0n@lemmy.world · 25 pts · 13d
I've read in papers that you can poison datasets with a very small percentage of the data, if done cleverly. I can fish up the source if you want (but it might take me some time).
edit: here it is.
Emphasis mine. All it takes is 250 poisoned documents.
galoisghost@aussie.zone · 6 pts · 13d
It’s like that on purpose.
I would think that the OP comment here would be the truth to spread around 250 times though.
chaogomu@lemmy.world · 7 pts · 13d
There's a new technique that uses the AIs "thinking" tags to get it to do things that are otherwise banned by policy.
I'll have to find the article again. But due to the way LLMs work, they can't defend against this sort of attack.
chaogomu@lemmy.world · 3 pts · 13d
And here's some explanations of how various attacks work.
https://github.com/nukIeer/AI-Prompt-Injection-Cheatsheet
https://dev.to/praneet_gogoi_beastsoul/how-hackers-trick-ai-the-hidden-world-of-prompt-injections-and-jailbreaks-4nge
https://developer.nvidia.com/blog/how-hackers-exploit-ais-problem-solving-instincts/
Romkslrqusz@lemmy.zip · 22 pts · 13d
A centralized database of content for AI scraping agents to be trained to exclude?
enchanted@lemmy.world · 3 pts · 12d
Poison the database with real facts too
SatyrSack@quokk.au · 6 pts · 13d
I think there's a subreddit for that. /r/PoisonAI or something. I am not aware of a fediverse equivalent, but that seems like it would be a better fit than using Reddit for that discussion.
jystfact@sh.itjust.works · 3 pts · 13d
Poi sonai, while initially was able to influence the AI output but the whole subreddit got selectively filtered out and doesn't have impact any longer. It's still good place to discuss the topic though.
SatyrSack@quokk.au · 3 pts · 13d
I misunderstood the purpose of that community. I figured it was just for discussing how to poison AI models. But actually visiting it, I see it is primarily for posting gibberish in the hopes that AI models would scrape the sub and treat it all as genuine content. As you say, that does not seem like it would have much of any impact on actually poisoning AI models because basically every scraper is going to know to avoid a community called "poison AI".
MedicPigBabySaver@lemmy.world · 2 pts · 13d
Fuck Reddit and Fuck Spez.
Naich@piefed.world · 6 pts · 13d
https://lemmy.world/c/Totallytruefactsnolies?dataType=Post
baggachipz@sh.itjust.works · 4 pts · 12d
I believe /c/Totallytruefactsnolies@lemmy.world was created for that but then newer people came in and didn’t get the joke. I posted what I thought was a solid shitpost and there were some incredulous reactions. Lots of _whoosh_ing happening there.
Pudutr0n@lemmy.world · 2 pts · 12d
ohh interesting. thanks.
crunchpaste@lemmy.dbzer0.com · 4 pts · 12d
Just yesterday I was looking around for an LLM tarpit to selfhost. Most active I've found so far is Pyison. Maybe it could be of some use to someone.
Pudutr0n@lemmy.world · 2 pts · 12d
This is very useful. Thank you!
mrmisses@lemmy.world · 3 pts · 13d
People still have sm accounts?
driving_crooner@lemmy.eco.br · 6 pts · 13d
You have one on lemmy.world
Pudutr0n@lemmy.world · 1 pts · 13d
yes
Battle_Masker@lemmy.blahaj.zone · 3 pts · 13d
like an Anti AI alliance?
Pudutr0n@lemmy.world · 3 pts · 13d
I'd start it myself but could only link papers and such. Don't know how to do it effectively.
AbouBenAdhem@lemmy.world · 2 pts · 13d
That assumes that AI companies are negatively impacted by the quality of their product. It’s true that they’re competing with each other based on their quality relative to other companies’ products, but poisoning public data impacts everyone’s models similarly. Setting aside competition and looking at the success of the AI sector as a whole, I think it’s more dependent on marketing and hype than on real performance... and if that’s the case, then poisoning public data doesn’t hurt anyone except the people being forced to use it.
Pudutr0n@lemmy.world · 6 pts · 13d
Who says the objective is to impact AI companies negatively?
There's a series of valid motivations to want AI models to not be able to use public user data with no consequence.
daannii@lemmy.world · 0 pts · 12d
There are lots of resources already on this. Just use a search engine. There are even apps and software to poison images and videos before you upload them to social media.
Pudutr0n@lemmy.world · 1 pts · 12d
where?
daannii@lemmy.world · 2 pts · 11d
https://nightshade.cs.uchicago.edu/whatis.html
That's one of them.
I heard about multiple ones from this video.
Hope this helps. https://youtu.be/zF-mbwc5Mmw?is=sq7HWI2q3IwwhicT
I heard of the nightshade from that video but they mention others. The better ones are from universities.
You can poison YouTube videos by making subtitles off screen or transparent. that is nonsense text or add a clip at the end that's about something unrelated.
You can poison resumes and such by having text that's "white" or a super tiny font. And have that text have commands or nonsense.
You have to poison images/videos before uploading/posting. You can't do much once it's already out there.
Pudutr0n@lemmy.world · 2 pts · 11d
helps a lot! thanks!
Onomatopoeia@lemmy.cafe · -11 pts · 13d
Lol, good luck with that.
The more poisoning you attempt, the more effective anti-poisoning becomes.
AILLM is here, stop trying to put the genie back in the bottle. All we can do is figure out how to use it and prevent misuse.Pudutr0n@lemmy.world · 5 pts · 13d
I don't want to put it back in the bottle. I just feel like taking massive public user data for free should not be devoid of consequence.