A Developer Accidentally Found CSAM in AI Data. Google Banned Him For It
https://www.404media.co/a-developer-accidentally-found-csam-in-ai-data-google-banned-him-for-it/
https://www.404media.co/a-developer-accidentally-found-csam-in-ai-data-google-banned-him-for-it/
89 Comments
Devial@discuss.online · 192 pts · 256d
forkDestroyer@infosec.pub · 23 pts · 255d
I'm being a bit extra but...
Your statement:
The article headline:
The general story in reference to the headline:
The article headline is accurate if you interpret it as
"A Developer Accidentally Found CSAM in AI Data. Google Banned Him For It" ("it" being "csam").
The article headline is inaccurate if you interpret it as
"A Developer Accidentally Found CSAM in AI Data. Google Banned Him For It" ("it" being "reporting csam").
I read it as the former, because the action of reporting isn't listed in the headline at all.
^___^
Blubber28@lemmy.world · 7 pts · 255d
This is correct. However, many websites/newspapers/magazines/etc. love to get more clicks with sensational headlines that are technically true, but can be easily interpreted as something much more sinister/exciting. This headline is a great example of it. While you interpreted it correctly, or claim to at least, there will be many people that initially interpret it the second way you described. Me among them, admittedly. And the people deciding on the headlines are very much aware of that. Therefore, the headline can absolutely be deemed misleading, for while it is absolutely a correct statement, there are less ambiguous ways to phrase it.
MangoCats@feddit.it · 3 pts · 255d
This is pretty much the art of sensational journalism, popular song lyric writing and every other "writing for the masses" job out there.
Factual / accurate journalism? More noble, but less compensated.
obsoleteacct@lemmy.zip · 1 pts · 255d
It is a terrible headline. It can be debated whether it's intentionally misleading, but if the debate is even possible then the writing is awful.
MangoCats@feddit.it · 2 pts · 255d
Awfully well compensated in terms of advertising views as compared with "good" writing.
Capitalism in the "free content market" at work.
WildPalmTree@lemmy.world · 1 pts · 254d
The inclusion of "found" indicates that it is important to the action taken by Google, would be my interpretation.
MangoCats@feddit.it · 17 pts · 255d
My experience of Google and the unban process is: it doesn't exist, never works, doesn't even escalate to a human evaluator in a 3rd world sweatshop - the algorithm simply ignores appeals inscrutably.
ulterno@programming.dev · 3 pts · 254d
Another point is, the reason Google's AI is able to identify CSAM is because it has that in its training data, flagged as such.
In that case, it would have detected the training material as ~100% match.
I don't get though, how it ended up being openly available as if it were properly tagged, they would probably exclude it from the open-sourced data. And now I see it would also not be viable to have an open-source, openly scrutinisable AI deployment for CSAM detection for the same reason.
And while some governmental body got a lot of backlash for trying to implement such an AI thing on chat stuff, Google gets to do so all it wants because it's E-Mail/GDrive and all on their servers and you can't expect privacy.
Considering how many such stories of people having problems due to this system is coming up, is there any statistic of legitimate catches using this model? I suspect not, because why would anyone use Google services for this kind of stuff?
arararagi@ani.social · 2 pts · 254d
You would think, but none of these companies actually make their own dataset, they buy from third parties.
ulterno@programming.dev · -1 pts · 254d
I am not sure which point you are answering to.
COuld you please specify.
ayyy@sh.itjust.works · 2 pts · 255d
A 404Media headline? The place exclusively staffed by former BuzzFeed/Cracked employees? Noooo, couldn’t be.
cupcakezealot@piefed.blahaj.zone · 1 pts · 256d
so they got mad because he reported it to an agency that actually fights csam instead of them so they can sweep it under the rug?
Devial@discuss.online · 25 pts · 256d
cupcakezealot@piefed.blahaj.zone · -5 pts · 255d
they obviously did if they banned him for it; and if they're training on csam and refuse to do anything about it then yeah they have a connection to it.
Devial@discuss.online · 7 pts · 255d
Devial@discuss.online · 3 pts · 255d
MangoCats@feddit.it · 3 pts · 255d
Google doesn't ban for hate or feels, they ban by algorithm. The algorithms address legal responsibilities and concerns. Are the algorithms perfect? No. Are they good? Debatable. Is it possible to replace those algorithms with "thinking human beings" that do a better job? Also debatable, from a legal standpoint they're probably much better off arguing from a position of algorithm vs human training.
Cybersteel@lemmy.world · -26 pts · 256d
We need to block access to the web to certain known actors and tie ipaddresses to IDs, names, passport number. For the children.
kylian0087@lemmy.dbzer0.com · 18 pts · 256d
Oh hell no. That's a privacy nightmare to he abused like hell.
Also that wouldn't work at all what you say.
Cybersteel@lemmy.world · -10 pts · 256d
In the current digitized world, trivial information is accumulating every second; preserved in all it's tritness, never fading, always accessible; rumors of petty issues, misinterpretations, slander.
All junk data preserved in an unfiltered state, growing at an alarming rate, it will only slow down social progress.
The digital society furthers human flaws and selectively rewards development of convenient half-truths. Just look at the strange juxtaposition of morality around us. Billions spent on new weapons to humanely murder other humans. Rights of criminals are given more respect than the privacy of their own victims. Although there are people in poverty, huge donations are made to protect endangered species; everyone grows up being told what to do.
"Be nice to other people."
"But beat out the competition."
"You're special, believe in yourself and you will succeed".
But it's obvious from the start that only a few can succeed.
You exercise your right to freedom and this is the result. All the rhetoric to avoid conflict and protect each other from hurt. The untested truths spun by different interests continue to churn and accumulate in the sandbox of political correctness and value systems.
Everyone withdrawals into their own small gated community, afraid of a larger forum; they stay inside their little ponds leaking what ever "truth" suits them into the growing cesspool of society at large.
The different cardinal truths neither clash nor mesh, no one is invalidated but no one is right. Not even natural selection can take place here.
The world is being engulfed in "Truth". And this is the way the world ends. Not with a BANG, but with a...
zalgotext@sh.itjust.works · 7 pts · 255d
Is this a fresh new copypasta, or are you just a really long-winded, elaborate troll?
tetris11@feddit.uk · 11 pts · 256d
Also, pay me exhorbitant amounts of tax-payer money to ineffectually enforce this. For the children.
jjlinux@lemmy.zip · 6 pts · 256d
Fuck you, and everything you stand for.
driving_crooner@lemmy.eco.br · 15 pts · 256d
That sounds like sarcasm to me
x0x7@lemmy.world · -2 pts · 256d
People on Lemmy don't understand sarcasm because they have brain damage.
asudox@lemmy.asudox.dev · 1 pts · 255d
including you
x0x7@lemmy.world · -1 pts · 254d
Apparently you didn't get the sarcasm.
asudox@lemmy.asudox.dev · 2 pts · 254d
I don't see the sarcasm in your comment.
NoForwardslashS@sopuli.xyz · 0 pts · 256d
No need to go that far. If we just require one valid photo ID for TikTok, the children will finally be safe.
bobzer@lemmy.zip · -33 pts · 256d
ATM machine
Goodlucksil@lemmy.dbzer0.com · 27 pts · 256d
CSAM stands for "material". Adding "image" specifies what kind of material it is.
bobzer@lemmy.zip · -23 pts · 256d
A "material image" doesn't make any sense. An image is material. It should be CSAI if you wanna be specific.
I don't know why this is the second time I've had a discussion about CSAM being a stupid acronym on Lemmy, but it's also the only place I've ever seen people use it.
Goodlucksil@lemmy.dbzer0.com · 4 pts · 255d
Material. Type of material: Image
bobzer@lemmy.zip · -3 pts · 255d
Why say sexual abuse material images, which is grammatically incorrect, instead of sexual abuse images, which is what you mean, and shorter?
Devial@discuss.online · 10 pts · 256d
bobzer@lemmy.zip · -6 pts · 255d
Material.
Devial@discuss.online · 5 pts · 255d
MangoCats@feddit.it · 1 pts · 255d
And, if you're trying to authorize law enforcement to arrest and prosecute, you want the broadest definitions possible.
bobzer@lemmy.zip · -4 pts · 255d
You're right. It can be images, that's exactly why saying "this man was found in possession of child abuse material images" does not make grammatical sense. It's why CP still defines it better as we're not arresting people for owning copies of Lolita, which you could argue is also CSAM.
The majority of people can be wrong.
Devial@discuss.online · 2 pts · 255d
bobzer@lemmy.zip · -3 pts · 255d
"I'm mad you're right so let me compare you to a hateful right wing grifter and also by the way, you're wrong because all my friends say so."
It may shock you but a handful of Lemmy users doesn't constitute the linguistic consensus you're trying to inherit here.
TheJesusaurus@sh.itjust.works · 106 pts · 256d
Why confront the glaring issues with your "revolutionary" new toy when you could just suppress information instead
Kyrgizion@lemmy.world · 22 pts · 256d
This was about sending a message: "stfu or suffer the consequences". Hence, subsequent people who encounter similar will think twice about reporting anything.
Devial@discuss.online · 30 pts · 256d
Whostosay@sh.itjust.works · 1 pts · 256d
It seems they did react to it though
Devial@discuss.online · 12 pts · 256d
Whostosay@sh.itjust.works · 3 pts · 256d
An automatic reaction is a reaction
Devial@discuss.online · 10 pts · 256d
AngryishHumanoid@lemmynsfw.com · 53 pts · 256d
"Sign up for free access."
danc4498@lemmy.world · 27 pts · 256d
https://archive.ph/d6LEb
floquant@lemmy.dbzer0.com · 9 pts · 256d
Nooo I was liking 404 :/
Sucks to see them enshittified too..edit: that was too harsh, I take it back.
StitchInTime@piefed.social · 22 pts · 256d
I think they’ve always been like this for some of their posts, and honestly I’m considering getting a paid subscription to support them. Sucks, but they’ve been putting out quality content in exchange for your email address and some metrics - I’d call it a fair trade.
tja@sh.itjust.works · 14 pts · 256d
They are doing it because of AI scraper. But that is for some time now already
Chozo@fedia.io · 2 pts · 256d
How does this stop AI scrapers?
Axolotl_cpp@feddit.it · 9 pts · 256d
It's more difficult to crawl a webpage if under a login wall so you will have less crawlers flooding your site
BarbecueCowboy@lemmy.dbzer0.com · 1 pts · 256d
It is legitimately free after you sign up, I get their reasoning but is kinda annoying.
killea@lemmy.world · 46 pts · 256d
So in a just world, google would be heavily penalized for not only allowing csam on their servers, but also for violating their own tos with a customer?
shalafi@lemmy.world · 19 pts · 256d
We really don't want that first part to be law.
Though I'm not sure if that applies to scraping other server's content. But I wouldn't say it's fair for the scraper to review everything. If we don't like that take, then we should illegalize scraping altogether, but I'm betting there are unwanted side effects to that.
mic_check_one_two@lemmy.dbzer0.com · 7 pts · 256d
While I agree with Section 230 in theory, it is often only used in practice to protect megacorps. For example, many Lemmy instances started getting spammed by CSAM after the Reddit API migration. It was very clearly some angry redditors who were trying to shut down instances, to try and keep people on Reddit.
But individual server owners were legitimately concerned that they could be held liable for the CSAM existing on their servers, even if they were not the ones who uploaded it. The concern was that Section 230 would be thrown out the window if the instance owners were just lone devs and not massive megacorps.
Especially since federation caused content to be cached whenever a user scrolled past another instance’s posts. So even if they moderated their own server’s content heavily (which wasn’t even possible with the mod tools that existed at the time), then there was still the risk that they’d end up cacheing CSAM from other instances. It led to a lot of instances moving from federation blacklists to whitelists instead. Basically, default to not federating with an instance, unless that instance owner takes the time to jump through some hoops and promises to moderate their own shit.
vimmiewimmie@slrpnk.net · 2 pts · 256d
Not to create an argument, which isn't my intent, as certainty there may be a thought such as, "scraping as it stands is good because of the simplification and 'benefit'". Which, sure, it's easiest to wide net and absorb, to simply the concept, at least as I'm also understanding it.
Yet, maybe it is the process of scraping, and also absorbing into databases including AI, which is a worthwhile point of conversation. Maybe how we've been doing something isn't the continued 'best course' for a situation.
Undeniably, more minutely monitoring what is scraped and stored creates large quantities, and large in scope, of questions and obstacles, but, maybe having that conversation is where things should go.
Thoughts?
killea@lemmy.world · 1 pts · 256d
Oh my, yes, you are correct. That was sort of knee jerk, as opposed to it being the reporting party's burden somehow. I simply cannot understand the legal gymnastics needed to punish your customers for this sort of thing; I'm tired but I feel like this is not exactly an uncommon occurrence. Anyways let us all learn from my mistake and do not be rash and curtail your own freedoms.
dev_null@lemmy.ml · 8 pts · 256d
They were not only not allowing it, they immediately blocked the user's attempt to put it on their servers and banned the user for even trying. That's as far from allowing it as possible.
abbiistabbii@lemmy.blahaj.zone · 5 pts · 256d
This, literally the only reason I could guess is that it is to teach AI to recognise childporn, but if that is the case, why is google going it instead of like, the FBI?
gustofwind@lemmy.world · 7 pts · 256d
Who do you think the FBI would contract to do the work anyway 😬
Maybe not Google but it would sure be some private company. Our government doesn’t do stuff itself almost ever. It hires the private sector
alias_qr_rainmaker@lemmy.world · 1 pts · 256d
alias_qr_rainmaker@lemmy.world · 2 pts · 256d
vimmiewimmie@slrpnk.net · 1 pts · 256d
What's the 'applescript'?
alias_qr_rainmaker@lemmy.world · 2 pts · 256d
alias_qr_rainmaker@lemmy.world · 1 pts · 256d
frongt@lemmy.zip · 1 pts · 256d
Google wants to be able to recognize and remove it. They don't want the FBI all up in their business.
Allero@lemmy.today · 1 pts · 256d
So, Google could be allowed to have the tools to collect, store, and process CSAM all over the Web without oversight?
Pretty much everyone else would get straight to jail for attempting that.
forkDestroyer@infosec.pub · 1 pts · 255d
Google isn't the only service checking for csam. Microsoft (and other file hosting services, likely) also have methods to do this. This doesn't mean they also host csam to detect it. I believe their checks use hash values to determine if a picture is already clocked as being in that category.
This has existed since 2009 and provides good insight on the topic, used for detecting all sorts of bad category images:
https://technologycoalition.org/news/the-tech-coalition-empowers-industry-to-combat-online-child-sexual-abuse-with-expanded-photodna-licensing/
finitebanjo@lemmy.world · 29 pts · 256d
My dumb ass sitting here confused for a solid minute thinking CSAM was in reference to a type of artillery.
pigup@lemmy.world · 15 pts · 255d
Combined surface air munitions
llama@lemmy.zip · 4 pts · 255d
Right I thought it was cyber security something or other like API keys now duck duck go probably thinks I'm a creep
echodot@feddit.uk · 1 pts · 254d
This guy is into some really weird stuff, and not the normal henti that everyone else is asking for.
Hozerkiller@lemmy.ca · 2 pts · 255d
I feel that I assumed it was something like SCCM.
cyberpunk007@lemmy.ca · 25 pts · 255d
Child sexual abuse material.
Is it just me or did anyone else know what "CSAM" was already?
pipe01@programming.dev · 13 pts · 255d
Yeah it's pretty common, unfortunately
TipsyMcGee@lemmy.dbzer0.com · 3 pts · 255d
Midvikudagur@lemmy.world · 11 pts · 254d
"Child pornography" is a term NGO's and Law enforcement are trying to get phased out. It makes it sound like CSAM is related to porn, when in fact it is simply abuse of a minor.
TipsyMcGee@lemmy.dbzer0.com · 1 pts · 254d
ulterno@programming.dev · 0 pts · 254d
It does.
Somehow acronyms just end up sounding cool. Guess we should just use the full form. That would be better.
chronicledmonocle@lemmy.world · 6 pts · 254d
I had no idea what the acronym was. Guess I'm just sheltered or something.
hummingbird@lemmy.world · 19 pts · 256d
It goes to show: developers should make sure they don't make their livelihood dependent on access to Google services.
arararagi@ani.social · 9 pts · 254d
"stop noticing things" -Google
rizzothesmall@sh.itjust.works · 8 pts · 256d
Never heard that acronym before...
TheJesusaurus@sh.itjust.works · 21 pts · 256d
Not sure where it originates but it's the preferred term in UK policing and therefore most media reporting to refer to what might have been called "CP" on the interweb in the past as CSAM. Probably because porn implies it's art rather than crime, and also just a wider umbrella term
Zikeji@programming.dev · 19 pts · 256d
It's also more distinct. CP has many potential definitions. CSAM only has the one I'm aware of.
yesman@lemmy.world · 11 pts · 256d
LOL, You mean the letters C and P can stand for lots of stuff. At first I thought you meant the term "child porn" was ambiguous.
drdiddlybadger@pawb.social · 7 pts · 256d
Weirdly people have also been intentionally diluting the term to expand it to other things which causes a number of legal issues.
QueenHawlSera@sh.itjust.works · 2 pts · 256d
I've seen Cyberpunk fans claim to be into "CP", usually it's an innocent mistake and don't catch it at first, but are very embarassed when they do
Tollana1234567@lemmy.today · 3 pts · 256d
its always been child porn, when your using cp in the context with media/human, if its medical forum, you rarely use it to describe chickenpox.
SereneSadie@quokk.au · 2 pts · 256d
finitebanjo@lemmy.world · -1 pts · 256d
Counter Surface to Air Missile equipment or ordinance.
Why not just say Child Porn or Explicit Images of Children? Who are we protecting by refusing to say the words?
rizzothesmall@sh.itjust.works · 12 pts · 256d
Lol why tf people downvoting that? Sorry I learned a new fucking thing jfc.
Deceptichum@quokk.au · -1 pts · 256d
DylanMc6@lemmy.ml · 7 pts · 255d
gill o' teens
devolution@lemmy.world · 6 pts · 256d
Gemini likes twins...
...I'll see myself out.
DylanMc6@lemmy.ml · 6 pts · 255d
time for guillotines
billwashere@lemmy.world · 2 pts · 254d
I imagine most of these models have all kinds of nefarious things in them, sucking up all the info they could find indiscriminately.
b_tr3e@feddit.org · 1 pts · 256d
That's what you get for critisising AI - and righ so. I for one, welcome our new electronic overlords!
b_tr3e@feddit.org · 3 pts · 256d
cough: https://knowyourmeme.com/memes/i-for-one-welcome-our-new-insect-overlords
j4k3@piefed.world · -12 pts · 256d
minorkeys@lemmy.world · 2 pts · 256d
Me stupid. Pls dumbsplain.
j4k3@piefed.world · 0 pts · 256d
minorkeys@lemmy.world · 1 pts · 256d
Mmmhmmm....mhhmmm yes, okay, uh huh, i see....so the conclusion I've reached is i'm stupid.