c/fuck_ai · by Dot@feddit.org · 1y

AI tools show biases in ranking job applicants’ names according to perceived race and gender.

https://www.washington.edu/news/2024/10/31/ai-bias-resume-screening-race-gender/

Research paper.

93 points · 11 comments · view on lemmy.world

11 Comments

FlyingSquid@lemmy.world · 18 pts · 1y (3 replies)

I knew it! I've been saying that it is!

And the AI algorithms they're using in real estate are doing the exact same thing. Mark my words.

henfredemars@infosec.pub · 11 pts · 1y (2 replies)

It’s just a method of shifting the blame And responsibility away from real people to try to avoid getting punished.

MajorHavoc@programming.dev · 4 pts · 1y (1 reply)

It’s just a method of shifting the blame And responsibility away from real people to try to avoid getting punished.

Yep. Biased AI works just fine when the goal is creation of an as-desperate-as-possible underclass.

Only, it still doesn't, because eventually even the ultra rich sponsor's niece is equally dead - once the AI takes over doctoring and mis-recommemds a needless and dangerous surgery.

thevoidzero@lemmy.world · 4 pts · 1y

Machine learning models in medicine has biases against darker skins for identifying diseases. So I'm guessing lot of that will be there too

lmaydev@lemmy.world · 13 pts · 1y

Wow they've mimicked human hiring managers so well!

TheFriar@lemm.ee · 13 pts · 1y

Shit in, shit out.

captainlezbian@lemmy.world · 11 pts · 1y

Ah the eternal question for female engineers: first name or first initial on your resume

huginn@feddit.it · 10 pts · 1y

Quote from abstract:

We find that the MTEs are biased, significantly favoring White-associated names in 85.1% of cases and female-associated names in only 11.1% of cases, with a minority of cases showing no statistically significant differences. Further analyses show that Black males are disadvantaged in up to 100% of cases, replicating real-world patterns of bias in employment settings, and validate three hypotheses of intersectionality. We also find an impact of document length as well as the corpus frequency of names in the selection of resumes.

Pretty damning... And not surprising.

iAvicenna@lemmy.world · 7 pts · 1y

"AI, don't rank applicants based on race and gender"

"Ok"

*Proceeds to rank applicants based on how western and male looking their names are"

PunnyName@lemmy.world · 0 pts · 1y (1 reply)

It's not really AI, that's why. It's not even generative.

It's a scraper that uses already available information to then "learn". But it doesn't actually learn, or it would also scrape systemic and societal changes that many deem necessary to overcome these biases.

prototype_g2@lemmy.ml · 1 pts · 1y

It's not even generative

It doesn't need to be generative to be AI.

It’s a scraper that uses already available information to then “learn”.

That's just every single "AI" product out there, that's how they work: They scrape data from all over the internet, create a model that makes predictions based on that data. Chat-GPT doesn't understand anything. It is simply a really complicated model which predicts what word is most likely to follow a given sequence of words. These "AI" aren't inteligent, nor are they creative. They, by their very nature, stay as close as possible to the data they are given and never deviate, as a deviation would mean inaccuracy.

From an historical point of view, the word "AI" simply means "cool new technology". That's what it has been used to describe, while people think that AI means "artificial person", like we see in the movies. So we need to be careful while using this word, because it can mean so many thing to the point that it has little meaning.

ZDL@ttrpg.network · -1 pts · 1y

[ removed ]