c/Technology · by Ek-Hou-Van-Braai@piefed.social · 334d

REALITY CHECK: Everyone thinks they can spot AI photos. Here is a game that tests if that's true.

The game is simple:

Two photos side by side
One's made by a human, one's made with AI
Pick the AI one
See if you're right (plus the source/prompt)

150 points · 73 comments · view on lemmy.world

73 Comments

Get_Off_My_WLAN@fedia.io · 60 pts · 334d (1 reply)

Is this site just going to use the results to train AI to generate better fake imagery?

sukhmel@programming.dev · 5 pts · 333d

I don't think much training can be done on 20 pictures

minorkeys@lemmy.world · 20 pts · 333d (1 reply)

This is literally us giving them training data for AI to identify what pictures look like AI...

WolfLink@sh.itjust.works · 2 pts · 333d

Which in turn is training data to help AI make better pictures

Galapagon@sh.itjust.works · 17 pts · 334d

I didn't zoom in or anything like that, but I also feel like I was playing against "Dev traps" instead of unbiased pairs if that makes sense.

🤖 Reality Check #1 15/20 🔥 Perfect Streak: 9

I can spot AI images 75% of the time. 🧠 Can you beat me? realitycheckk.com

Schal330@lemmy.world · 15 pts · 334d (1 reply)

🤖 Reality Check #1 20/20 🔥 Perfect Streak: 20

I can spot AI images 100% of the time. 🧠

Most of the AI generated images looked too clean and smooth. It made me appreciate the real images more.

Fermion@mander.xyz · 17 pts · 334d

Yeah, the quickest way to tell was to look for things that AI wouldn't include, like brown tips on flower petals or broken sticks that have the right fracture pattern and don't just fade away. AI tends to generate things that are very averaged/ideal looking. The textures of ai images also look wavy with like a 10-20pixel periodicity when zoomed in. The noise from real cameras is much more speckly.

I also got 20/20, but only because it was an A/B comparison. If asked AI or real on each image indlividually, I'm not sure I would get much above 65%.

Raiderkev@lemmy.world · 14 pts · 334d (2 replies)

70%. I noticed that the more I tried to dissect it, the worse I was. I was far more accurate taking a quick glance at both and making a decision after like 5 seconds.

shalafi@lemmy.world · 2 pts · 333d

Just like multiple choice quizzes, go with your first guess if unsure.

WolfLink@sh.itjust.works · 2 pts · 333d

Interesting, I was kinda the opposite. I got 80% looking very carefully at each pair.

RonSijm@programming.dev · 13 pts · 334d (6 replies)

Reality Check #1 19/20

This thing messed me up:

They both looked like they could be AI - but to me it looks like that one has an AI artifact

shalafi@lemmy.world · 6 pts · 333d

Meh, the block just got pushed vertical, but it is weird, agreed. The one on the left doesn't strike me as natural. Ice neither forms nor breaks up like it's been put through a blender.

ApollosArrow@lemmy.world · 3 pts · 333d (1 reply)

This broke my streak…

TempermentalAnomaly@lemmy.world · 1 pts · 333d

Same

Cethin@lemmy.zip · 2 pts · 333d

Here's the thing: you should expect some things that aren't average in reality. Sure, most ice won't do what you're looking at, but some will. The left image all looks the same across the image. There's nothing that is non-average looking.

sukhmel@programming.dev · 1 pts · 333d

That's one of the problems with AI images, anything that looks weird is now dismissed as AI even if the photos are century old. A colleague of mine said crabeater seal teeth are definitely AI, for instance.

::: spoiler crabeater seal skull :::

NigelFrobisher@aussie.zone · 1 pts · 333d

That’s the one got me too.

snooggums@piefed.world · 11 pts · 334d (1 reply)

So the hardest part was that a lot of the pairings both looked like AI and every 50/50 guess I made was wrong. Many real images that I got wrong looked like staged professional shots with a ton of digital manipulation afterward.

Final Accuracy: 60%

AI image generation is in this weird place between professional staged shots and amateur photography and is probably there because the training set includes both.

Hazzard@lemmy.zip · 3 pts · 333d

Same, I was looking for inconsistencies in lighting, black/white points, etc, and I think in several cases I picked an image as AI based on details that were a result of photo manipulation and not "real". I got the same score as you in the end here.

Kinda interesting, somewhat invalidated what I thought this was testing, but it's an interesting test nonetheless. I may retake this in a few days now that I know it's comparing AI to photoshop and not "reality". Would certainly need a different strategy.

Kayday@lemmy.world · 9 pts · 334d

🤖 Reality Check #1 16/20
🔥 Perfect Streak: 12
I can spot AI images 80% of the time. 🧠

It's almost always, "this looks too clean/perfect."

Valmond@lemmy.world · 8 pts · 333d

Those images are very AI-easy IMO. No crowds or complex images.

remon@ani.social · 7 pts · 334d

85% ... and a lot of these pictures were just landscapes. If there we more pictures with people, animals or other objects, this would be even easier.

sip@programming.dev · 6 pts · 333d (2 replies)

fuck that UI. tapping to select and zoom below is odd. I messed up a few and exited.

Dragonstaff@leminal.space · 2 pts · 333d

The swipe to move to the next slide was so similar to the swipe backwards, I exited twice and gave up.

Ek-Hou-Van-Braai@piefed.social · 1 pts · 333d

Skill issue /s

Etterra@discuss.online · 6 pts · 333d (1 reply)

Nice try but I'm not doing QA checks for generative AI slop.

Dragonstaff@leminal.space · 1 pts · 333d

It seems like a reply to the wrong point.

I'm not saying most people can tell the difference. If they can now, they won't soon. My issues with AI are about training on people's work and then killing their jobs. Also destroying the environment, and other issues.

pero@lemmy.zip · 5 pts · 333d (1 reply)

🤖 Reality Check #1 19/20 🔥 Perfect Streak: 18

I can spot AI images 95% of the time. 🧠 Can you beat me? realitycheckk.com

The one pic with the pink flower and water bubbles under water really put me off

crapwittyname@feddit.uk · 3 pts · 333d

Yep that got me too. Those lily pads looked very unlikely

Blaze@piefed.zip · 5 pts · 333d

🤖 Reality Check #1 16/20
🔥 Perfect Streak: 7

I can spot AI images 80% of the time. 🧠
Can you beat me? realitycheckk.com

Quite interesting

Thorry@feddit.org · 5 pts · 334d

🤖 Reality Check #1 18/20 🔥 Perfect Streak: 7

I can spot AI images 90% of the time. 🧠 Can you beat me? realitycheckk.com

The two I got wrong were both very artsy and abstract. Anytime there is anything with detail in the image either check it's super fuzzy and dreamlike or it's way too sharp. I looked at each image for no more than a few secs, I feel with say a minute per image you could easily get close to 100%.

Zacryon@feddit.org · 5 pts · 333d

Plot twist: they were all AI generated.

That's what I would do if I would create such a test. But I would add the option "can not decide" to keep it a bit more fair.

popcar2@programming.dev · 4 pts · 334d

85%! I've gotten pretty good at this.

🤖 Reality Check #1 17/20

🔥 Perfect Streak: 8

LemmyKnowsBest@lemmy.world · 4 pts · 333d (1 reply)

Am dumb?

🤖 12/20 🔥 Perfect Streak: 4

I can spot AI images 60% of the time. 🧠

Fredselfish@lemmy.world · 3 pts · 333d

Got up to 11 myself before I lost it.

kittenzrulz123@lemmy.blahaj.zone · 4 pts · 334d

FUCK AI

TempermentalAnomaly@lemmy.world · 4 pts · 333d

🤖 Reality Check #1 18/20 🔥 Perfect Streak: 10

I can spot AI images 90% of the time. 🧠 Can you beat me? realitycheckk.com

hamms@lemmy.world · 4 pts · 334d

🤖 Reality Check #1 19/20

🔥 Perfect Streak: 14

Damn, so close

JackbyDev@programming.dev · 3 pts · 334d (1 reply)

I got 14/20. Once I stopped over thinking it it was easier. Sometimes I found myself picking the ones I thought looked "more realistic" because I kept getting them wrong and if worked better. Something something hyperreal.

shalafi@lemmy.world · 2 pts · 333d

I got 90% and most of my guesses we against the pic that looked too perfect. Hyperreal indeed!

AnIntenseMoist@lemmy.world · 3 pts · 334d

75%

At first I was analyzing both and picking the wrong one (one of the human ones had a tree that looked like early AI lmao), eventually I just went off on instinct and ended up being right on 90% of them. I guess the details were fine-tuned, but there's still something off about them.

Ek-Hou-Van-Braai@piefed.social · 3 pts · 334d (1 reply)

I got 70%, Which is not that much better than random chance (50%) Scary stuff

Protoknuckles@lemmy.world · 2 pts · 334d

I only ot 50% lol

Steve@startrek.website · 3 pts · 334d

80%

hperrin@lemmy.ca · 3 pts · 333d

🤖 Reality Check #1 18/20 🔥 Perfect Streak: 9

I can spot AI images 90% of the time. 🧠 Can you beat me? realitycheckk.com

SaveTheTuaHawk@lemmy.ca · 3 pts · 334d

The problem with this is that it's mostly nature scenes. Once images of people are used, it's much easier to detect slop.

Gobbel2000@programming.dev · 3 pts · 333d

Reality Check #1 18/20
Perfect Streak: 13

I can spot AI images 90% of the time.

BatmanAoD@programming.dev · 2 pts · 333d

🤖 Reality Check #1 20/20 🔥 Perfect Streak: 20

...some of those looked much more impressive than I expected, though.

hedge_lord@lemmy.world · 2 pts · 333d

It looks too perfect and kinda slimy so that's how I know.

BallShapedMan@lemmy.world · 2 pts · 334d (2 replies)

I miss clicked once by mistake but otherwise this is my score. Pretty interesting. Though if I wasn't given a side by side to pick just one I'm not sure I would have guessed as many of them. When it's just two it's pretty easy to get lucky. I'd like to see something like this with more than two at once and told X amount are AI and see how well we do. My bet is we do worse.

🤖 Reality Check #1 16/20 🔥 Perfect Streak: 12

I can spot AI images 80% of the time. 🧠 Can you beat me? realitycheckk.com

Kayday@lemmy.world · 3 pts · 334d (1 reply)

You could also do one at a time and have to guess on it's own

BallShapedMan@lemmy.world · 1 pts · 334d

Oh great point! I'm sure I would have done worse that way as well. Particularly if it didn't give me feedback until the end.

dylanmorgan@slrpnk.net · 2 pts · 334d

80%, better than I expected for myself. It does help that they are showing similar images side by side.

Kolanaki@pawb.social · 2 pts · 333d

20/20.

The nature shots were harder than shit that had man-made things or people in them. But also, the resolution of each pairing being dramatically different kinda gave a pattern to which one would be which.

eleijeep@piefed.social · 2 pts · 334d

ezpz

🤖 Reality Check #1 20/20
🔥 Perfect Streak: 20

I can spot AI images 100% of the time. 🧠
Can you beat me? realitycheckk.com

ALilOff@lemmy.world · 2 pts · 333d

Just me not reading any directions till 3 in, I thought I was picking the real one at first

474D@lemmy.world · 2 pts · 334d

The problem with this test is that it all depends on the model for the AI pictures. There are models now that specifically aim to have imperfections to emulate real pictures, unlike these

starman2112@sh.itjust.works · 2 pts · 333d

🤖 Reality Check #1   14/20
🔥 Perfect Streak: 8

I can spot AI images 70% of the time. 🧠
Can you beat me? realitycheckk.com

bdonvr@thelemmy.club · 2 pts · 334d

It's a lot easier if you zoom in and check fine patterns.

mcmodknower@programming.dev · 2 pts · 334d

🤖 Reality Check #1 16/20 🔥 Perfect Streak: 7

I can spot AI images 80% of the time. 🧠 Can you beat me? realitycheckk.com

Someone should make a study about this.

RandomStickman@fedia.io · 2 pts · 334d

🤖 Reality Check #1 18/20 🔥 Perfect Streak: 9

I can spot AI images 90% of the time. 🧠 Can you beat me? realitycheckk.com

Granted I accidentally refreshed half way through so maybe my score should be 17/20

The images are pretty good but a lot of them still have the "too polished" look.

Also wish the text to copy doesn't look so AI itself lol

BaroqueInMind@piefed.social · 2 pts · 334d

We are cooked.

ChexMax@lemmy.world · 1 pts · 333d

🤖 Reality Check #1 15/20 🔥 Perfect Streak: 4

I can spot AI images 75% of the time. 🧠 Can you beat me? realitycheckk.com

The stars ones beat me every time. How can that many stars be a real photo?? Long exposure blah blah blah

itsgroundhogdayagain@lemmy.ml · 1 pts · 334d

75% I found the landscape photos much more difficult.

samus12345@sh.itjust.works · 1 pts · 333d

78%, not bad. Anything with no people or animals are always harder to spot, especially with how small the pictures are. I want to try one with with bigger pictures and people prominently featured.

ABetterTomorrow@sh.itjust.works · 1 pts · 334d (1 reply)

🤖 Reality Check #1 14/20 🔥 Perfect Streak: 5

I can spot AI images 70% of the time. 🧠 Can you beat me? realitycheckk.com

ABetterTomorrow@sh.itjust.works · 1 pts · 334d

The ones I got wrong I rushed and selected my first instinct without looking them over for at least ten seconds.

Saledovil@sh.itjust.works · 1 pts · 333d

🤖 Reality Check #1 13/20 🔥 Perfect Streak: 10

I can spot AI images 65% of the time. 🧠 Can you beat me? realitycheckk.com

I can clearly do it sometimes, but the other people in this thread got me beat.

BlameTheAntifa@lemmy.world · 1 pts · 334d

🤖 Reality Check #1 20/20 🔥 Perfect Streak: 20

I can spot AI images 100% of the time. 🧠 Can you beat me? realitycheckk.com