cross-posted from: https://beehaw.org/post/27795833
In a study published in the journal Judgment and Decision Making, 1,682 adults were asked to read one of six short stories, three of which were written by humans and three by ChatGPT. Each AI story had a similar theme to one of the human-authored works.
The team told participants whether their story was written by a human or AI, but this information was not always correct. The researchers then asked participants to rate how absorbing and engaging they found the story, and its quality.
Participants who read an AI-generated story rated it as more absorbing and of higher quality than those who read a story written by a human. However, participants gave higher ratings to stories they had been told were written by people.
Dr Deena Skolnick Weisberg, a senior author of the research from Villanova University in Pennsylvania, said: “AI systems can already generate short stories that are seen as being at least as good as – if not better than – human-written stories. We should update our views of AI’s abilities accordingly.”
However, Weisberg, who is herself a creative writer, said that did not mean writing should be left to AI, noting that novels produced by tech were probably going to be different from those written by humans.
“We may need to make room for AI-generated novels, and for AI/human co-written novels, but that doesn’t mean that there’s no longer space for us to appreciate the process of human creativity,” she said.
17 Comments
DougPiranha42@lemmy.world · 15 pts · 1d
I didn’t read the article, but from the summary, it’s not obvious which humans wrote the stories. Famously, the ability of humans for writing is not the same for everyone. The average person writes like shit. The average writer writes kind of ok. The top few percentile of writers write really well, and the rest of humans like to read what those writers write. So if in the study, they asked a 6 year old, a plumber, and an accountant for a story, I completely understand that ai rated better. But even then, I would rather read 10 bad stories by random people than 10 passable-appearing ai stories.
joe@lemmy.world · 2 pts · 1d
From the study. They go on to list out all the stories, where they were published, when, the author, and the prompt they used for the AI equivalent. (It was a lot to copy here, though, so I didn't.)
Part of the study is that you probably wouldn't be able to tell, and that if you were told that a story was written by AI, and you are biased against AI, you'd rate that story as worse, even if it were written by a human.
markstos@lemmy.world · 9 pts · 1d
The first AI story I read: sure.
After a few more? I felt a certain sameness to them.
plantfanatic@sh.itjust.works · 2 pts · 1d
I mean… yeah? All authors have their own styles and they sound the same if you read a decent chunk of a single author.
A different AI and different prompts would be like a different author and would relieve that issue.
It’s emulating real life pretty well if that’s the case.
SchmidtGenetics@lemmy.world · 2 pts · 1d
Do you have any examples? The little I’ve played with Ai the different models can give wildly different results from even just asking for a synopsis of an email.
If you are just using the same Ai and similar prompts like “write a murder mystery” I could see it happening. But that’s just not understating AI at that point.
But at the same time, ask the same author to write 10 murder mysteries and they’ll sound similar too.
joe@lemmy.world · 1 pts · 1d
Do you have data to support this claim? Or do you mean if you keep giving the same prompt you'll get very similar outputs?
markstos@lemmy.world · 1 pts · 1d
Personal experience.
joe@lemmy.world · -3 pts · 1d
Okay, your opinion isn't very valuable in this context, but maybe we can salvage something. How many AI stories have you read?
cinoreus@lemmy.world · 8 pts · 1d
Don't underestimate an average person's love for slop content. Yes it's true AI systems have gotten better and will continue to get better, but it's also not what most people care. My mom watches AI generated drama everyday. My dad too has no problem with AI short form content.
It sucks but that's the state of the world right now.
Edit: thanks to the guy who posted the link to the study below. In that study it's written that a similar test was performed in 2024 and they had similar results. Somehow only proving my point.
sanguinepar@lemmy.world · 6 pts · 1d
I don't give two good shits if AI does write better (which I doubt). I don't want to read artificially created writing based on material stolen from its human authors.
HubertManne@piefed.social · 0 pts · 1d
my thing is whatever. we will see if I get excited by some sort of media but currently even before ai stuff its been pretty enshitified. LOTR and firefly is sorta the height for me with initial marvel being decent and there is still some good animation for now.
sanguinepar@lemmy.world · 2 pts · 1d
To me it's just about how it came to be - if there any value in art (be it film, books, photos, TV, whatever) then it's because a human (or humans) made it. Because they did what someone else couldn't have done, because the other person wasn't them. A machine that synthesises other people's work and makes an imitation of it can't ever be the same.
HubertManne@piefed.social · 1 pts · 1d
meh. im fine with entertainment if its good. I have no specific thing with ai other than its power usage, the effect on society which is more about a lack of social benefits, and that it be good. so far I have not had any evidence it can make something good.
sanguinepar@lemmy.world · 1 pts · 22h
Fair enough 👍
bravesilvernest@lemmy.ml · 2 pts · 1d
Damnit I'm gonna see this all over now aren't I?
The actual study
eleijeep@piefed.social · 2 pts · 1d
Here are the stories they used, if you want to decide for yourself. Although it won't be a blind test since in these documents they tell you which is which:
Story pair 1
Story pair 2
Story pair 3
odelik@lemmy.today · 3 pts · 22h
After reading through these, I have to say, none of these are stellar examples of writing from the human authors (good, or above average for sure). However I can tell that there's a personal touch there.
However, the AI generated stuff easily starts falling into purple prose, and other bad literary patterns that show up commonly. There's also something missing, disconnected, and off. Everything is there for the story, yet it's organization, telling of events, and reaching for odd details triggers that uncanney valley response of "something isn't right here".