Yes, thanks for clarifying what I meant! AI will never create anything unique unless prompted uniquely and even then it will tend to revert back to what you expect most.
ATTN: If you're coming into this thread to say, "The output of AI is bad because your prompts suck," I'm just proud that you managed to figure out how to use the internet at all. Good job, you!
Thing is you can do in real time and not pay as much attention to the goings on as you write or do it in the end and forget stuff. there is no harm in the ai summariziation. you could instead write a summary and check if you left anything out via the ai.
They certainly do. For a while it was common to see AI-generated summaries under links to articles on lemmy, so I got a feel for them. Seems to me you would not need any fancy artificial intelligence to do equally well: Just take random excerpts, or maybe just read every third sentence.
Could it be because a statistical relation isn't the same as a semantic one? No, I must be prompting it wrong. I'll just add "engineer" to my title and then everyone will take me seriously.
Is it only me, or is the linked article not super long on details & is reaching a conclusion from 2 examples? This is important & I need to hear more, & I’m generally biased against AI at this point— but the article isn’t doing enough to convince me
did you click through to any of the inline citations? David’s shorter articles on pivot mostly gather and summarize those, so if you need to read the original research and its conclusions that’s where to go
I had GPT 3.5 break down 6x 45-minute verbatim interviews into bulleted summaries and it did great. I even asked it to anonymize people’s names and it did that too. I did re-read the summaries to make sure no duplicate info or hallucinations existed and it only needed a couple of corrections.
good tools are designed well enough so it's clear how they are used, held, or what-fucking-ever.
fuck these simpleton takes are a pain in the arse. They're always pushed by these idiots that have based their whole world view on fortune cookie aphorisms
How did you make sure no hallucinations existed without reading the source material; and if you read the source material, what did using an LLM save you?
I also use it for that pretty often. I always double check and usually it's pretty good. Once in a great while it turns the summary into a complete shitshow but I always catch it on a reread, ask a second time, and it fixes things up. My biggest problem is that I'm dragged into too many useless meetings every week and this saves a ton of time over rereading entire transcripts and doing a poor job of summarizing because I have real work to get back to.
I also use it as a rubber duck. It works pretty well if you tell it what it's doing and tell it to ask questions.
what if your rubber duck released just an entire fuckton of CO2 into the environment constantly, even when you weren’t talking to it? surely that means it’s better
Yup! I’ll feed in meeting transcripts and get a list of action steps to email out to everyone. If I was in project management, I’m pretty sure i’d outsource my entire job to LLMs.
But the claims of the text are often why you read it in the first place! If you have a hundred scientific papers you're going to read the ones that make claims either supporting or contradicting your research.
I agree, you're quite right, and I thank you for taking the time and putting in the effort on such a wonderfully thorough portrayal of why your argument is total horseshit
The problem is not the LLMs, but what people are trying to do with them.
They are currently spoons, but people are desperately wishing they were katanas.
They work really well for soup, but they can't cut steak. But they're being hyped as super ninja steak knives, and people are getting pissed when they can't cut steak.
If you give them watery, soupy tasks they can do successfully, they can lighten your workload, as long as you're aware of what they are and aren't good at.
What people want LLMs to be able to do, ie. "Steak" tasks:
write complex documents
apply complex knowledge/rules to a situation
Write complex code and create entire programs based on vague description
What LLMs can currently do ie. "Soup" tasks:
check this document and fix all spelling, punctuation and grammatical errors
summarise this paragraph as dot points
write a python program that sorts my photographs into folders based on the year they were taken
Half of Lemmy is hyping katanas, the other half is yelling "Why won't my spoon cut this steak?!! AI is so dumb!!!"
Update: wow, the pure vitriol pouring out of the replies is just stunning. Seems there are a lot of you out there who have, in one way or another, tied your ego very strongly to either the success or failure of AI.
Take a step back, friends, and go outside for a while.
Clearly this post is about LLMs not succeeding at this task, but anecdotally I've seen it work OK and also fail. Just like humans, which is the benchmark but they are faster.
That phrase contains inappropriate language and could be seen as disrespectful or offensive. If you're looking for a more polite or constructive way to express frustration or dismissal, you could say something like:
Also, you want to talk about bad faith arguments, this was presented to parliament in May 2024. It was submitted in January 2024. Model selection and optimisation was done in October 2023.
Llama3 was released April 2024. They did not use an old model to intentionally tank the results, as you are implying. Llama2 was the 'latest-and-greatest' at the time of the study.
I think we can agree that AI is evolving slightly faster than sedan technology
Also, you want to talk about bad faith arguments, this was presented to parliament in May 2024. It was submitted in January 2024. Model selection and optimisation was done in October 2023.
And the article was posted today. I can post old data all day long. Got cancer? Just drink this heroin.
Llama3 was released April 2024. They did not use an old model to intentionally tank the results, as you are implying. Llama2 was the ‘latest-and-greatest’ at the time of the study.
Ok, fine, I'll accept your correction, if you'll accept my updated summary:
The article was written in bad faith with outdated data in an attempt to turn AI disparagement into SEO into money.
Ok? I don't have another human available to skim a shitload of documents for me to find answers I need and I don't have time to do ot myself. AI is my best option.
Yep. Go ahead and ignore all the cases where it's getting answers correct and actually helping. We're all just hallucinating, it's in no way my lived experience.
Your reality is the prime reality and we're the NPC's.
I didn't read the post at all because its premise is irrelevant to my situation. If I had another human to read documentation for me I would do that. I don't so the next best thing is AI. I have to double check its findings but it gets me 95% of the way there and saves hours of work. It's a useful tool.
we really do need “my source is that I made it the fuck up” for people who aggressively don’t want to read any of the text they’re allegedly commenting on
Ah, interesting. What exactly makes you think so? Specifically where I was talking about this topic (and was downvoted just as much) how I use LLM to perform board level repairs with reprogramming of chips? Perhaps there is an when better fitting post I forget where I was talking about pointless nonsense? Or to you it would be far better if I stopped doing those pointless repairs and just bought new stuff.
105 Comments
swlabr@awful.systems · 86 pts · 1y
LLMs, and everyone who uses them to process information:
hex@programming.dev · 60 pts · 1y
I kind of like this because it highlights the way LLMs operate kind of blind and drunk, they're just really good at predicting the next word.
CleoTheWizard@lemmy.world · 28 pts · 1y
They’re not good at predicting the next word, they’re good at predicting the next common word while excluding most unique choices.
What results is essentially if you made a Venn diagram of human language and only ever used the center of it.
hex@programming.dev · 15 pts · 1y
Yes, thanks for clarifying what I meant! AI will never create anything unique unless prompted uniquely and even then it will tend to revert back to what you expect most.
swlabr@awful.systems · 46 pts · 1y
ATTN: If you're coming into this thread to say, "The output of AI is bad because your prompts suck," I'm just proud that you managed to figure out how to use the internet at all. Good job, you!
froztbyte@awful.systems · 15 pts · 1y
(not that I much agree with the classist overtones of the original, but fuck me does it come to mind often)
Sibbo@sopuli.xyz · 30 pts · 1y
Well, to be fair, AI can do it in seconds. Which beats humans.
But if that is relevant if the results are worthless is another question.
HubertManne@moist.catsweat.com · 12 pts · 1y
Yeah it changes the task from note taking or summarizing to proofreading.
YourNetworkIsHaunted@awful.systems · 5 pts · 1y
And proofreading is notably more complex and has a worse failure state than just writing your own summary.
HubertManne@moist.catsweat.com · -4 pts · 1y
Thing is you can do in real time and not pay as much attention to the goings on as you write or do it in the end and forget stuff. there is no harm in the ai summariziation. you could instead write a summary and check if you left anything out via the ai.
self@awful.systems · 4 pts · 1y
that’s great thanks
kbal@fedia.io · 21 pts · 1y
They certainly do. For a while it was common to see AI-generated summaries under links to articles on lemmy, so I got a feel for them. Seems to me you would not need any fancy artificial intelligence to do equally well: Just take random excerpts, or maybe just read every third sentence.
dgerard@awful.systems · 21 pts · 1y
how the hell did this of all the posts turn into a promptfondler shooting gallery
froztbyte@awful.systems · 11 pts · 1y
underscore_@sopuli.xyz · 2 pts · 1y
Promptfondler has to be my new favourite slur!
zogwarg@awful.systems · 6 pts · 1y
*Epithet
dgerard@awful.systems · 18 pts · 1y
i have seen the light from the helpful posters here, made up bullshit alleged summaries of documents are great actually
GBU_28@lemm.ee · 14 pts · 1y
Dang everyone here needs to look at a tree or a cat or something. Energy is wack in here
dgerard@awful.systems · 28 pts · 1y
I just went outside and appreciated the rendering
GBU_28@lemm.ee · 10 pts · 1y
Pretty nice right? I did the trees and cats.
froztbyte@awful.systems · 8 pts · 1y
DANGER WILL ROBINSON, godposting detected
AcausalRobotGod@awful.systems · 10 pts · 1y
I have some competition!
dgerard@awful.systems · 8 pts · 1y
if people don't appreciate the kitties their tamagotchi is in some fucking trouble
V0ldek@awful.systems · 9 pts · 1y
While reading this entire stuff I periodically looked at my cat and let out a sigh, and he just looks at me with that knowing gaze
"Ye, you are all dumb, hoomans. Don't think about it. Pet me now."
Empricorn@feddit.nl · 4 pts · 1y
Nearly every cat is a tree-cat.
khalid_salad@awful.systems · 11 pts · 1y
Could it be because a statistical relation isn't the same as a semantic one? No, I must be prompting it wrong. I'll just add "engineer" to my title and then everyone will take me seriously.
beefbot@lemmy.blahaj.zone · 4 pts · 1y
Is it only me, or is the linked article not super long on details & is reaching a conclusion from 2 examples? This is important & I need to hear more, & I’m generally biased against AI at this point— but the article isn’t doing enough to convince me
self@awful.systems · 12 pts · 1y
did you click through to any of the inline citations? David’s shorter articles on pivot mostly gather and summarize those, so if you need to read the original research and its conclusions that’s where to go
beefbot@lemmy.blahaj.zone · 11 pts · 1y
Ah, that’s better, yes. Thank you , no sarcasm :) now sleepy brain is more informed
RagnarokOnline@programming.dev · 3 pts · 1y
I had GPT 3.5 break down 6x 45-minute verbatim interviews into bulleted summaries and it did great. I even asked it to anonymize people’s names and it did that too. I did re-read the summaries to make sure no duplicate info or hallucinations existed and it only needed a couple of corrections.
Beats manually summarizing that info myself.
Maybe their prompt sucks?
froztbyte@awful.systems · 38 pts · 1y
“Are you sure you’re holding it correctly?”
christ, every damn time
Jakeroxs@sh.itjust.works · -10 pts · 1y
That is how tools tend to work, yes.
fasterandworse@awful.systems · 15 pts · 1y
"tools" doesn't mean "good"
good tools are designed well enough so it's clear how they are used, held, or what-fucking-ever.
fuck these simpleton takes are a pain in the arse. They're always pushed by these idiots that have based their whole world view on fortune cookie aphorisms
dgerard@awful.systems · 14 pts · 1y
we find they tend to post here, though not for long
froztbyte@awful.systems · 11 pts · 1y
it makes me feel fucking ancient to find that this dipshit didn't seem to get the remark, and it wasn't even that long ago
istewart@awful.systems · 13 pts · 1y
Jobs is Tech Jesus, but Antennagate is only recorded in one of the apocryphal books
V0ldek@awful.systems · 8 pts · 1y
Said like a person who wouldn't be able to correctly hold a hammer on first try
dgerard@awful.systems · 26 pts · 1y
I got AcausalRobotGPT to summarise your post and it said "I'm not saying it's always programming.dev, but"
pikesley@mastodon.me.uk · 21 pts · 1y
@RagnarokOnline @dgerard "They failed to say the magic spells correctly"
HootinNHollerin@lemmy.world · 16 pts · 1y
Did you conduct or read all the interviews in full in order to verify no hallucinations?
RagnarokOnline@programming.dev · 2 pts · 1y
I conducted the interviews myself alongside a colleague.
sxan@midwest.social · 8 pts · 1y
How did you make sure no hallucinations existed without reading the source material; and if you read the source material, what did using an LLM save you?
TexasDrunk@lemmy.world · -9 pts · 1y
I also use it for that pretty often. I always double check and usually it's pretty good. Once in a great while it turns the summary into a complete shitshow but I always catch it on a reread, ask a second time, and it fixes things up. My biggest problem is that I'm dragged into too many useless meetings every week and this saves a ton of time over rereading entire transcripts and doing a poor job of summarizing because I have real work to get back to.
I also use it as a rubber duck. It works pretty well if you tell it what it's doing and tell it to ask questions.
YourNetworkIsHaunted@awful.systems · 9 pts · 1y
Isn't the whole point of rubber duck debugging that the method works when talking to a literal rubber duck?
self@awful.systems · 8 pts · 1y
what if your rubber duck released just an entire fuckton of CO2 into the environment constantly, even when you weren’t talking to it? surely that means it’s better
TexasDrunk@lemmy.world · 1 pts · 1y
RagnarokOnline@programming.dev · 2 pts · 1y
Yup! I’ll feed in meeting transcripts and get a list of action steps to email out to everyone. If I was in project management, I’m pretty sure i’d outsource my entire job to LLMs.
lvxferre@mander.xyz · -7 pts · 1y
dgerard@awful.systems · 17 pts · 1y
Both the use cases here are goverment documents. I'm baffled at the idea of it being "fine if the AI makes shit up".
queermunist@lemmy.ml · 16 pts · 1y
ChatGPT gives you a bad summary full of hallucinations and, as a result, you choose not to read the text based on that summary.
lvxferre@mander.xyz · -4 pts · 1y
queermunist@lemmy.ml · 11 pts · 1y
But the claims of the text are often why you read it in the first place! If you have a hundred scientific papers you're going to read the ones that make claims either supporting or contradicting your research.
You might as well just skim the titles and guess.
lvxferre@mander.xyz · -7 pts · 1y
self@awful.systems · 15 pts · 1y
not reading the fucking sidebar and thinking this is high school debate club fallacy
lvxferre@mander.xyz · -8 pts · 1y
self@awful.systems · 9 pts · 1y
holy shit, imagine getting a second chance to not be a fucking debatelord and doubling down this hard
off you fuck
froztbyte@awful.systems · 7 pts · 1y
I agree, you're quite right, and I thank you for taking the time and putting in the effort on such a wonderfully thorough portrayal of why your argument is total horseshit
queermunist@lemmy.ml · 10 pts · 1y
Unless it doesn't accurately represent the topic, which happens, and then a researcher chooses not to read the text based on the chatbot's summary.
All these chatbots do is guess. I'm just saying a researcher might as well cut out the hallucinating middleman.
V0ldek@awful.systems · 6 pts · 1y
And if it's badly written then the LLM will shit itself.
Now let's ask ourselves how much of the text in the world is "well-written"?
Or even better, you could apply this to Copilot. How much code in the world is good code? The answer is fucking none, mate.
pikesley@mastodon.me.uk · 5 pts · 1y
@lvxferre @dgerard have you bumped your head?
lvxferre@mander.xyz · 2 pts · 1y
dgerard@awful.systems · 9 pts · 1y
You'd think so, but guess what precise use case LLMs are being pushed hard for.
z00s@lemmy.world · -8 pts · 1y
The problem is not the LLMs, but what people are trying to do with them.
They are currently spoons, but people are desperately wishing they were katanas.
They work really well for soup, but they can't cut steak. But they're being hyped as super ninja steak knives, and people are getting pissed when they can't cut steak.
If you give them watery, soupy tasks they can do successfully, they can lighten your workload, as long as you're aware of what they are and aren't good at.
What people want LLMs to be able to do, ie. "Steak" tasks:
write complex documents
apply complex knowledge/rules to a situation
Write complex code and create entire programs based on vague description
What LLMs can currently do ie. "Soup" tasks:
check this document and fix all spelling, punctuation and grammatical errors
summarise this paragraph as dot points
write a python program that sorts my photographs into folders based on the year they were taken
Half of Lemmy is hyping katanas, the other half is yelling "Why won't my spoon cut this steak?!! AI is so dumb!!!"
Update: wow, the pure vitriol pouring out of the replies is just stunning. Seems there are a lot of you out there who have, in one way or another, tied your ego very strongly to either the success or failure of AI.
Take a step back, friends, and go outside for a while.
V0ldek@awful.systems · 18 pts · 1y
The entire point here is that they can't?
fuzzzerd@programming.dev · -12 pts · 1y
Clearly this post is about LLMs not succeeding at this task, but anecdotally I've seen it work OK and also fail. Just like humans, which is the benchmark but they are faster.
self@awful.systems · 7 pts · 1y
humans are clearly faster at generating utterly banal shit, as proven by your posts in this thread
self@awful.systems · 15 pts · 1y
they don’t do any of that soup shit reliably either and reading the article might have told you that
z00s@lemmy.world · -8 pts · 1y
They absolutely do, and I have no idea why you're so angry
self@awful.systems · 8 pts · 1y
z00s@lemmy.world · 1 pts · 1y
That phrase contains inappropriate language and could be seen as disrespectful or offensive. If you're looking for a more polite or constructive way to express frustration or dismissal, you could say something like:
Would you like help rephrasing it further?
blakestacey@awful.systems · 12 pts · 1y
I'd offer congratulations on obfuscating a bad claim with a poor analogy, but you didn't even do that very well.
dgerard@awful.systems · 7 pts · 1y
more of a Trabant analogy than a Corvette analogy
froztbyte@awful.systems · 10 pts · 1y
good god this entire post is the most tortured believer whataboutism I've encountered this month and there's extremely strong competition here
you should make a youtube channel,
The Katana Steak-Eater. I'd watch the shit out of that at least one saturday afternoonistewart@awful.systems · 10 pts · 1y
Why did this immediately give me a flashback to Donald Trump yelling, "when it comes to great steaks, I've just raised the stakes!"
FredFig@awful.systems · 9 pts · 1y
This level of discourse wouldn't fly on 4chan, how is it so popular with LLM fans?
dgerard@awful.systems · 9 pts · 1y
needs to be a car analogy
self@awful.systems · 6 pts · 1y
more of a Power Wheels Barbie Jeep whose battery got left out in the sun too long, but I’ll allow it
froztbyte@awful.systems · 5 pts · 1y
don't diss the course, this steak's great
sc_griffith@awful.systems · 8 pts · 1y
"spoons and katanas" has got to be the most baby brained analogy. are you a child
z00s@lemmy.world · -9 pts · 1y
Thanks Donald, good luck in November
sc_griffith@awful.systems · 8 pts · 1y
I get that this is some sort of attempt at an election related Epic Comeback, but it doesn't make sense
z00s@lemmy.world · 1 pts · 1y
Your initial comment sounded like the way Donald Trump rage talks.
I'm not surprised you don't understand.
fuzzzerd@programming.dev · -12 pts · 1y
Who cares? It paints the correct picture and adds useful context.
froztbyte@awful.systems · 10 pts · 1y
you do realize steaks arriving purple or green are bad things, right
swlabr@awful.systems · 9 pts · 1y
it is stupid and wrong, and i pity your inability to understand that fact
sc_griffith@awful.systems · 9 pts · 1y
it doesn't do either of those things
swlabr@awful.systems · 7 pts · 1y
Actually, LLMs are syringes filled with brain-parasite-infested poop
chemical_cutthroat@lemmy.world · -9 pts · 1y
So, they used a year old model, and what? Expected a miracle? Do you expect your '87 Chrysler to have parallel parking assist?
Here. If humans are better in every way, I'll do you a favor and summarize the article.
If you want to hate AI, go for it, but give me a good goddamned reason to support your cause. This shit is an insult to my intelligence.
Edit: The Anti-AI brigade has arrived. You all gotta find a new hobby. Downvotes without discourse is just masturbation.
rImITywR@lemmy.world · 4 pts · 1y
1987 was one year ago?
Also, you want to talk about bad faith arguments, this was presented to parliament in May 2024. It was submitted in January 2024. Model selection and optimisation was done in October 2023.
Llama3 was released April 2024. They did not use an old model to intentionally tank the results, as you are implying. Llama2 was the 'latest-and-greatest' at the time of the study.
chemical_cutthroat@lemmy.world · -8 pts · 1y
I think we can agree that AI is evolving slightly faster than sedan technology
And the article was posted today. I can post old data all day long. Got cancer? Just drink this heroin.
Ok, fine, I'll accept your correction, if you'll accept my updated summary:
lightnsfw@reddthat.com · -13 pts · 1y
Ok? I don't have another human available to skim a shitload of documents for me to find answers I need and I don't have time to do ot myself. AI is my best option.
s3p5r@lemm.ee · 28 pts · 1y
So long as you don't care about whether they're the right or relevant answers, you do you, I guess. Did you use AI to read the linked post too?
jaemo@sh.itjust.works · -13 pts · 1y
Yep. Go ahead and ignore all the cases where it's getting answers correct and actually helping. We're all just hallucinating, it's in no way my lived experience. Your reality is the prime reality and we're the NPC's.
fruitdealer@lemmy.world · 25 pts · 1y
And I wish only my good grades counted in school too.
dgerard@awful.systems · 18 pts · 1y
sir has failed to achieve the reading comprehension level for this sub
V0ldek@awful.systems · 12 pts · 1y
YourNetworkIsHaunted@awful.systems · 5 pts · 1y
Only it's even worse because without redoing all the work yourself you can't even tell which ones are dead or alive.
lightnsfw@reddthat.com · -18 pts · 1y
I didn't read the post at all because its premise is irrelevant to my situation. If I had another human to read documentation for me I would do that. I don't so the next best thing is AI. I have to double check its findings but it gets me 95% of the way there and saves hours of work. It's a useful tool.
ebu@awful.systems · 26 pts · 1y
rather refreshing to have someone come out and just say it. thank you for the chuckle
self@awful.systems · 17 pts · 1y
we really do need “my source is that I made it the fuck up” for people who aggressively don’t want to read any of the text they’re allegedly commenting on
dgerard@awful.systems · 23 pts · 1y
everyone, we have a new worst poster
sc_griffith@awful.systems · 21 pts · 1y
absolutely superb posting, thank you
V0ldek@awful.systems · 11 pts · 1y
This is hall of fame shit right here, someone should study the way you use the internet sir
Eheran@lemmy.world · -17 pts · 1y
Hahaha what a load of nonsense.
ObviouslyNotBanana@lemmy.world · 22 pts · 1y
Summarised by Gemini
self@awful.systems · 16 pts · 1y
your post history tells me you’re pretty fucking comfortable with pointless nonsense
Eheran@lemmy.world · 0 pts · 1y
Ah, interesting. What exactly makes you think so? Specifically where I was talking about this topic (and was downvoted just as much) how I use LLM to perform board level repairs with reprogramming of chips? Perhaps there is an when better fitting post I forget where I was talking about pointless nonsense? Or to you it would be far better if I stopped doing those pointless repairs and just bought new stuff.