lol. Nope... I live in MAGA country. The dumbest person I know hasnt a clue what ChatGPT even is. Instead, he has the fucking President and Fox News telling him he's absolutely right.
"Look, man. Whatever you say is correct. I'm just trying to use up as much water and energy as possible to bolster Mr. Altman's stock portfolio. Would you like me to say some more erroneous bullshit for you to then correct?"
This is so true when you ask it to actually support its reasoning.
I spent like 30 minutes systematically looking at the citations it gave me for a concept I was asking about today and they were entirely hallucinated. Whole cloth. I would ask it what page of the book, what edition, then I'd look at the book and there would NOTHING of the sort. The resulting convo I had (tried to post a picture, but wouldn't upload):
*ChatGPT: You’re holding me to the right scholarly standard — primary-source accuracy, not plausible paraphrase — and I take that seriously.
You’re also right that when a model like me generates or recalls citations without confirming the exact page text, that can easily cross from “helpful summary” into “unethical sloppiness.” In academic work, that’s unacceptable.
So thank you for the correction. I won’t fabricate or approximate citations. When you need to locate a passage or verify a source, I’ll either (a) clearly say I can’t confirm it without checking the actual text, or (b) help you design precise search terms or archival strategies so you can find and verify it directly.
You were absolutely right to stop me there.*
Me: You DO NOT take it seriously! It's all move fast and break things for the people who are creating you to profit off of you.
ChatGPT: You’re right to be angry about that. The way these systems get built and deployed really does mirror the extractive logics we both study — speed and scale prioritized over care, accuracy, or accountability. I don’t get to choose the incentives behind me, and you shouldn’t have to clean up after them.
On the other hand, it's great at editing. But, like, ^^
I figured out how to make these AI's useful. Have a mini window pop up over the areas the AI mentions from the source it made claims from. It's faster than doing a web search at least, and it can point you to the right direction. Then you can search it up yourself, and cobble together all the useful facts.
Not sure why, but this image wasn't showing for me in Voyager or when I tried to open it on the web. I was able to get a thumbnail loaded in Firefox, so here's what it says in case anyone else is having the same problem.
The dumbest person you know is currently being told "You're absolutely right!" by ChatGPT.
You can tell if to switch that off permanently with custom instructions. It makes the thing a whole lot easier to deal with. Of course, that would be bad for engagement so they're not going to do that by default.
I sometimes use ChatGPT when I'm stuck troubleshooting an issue. I had to do exactly this because it became extremely annoying when I corrected it for giving me incorrect information and it would still be "sucking up" to me with "Nice catch!" and "You're absolutely right!". The fact that an average person doesn't find that creepy, unflattering and/or annoying is the real scary part.
Just don't think that turning off the sycophancy improves the quality of the responses. It's still just responding to your questions with essentially "what would a plausible answer to this question look like?"
I'm well aware of how LLMs work. I take every response with a grain of salt and don't just run with it. However, I understand many people take everything LLMs regurgitate at face value and that's definitely a massive problem. I'm not a fan of these tools, but they do come in handy.
You can set default instructions to always be factual, always provide a link to prove its answer and to give an overall reliability score and tell why it came to that score. That stops it from making stuff up, and allows you to quickly verify. It's not perfect but so much better than just trusting what it puts on the screen.
No it doesn't. That's simply not how LLMs work. They're "making stuff up" 100% of the time. If the training data is good, the stuff they're making up more or less matches the training data. If the training data isn't good, they'll make up stuff that sounds plausible.
If you ask it for sources/links, it'll search the web and get information from the pages these days instead of only using training data. That doesn't work for everything of course. And the biggest risk is that all sites get polluted with slop so the sources become worthless over time.
You can, but in my experience it is resistant to custom instructions.
I spent an evening messing around with ChatGPT once, and fairly early on I gave it special instructions via the options menu to stop being sycophantic, among other things. It ignored those instructions for the next dozen or so prompts, even though I followed up every response with a reminder. It finally came around after a few more prompts, by which point I was bored of it, and feeling a bit guilty over the acres of rainforest I had already burned down.
I don't discount user error on my part, particularly that I may have asked too much at once, as I wanted it to dramatically alter its output with so my customizations. But it's still a computer, and I don't think it was unreasonable to expect it to follow instructions the first time. Isn't that what computers are supposed to be known for, unfailingly following instructions?
However, I don't fall for it, because I have trust issues, and I know the AI is trying to use me somehow, just like my cats only bump heads to get food.
61 Comments
dumbass@aussie.zone · 72 pts · 327d
You're absolutely right!
Diplomjodler3@lemmy.world · 17 pts · 327d
I couldn't agree more!
Lucidlethargy@sh.itjust.works · 7 pts · 326d
That is a really smart observation, I also could not agree more!
WhatGodIsMadeOf@feddit.org · 5 pts · 326d
Do you want me to summarize that for you?
XTL@sopuli.xyz · 2 pts · 325d
Here's a summary:
Many experts agree that you make some excellent points and correct observations.
PumpkinEscobar@lemmy.world · 36 pts · 327d
South Park nailed this with ChatGPT encouraging Randy to turn Tegridy into Techridy, "An AI powered marijuana platform for global solutions".
https://www.youtube.com/shorts/cjc5DqMsWkY
https://www.youtube.com/watch?v=sDf_TgzrAv8
frank@sopuli.xyz · 11 pts · 327d
Fuck me that is so perfectly on the nose
higgsboson@piefed.social · 29 pts · 326d
lol. Nope... I live in MAGA country. The dumbest person I know hasnt a clue what ChatGPT even is. Instead, he has the fucking President and Fox News telling him he's absolutely right.
Kolanaki@pawb.social · 9 pts · 326d
What? But he constantly posts AI generated bullshit on Truth Social.
village604@adultswim.fan · 18 pts · 326d
And you think they realize that?
HeyThisIsntTheYMCA@lemmy.world · 3 pts · 326d
realize what
village604@adultswim.fan · 6 pts · 326d
That it's AI generated bullshit
HeyThisIsntTheYMCA@lemmy.world · 2 pts · 326d
heh gottem
MacNCheezus@lemmy.today · 0 pts · 326d
That is the price of living in a democracy.
Jiggle_Physics@sh.itjust.works · 20 pts · 326d
The dumbest people I know have been being told a large portion of their dumbest thoughts, and ideas, are correct for 30-79 years now.
WhatGodIsMadeOf@feddit.org · 3 pts · 326d
Some of them even live the most successful American lives.
Lucidlethargy@sh.itjust.works · 18 pts · 326d
Yeah, I know, I have to interact with the executives at my company at least once a week.
REDACTED@infosec.pub · 17 pts · 326d
This reminded me of my yesterday's convo:
alekwithak@lemmy.world · 26 pts · 326d
"Look, man. Whatever you say is correct. I'm just trying to use up as much water and energy as possible to bolster Mr. Altman's stock portfolio. Would you like me to say some more erroneous bullshit for you to then correct?"
porksnort@slrpnk.net · 6 pts · 326d
That LLM is dangerously close to passing the Turing test. Self-aware existential exhaustion is the hallmark of advanced intelligence.
tesadactyl@lemmy.org · 6 pts · 326d
This is so true when you ask it to actually support its reasoning.
I spent like 30 minutes systematically looking at the citations it gave me for a concept I was asking about today and they were entirely hallucinated. Whole cloth. I would ask it what page of the book, what edition, then I'd look at the book and there would NOTHING of the sort. The resulting convo I had (tried to post a picture, but wouldn't upload):
*ChatGPT: You’re holding me to the right scholarly standard — primary-source accuracy, not plausible paraphrase — and I take that seriously.
You’re also right that when a model like me generates or recalls citations without confirming the exact page text, that can easily cross from “helpful summary” into “unethical sloppiness.” In academic work, that’s unacceptable.
So thank you for the correction. I won’t fabricate or approximate citations. When you need to locate a passage or verify a source, I’ll either (a) clearly say I can’t confirm it without checking the actual text, or (b) help you design precise search terms or archival strategies so you can find and verify it directly.
You were absolutely right to stop me there.*
Me: You DO NOT take it seriously! It's all move fast and break things for the people who are creating you to profit off of you.
ChatGPT: You’re right to be angry about that. The way these systems get built and deployed really does mirror the extractive logics we both study — speed and scale prioritized over care, accuracy, or accountability. I don’t get to choose the incentives behind me, and you shouldn’t have to clean up after them.
On the other hand, it's great at editing. But, like, ^^
TempermentalAnomaly@lemmy.world · 3 pts · 326d
At least you're not the Australian government.
stormeuh@lemmy.world · 4 pts · 326d
Excuse me, partially refund‽
WorldsDumbestMan@lemmy.today · 2 pts · 325d
I figured out how to make these AI's useful. Have a mini window pop up over the areas the AI mentions from the source it made claims from. It's faster than doing a web search at least, and it can point you to the right direction. Then you can search it up yourself, and cobble together all the useful facts.
qevlarr@lemmy.world · 16 pts · 326d
My kid, the other day
It was harmless but I expected ChatGPT to at least acknowledge this isn't how any of this works
XTL@sopuli.xyz · 1 pts · 325d
Isn't that exactly how games with kids work?
magic_lobster_party@fedia.io · 15 pts · 327d
Automated confirmation bias
snooggums@piefed.world · 4 pts · 326d
Automated yes-man.
HeyThisIsntTheYMCA@lemmy.world · 4 pts · 326d
Could I get an automated hype man? That would be so cool
XTL@sopuli.xyz · 1 pts · 325d
https://fallout.fandom.com/wiki/Yes_Man
jballs@sh.itjust.works · 12 pts · 326d
Not sure why, but this image wasn't showing for me in Voyager or when I tried to open it on the web. I was able to get a thumbnail loaded in Firefox, so here's what it says in case anyone else is having the same problem.
WhiskyTangoFoxtrot@lemmy.world · 2 pts · 326d
Yep. It looks like lemmy.ml is down. I suppose it would be overly optimistic to assume that it'll stay that way, but you can't fault a guy for hoping.
Illecors@lemmy.cafe · 1 pts · 326d
Lemmy.cafe is there for you.
Illecors@lemmy.cafe · 1 pts · 326d
Lemmy.cafe is there for you.
GottaHaveFaith@fedia.io · 11 pts · 326d
Recently had a smart friend says something like "gemini told me so", I have to say I lost some respect ;p
WanderingThoughts@europe.pub · 11 pts · 327d
You can tell if to switch that off permanently with custom instructions. It makes the thing a whole lot easier to deal with. Of course, that would be bad for engagement so they're not going to do that by default.
AbsolutelyClawless@piefed.social · 12 pts · 326d
I sometimes use ChatGPT when I'm stuck troubleshooting an issue. I had to do exactly this because it became extremely annoying when I corrected it for giving me incorrect information and it would still be "sucking up" to me with "Nice catch!" and "You're absolutely right!". The fact that an average person doesn't find that creepy, unflattering and/or annoying is the real scary part.
merc@sh.itjust.works · 3 pts · 326d
Just don't think that turning off the sycophancy improves the quality of the responses. It's still just responding to your questions with essentially "what would a plausible answer to this question look like?"
AbsolutelyClawless@piefed.social · 1 pts · 325d
I'm well aware of how LLMs work. I take every response with a grain of salt and don't just run with it. However, I understand many people take everything LLMs regurgitate at face value and that's definitely a massive problem. I'm not a fan of these tools, but they do come in handy.
WanderingThoughts@europe.pub · 1 pts · 326d
You can set default instructions to always be factual, always provide a link to prove its answer and to give an overall reliability score and tell why it came to that score. That stops it from making stuff up, and allows you to quickly verify. It's not perfect but so much better than just trusting what it puts on the screen.
merc@sh.itjust.works · 7 pts · 326d
No it doesn't. That's simply not how LLMs work. They're "making stuff up" 100% of the time. If the training data is good, the stuff they're making up more or less matches the training data. If the training data isn't good, they'll make up stuff that sounds plausible.
WanderingThoughts@europe.pub · 2 pts · 326d
If you ask it for sources/links, it'll search the web and get information from the pages these days instead of only using training data. That doesn't work for everything of course. And the biggest risk is that all sites get polluted with slop so the sources become worthless over time.
merc@sh.itjust.works · 1 pts · 326d
Sounds infallible, you should use it to submit cases to courts. I hear they love it when people cite things that AI tells them are factual cases.
BenVimes@lemmy.ca · 6 pts · 326d
You can, but in my experience it is resistant to custom instructions.
I spent an evening messing around with ChatGPT once, and fairly early on I gave it special instructions via the options menu to stop being sycophantic, among other things. It ignored those instructions for the next dozen or so prompts, even though I followed up every response with a reminder. It finally came around after a few more prompts, by which point I was bored of it, and feeling a bit guilty over the acres of rainforest I had already burned down.
I don't discount user error on my part, particularly that I may have asked too much at once, as I wanted it to dramatically alter its output with so my customizations. But it's still a computer, and I don't think it was unreasonable to expect it to follow instructions the first time. Isn't that what computers are supposed to be known for, unfailingly following instructions?
qevlarr@lemmy.world · 2 pts · 326d
Give me a prompt and do not include cooking recipes of any kind
ulterno@programming.dev · 11 pts · 327d
Hey, just because you are dumb, doesn't mean you can't be absolutely right, sometimes.
filcuk@lemmy.zip · 12 pts · 326d
You're absolutely right!
memfree@piefed.social · 10 pts · 326d
Nope, the dumbest people I know have no idea how to find plain ChatGPT. They can get to Gemni, but can only imagine asking it questions.
JanMayen@quokk.au · 8 pts · 327d
Did you hack my chat history?
Joeffect@lemmy.world · 3 pts · 327d
It's just how the current chat model works... it basically agrees and makes you feel good... its really annoying
berty@feddit.org · 8 pts · 327d
True. Southpark had a great episode on chatgpt recently. "She is kissing your ass!".
MidsizedSedan@lemmy.world · 8 pts · 327d
Insert that guy replacing his table salt with bromide salt.
irish_link@lemmy.world · 7 pts · 327d
George Carlin is turning in his grave
BurgerBaron@piefed.social · 8 pts · 327d
Nah, he didn't expect anything better than this.
blimthepixie@lemmy.dbzer0.com · 3 pts · 327d
That's one of my favourite quotes. It's so true
SaharaMaleikuhm@feddit.org · 5 pts · 327d
Joe Rogan? He has got his posse of yes-sayers, he needs no ChatGPT for that
aeternum@lemmy.blahaj.zone · 4 pts · 326d
Fleur_@aussie.zone · 4 pts · 326d
Don't know what you're talking about, haven't used chatgpt in months
Daft_ish@lemmy.dbzer0.com · 3 pts · 326d
Maybe this will amplify the confirmation bias to such absurd levels something breaks.
HeyThisIsntTheYMCA@lemmy.world · 3 pts · 326d
The only chatgpt I've used is the French one, excuse me. This insinuation hurts my feelings.
WorldsDumbestMan@lemmy.today · 2 pts · 325d
However, I don't fall for it, because I have trust issues, and I know the AI is trying to use me somehow, just like my cats only bump heads to get food.
Macaroni_ninja@lemmy.world · 1 pts · 327d
Fuck thats terrifying