Chatgpt just making stuff up

First time? This is indeed how LLMs work.

Whenever someone confidently states "I asked ChatGPT..." in a conversation, I die a little inside. I'm tired of explaining this shit to people.

All LLMs act like improv artists, they almost never stop riffing because they always say “yes and”

LLMs can't say they don't know. It's better for the business to make up some bullshit than just say "I don't know" because it would show how useless they can be.

I've had LLMbeciles make up an entire discography, track list, and even lyrics of "obscure black metal bands" that don't exist. It doesn't take much to have them start to spew non-stop grammatically correct gibberish.

I've also had them make up lyrics for bands and songs that actually exist. Specifically completely made-up lyrics for the song "One Chord Wonders" by The Adverts. And then, when I quote the actual lyrics to correct them, incorporate that into their never-ending hallucinations by claiming that was a special release for a television special, but that the album had their version.

Despite their version and the real version having entirely different scansion.

These things really are just hallucination machines.

Someone here said that LLM chatbots are always "hallucinating" and it stuck with me. They happen to be correct a lot of the time but they are always making stuff up. That's what they do that's how they work.

Your specific wording is telling it to make up an answer.

What "would" this word mean? Implying it doesn't mean anything currently, so guess a meaning for it.

But yes, in general always assume they don't know what they are saying, as they aren't really capable of knowing. They do a really good job of mimicking knowledge, but they don't actually know.

Because AI is a predictive transformer/generator, not an infinite knowledge machine.

you just asked it to imagine what the inexistant word would mean, than complained that it did it's job?

lmao

like, i thought this community is for people sharing the hate for cheap corpo hype over ai, not trying to hype up the hate for the otherwise useful instrument. You're swaying from one extreme to anothar.

I'm no AI proponent, but phrasing is important. Would should be replaced with does. Would implies a request for speculation, specifically, or even actively creative output.

As in, if it existed, what would...

Yes. One of the original instances of this is to make up a saying and ask it what it means. Like "you can't shave a cat until it has had its dinner." It'll make up what it means.

I have to give it props for dropping in "dissorophoid temnospondyl" which I figured even odds on also being made up, but it is not!

https://en.m.wikipedia.org/wiki/Dissorophoidea

For a while I was thinking I might eventually use AI for more than a code completer. But that looks less likely every day.

Actually its correct. If you're at all familiar with the Permian of New Mexico you would surely have heard of Batrapeton. Maybe read more?

It’s cause you said “would” not “does”

ChatGPT learns from your previous threads.

If you're using ChatGPT for your writing, it probably used that as information to answer the question.

After asking it a similar question, it answered in a similar way.

When asking for sources, it spit out information about a name that's very similar, which it seems to have used too to describe the fictional species.

When pressed a little more, it even linked this very post.