tbf they are getting significantly better, one of the best things that hasn't really filtered through to the mainstream is MOE / mixture of experts
the tldr is that back in the chatgpt4 days, wayyy back in the ye olden times of 2024, ai would essentially go through the entire library every single question to find an answer
Now the libraries are getting massive but the queries are getting faster at responding because instead of going through the entire library for every question, they only need a part of it, just like in a library instead of querying all of human knowledge for what is 8x12, it just goes to the maths section saving a lot of power and time
In the case of chat.mistral.ai it doesn't even go through the library to maths, it just makes a quick python script and outputs the answer that way:
I imagine we might wrangle the hallucination thing (or at least be more verbose about it's uncertainty), but I doubt it will ever identify a poorly chosen question.
Making the LLMs warn you when you ask a known bad question is just a matter of training it differently. It's a perfectly doable thing, with a known solution.
That's because it's a false premise. LLMs don't hallucinate, they do exactly what they're meant to do; predict text, and output something that's legible and human written. There's no training for correctness, how do you even define that?
There’s no training for correctness, how do you even define that?
I guess can chat to these guys who are trying:
By scaling reasoning with reinforcement learning that rewards correct final answers, LLMs have improved from poor performance to saturating quantitative reasoning competitions like AIME and HMMT in one year
Sure, when it comes to mathematics you can do that with extreme limitations on success, but what about cases where correctness is less set? Two opposing statements can be correct if a situation changes, for example.
The problems language models are expected to solve go beyond the scope of what language models are good for. They’ll never be good at solving such problems.
i duno you’re in the wrong forum, you want hackernews or reddit, no one here knows much about ai
although you do seem to be making the same mistake others made before where you want point to research happening currently and then extrapolating that out to the future
ai has progressed so fast i wouldn’t be making any “they’ll never be good at” type statements
It's shit a lot of people don't think about tbh. Like imagine all the meaningless shit people have done on the internet, and that shit literally travels across the whole world using impressive feats of technology. I'm saving this post for later. 🤣
Infrastructure is all about unbelievable feats of engineering that are taken for granted. Sewage systems, running water, electricity, roads, public transport, cars, physical mail, and grocery stores/supermarkets are all unbelievable achievements that we all take for granted to varying degrees, and that's just off the top of my head. IP networking is just more of that. Absolutely crazy, and by design we don't think about it.
But AI (also depicted in this gif) is not in the same category IMO, for a lot of reasons.
Simply doing a traceroute to a website in another country a long time ago fascinated me. Seeing it hit all of the routers in other cities then across the ocean to another continent and back in less than 100ms blew my mind.
Led me down that path and now I’ve been a network engineer for over 10 years.
You might not like AI for what it stands or for the negative impact it has on the world, but you can't deny that LLM like we have today are a marvel of technology, an incredibly complex technology that would have felt science fiction just a decade ago.
Yeah, but it's not good infrastructure. It's not sustainable, it's privately controlled, and it's destined to be enshittified. Infrastructure needs to be well thought out and publicly regulated, AI is the opposite.
I have my local LLM rig (powered by solar) for asking stupid questions because I feel it's unreasonable to ask a data centre somewhere why spoons taste funny
31 Comments
Una@europe.pub · 43 pts · 260d
Just take a div and put it in the center
Eyekaytee@aussie.zone · 27 pts · 260d
Of course!
Goodman@discuss.tchncs.de · 37 pts · 260d
Idk why I like this, high quality healthy shit post.
chiliedogg@lemmy.world · 28 pts · 260d
I think doing this exact post but changing the request could really speak to the inefficiency of AI.
Something like "What is 8x12", go through the whole sequence, and have it spit out "Eight times twelve is 114"
Eyekaytee@aussie.zone · 15 pts · 260d
tbf they are getting significantly better, one of the best things that hasn't really filtered through to the mainstream is MOE / mixture of experts
the tldr is that back in the chatgpt4 days, wayyy back in the ye olden times of 2024, ai would essentially go through the entire library every single question to find an answer
Now the libraries are getting massive but the queries are getting faster at responding because instead of going through the entire library for every question, they only need a part of it, just like in a library instead of querying all of human knowledge for what is 8x12, it just goes to the maths section saving a lot of power and time
In the case of chat.mistral.ai it doesn't even go through the library to maths, it just makes a quick python script and outputs the answer that way:
marcos@lemmy.world · 23 pts · 260d
Btw, the correct answer is "use flexbox".
TeddE@lemmy.world · 3 pts · 259d
Just as a tangent:
This is one reason why I'll never trust AI.
I imagine we might wrangle the hallucination thing (or at least be more verbose about it's uncertainty), but I doubt it will ever identify a poorly chosen question.
marcos@lemmy.world · 3 pts · 259d
Making the LLMs warn you when you ask a known bad question is just a matter of training it differently. It's a perfectly doable thing, with a known solution.
Solving the hallucinations in LLMs is impossible.
Dojan@pawb.social · 3 pts · 259d
That's because it's a false premise. LLMs don't hallucinate, they do exactly what they're meant to do; predict text, and output something that's legible and human written. There's no training for correctness, how do you even define that?
Eyekaytee@aussie.zone · 1 pts · 259d
I guess can chat to these guys who are trying:
https://huggingface.co/deepseek-ai/DeepSeek-Math-V2
Dojan@pawb.social · 1 pts · 259d
Sure, when it comes to mathematics you can do that with extreme limitations on success, but what about cases where correctness is less set? Two opposing statements can be correct if a situation changes, for example.
The problems language models are expected to solve go beyond the scope of what language models are good for. They’ll never be good at solving such problems.
Eyekaytee@aussie.zone · 1 pts · 259d
i duno you’re in the wrong forum, you want hackernews or reddit, no one here knows much about ai
although you do seem to be making the same mistake others made before where you want point to research happening currently and then extrapolating that out to the future
ai has progressed so fast i wouldn’t be making any “they’ll never be good at” type statements
the_real_monte@sh.itjust.works · 3 pts · 260d
You could also use margin: 0 auto;
marcos@lemmy.world · 9 pts · 260d
Where it works, yes. If you know where it works, it won't be a problem for you.
serpineslair@lemmy.world · 21 pts · 260d
It's shit a lot of people don't think about tbh. Like imagine all the meaningless shit people have done on the internet, and that shit literally travels across the whole world using impressive feats of technology. I'm saving this post for later. 🤣
NeatNit@discuss.tchncs.de · 11 pts · 260d
Infrastructure is all about unbelievable feats of engineering that are taken for granted. Sewage systems, running water, electricity, roads, public transport, cars, physical mail, and grocery stores/supermarkets are all unbelievable achievements that we all take for granted to varying degrees, and that's just off the top of my head. IP networking is just more of that. Absolutely crazy, and by design we don't think about it.
But AI (also depicted in this gif) is not in the same category IMO, for a lot of reasons.
Fergie434@lemmy.world · 6 pts · 260d
Simply doing a traceroute to a website in another country a long time ago fascinated me. Seeing it hit all of the routers in other cities then across the ocean to another continent and back in less than 100ms blew my mind.
Led me down that path and now I’ve been a network engineer for over 10 years.
bobo1900@startrek.website · 1 pts · 259d
You might not like AI for what it stands or for the negative impact it has on the world, but you can't deny that LLM like we have today are a marvel of technology, an incredibly complex technology that would have felt science fiction just a decade ago.
NeatNit@discuss.tchncs.de · 2 pts · 259d
Yeah, but it's not good infrastructure. It's not sustainable, it's privately controlled, and it's destined to be enshittified. Infrastructure needs to be well thought out and publicly regulated, AI is the opposite.
fubarx@lemmy.world · 14 pts · 260d
REDACTED@infosec.pub · 1 pts · 259d
petey@aussie.zone · 7 pts · 259d
I have my local LLM rig (powered by solar) for asking stupid questions because I feel it's unreasonable to ask a data centre somewhere why spoons taste funny
Katana314@lemmy.world · 6 pts · 259d
Whoa, I was weirdly into My Little Pony for a while but I didn’t realize it powered data centers.
starelfsc2@sh.itjust.works · 1 pts · 259d
Lol
TemplaerDude@sh.itjust.works · 5 pts · 260d
I need that keyboard
orenj@lemmy.sdf.org · 4 pts · 259d
I have that keyboard, its a magegee typewriter style.
TemplaerDude@sh.itjust.works · 1 pts · 259d
Heck yes thank you
dual_sport_dork@lemmy.world · 5 pts · 260d
Uh-huh. Now ask it how to center a div vertically.
f314@lemmy.world · 5 pts · 260d
The real answer is of course, as in most cases, “it depends.”
danc4498@lemmy.world · 5 pts · 260d
sundray@lemmus.org · 4 pts · 259d
I suppose this is preferable to human intelligence, where the answer would come back as "Why would you want to do that?"