and the former kings of tuning the algorithms to favor seo that bubbled useful info up harder have thrown it all out in the name of impressing everyone with how space age and sci fi their tech is. it's not about advancing science or even pushing a useful product. it's strictly a tool for scams. is it a surprise that scammers are gaming the google scam better than anyone else? not really. they've always had a step up compared to the average internet denizen thanks to practice. this is why i get so frustrated when people dismiss ai skepticism as being a product of luddites.
you're getting scammed to think ai will benefit you
systems built by scammers will always benefit scammers
the luddites were right. scientific advancements should benefit the workers, not the rich
I mean, this isn't specifically an AI issue, this is scammers updating the info in Google business listings because the airlines don't actually care to maintain those pages (and Google doesn't want actual humans doing any work to make sure their shit is accurate). This has been going on before AI, AI is just following the garbage in, garbage out model that everyone said was going to be the result of this push.
This is why "AI" should be avoided at all cost. It's all bullshit. Any tool that "hallucinates" - I. E. Is error strewn - is not fit for purpose. Gaming the AI is just the latest example of the crap being spewed by these systems.
The underlying technology has its uses but its niche and focused applications, nowhere near as capable or as ready as the hype.
We don't use Wikipedia as a primary source because it has to be fact checked. AI isn't anywhere as near accurate as Wikipedia.so why use it?
Sometimes BS is exactly what I need! Like, hallucinated brainstorm suggestions can work for some workflows and be safe when one is careful to discard or correct them. Copying a comment I made a week ago:
I don’t love it for summarization. If I read a summary, my takeaway may be inaccurate.
Brainstorming is incredible. And revision suggestions. And drafting tedious responses, reformatting, parsing.
In all cases, nothing gets attributed to me unless I read every word and am in a position to verify the output. And I internalize nothing directly, besides philosophy or something. Sure can be an amazing starting point especially compared to a blank page.
Gotta tell you, you made a fairly extreme pronouncement against a very general term / idea with this:
"AI" should be avoided at all cost
Do you realize how ridiculous this sounds? It sounds, to me, like this - "Vague idea I poorly understand ('AI') should be 'avoided' (???) with disregard for any negative consequences, without considering them at all"
Cool take you've got?
Edit to add: whoops! Just realized the community I'm in. Carry on, didn't mean to come to the precise wrong place to make this argument lol.
Listen, I know that the term "AI" has been, historically, used to describe so many things to the point of having no meaning, but I think, given the context, it is pretty obvious what AI they are referring to.
Well, fair enough, folks seem to agree with you and that commenter. I'm not being deliberately uncharitable, "avoid AI at all costs" seems both poorly defined and hyperbolic to me, even given the context. Scams and inaccuracy are a problem in lots of situations, Google search results have been getting increasingly bad to the point of unusable for a while now (I'd argue long before LLM saturation), and I've personally been getting mileage with some LLMs, already at kind of an early stage, over wading through every crappy search result.
I wouldn't call myself an enthusiast or on the hype train, I work in the industry. But it's clearly useful, while clearly having many tradeoffs (energy use maybe much worse than inaccuracy / scam potential), and "avoid at all cost" is silly to me. But cheers, happy to simply disagree!
Ironically, that is possibly one of the few legit uses.
Doctors can't learn about every obscure condition and illness. This means they can miss the symptoms of them for a long time. An AI that can check for potential matches to the symptoms involved could be extremely useful.
The provisio is that it is NOT a replacement for a doctor. It's a supplement that they can be trained to make efficient use of.
That requires the symptoms to be entered correctly, and significant effort from (already overworked) doctors. A fuzzy logic system that can process standard medical notes, as well as medical research papers would be far more useful.
Basically, a quick click, and the paperwork is scanned. If it's a match for the "bongo dancing virus" or something else obscure, it can flag it up. The doctor can now invest some effort into looking up "bongo dancing virus" to see if it's a viable match.
It could also do it's own pattern matching. E.g. if a particular set of symptoms is often followed 18-24 hours later by a sudden cardiac arrest. Flagging this up could be completely false. However, it could key doctors in on something more serious happening, before it gets critical.
An 80% false positive is still quite useful, so long as the 20% helps and the rest is easy for a human to filter.
The key is considering who is going to be using these systems. Certainly Google search AI is never going to be useful in this way because the kind of info a patient needs is very different to what a doctor would find useful.
And if we do make systems for doctors, then it's pretty damn important that we consider things like you have, taking into account that doctors are already overwhelmed and spending way too much effort juggling medical notes. I read a thing a while back which highlighted how many doctors are struggling with information management and processing all the info they need to because of how IT systems have tended to be enforced on them from the top down, with some doctors even saying paper notes were far easier to deal with (especially for complex cases). Digitisation definitely has huge benefits, but it seems like the needs of doctors have been largely ignored.
Even besides doctors, I feel like the field of Human-Computer Interaction (HCI) has been way too focussed on ways of wringing out more money from people, with not enough focus put on how we can make technology that empowers people. It's no wonder why: If I were a HCI researcher, I know what kind of project would be more likely to get research funding, and it's the ruthlessly capitalistic ones.
"An 80% false positive is still quite useful, so long as the 20% helps and the rest is easy for a human to filter."
This gets at a key point, in my opinion — even when one ignores the straightforwardly scammy "AI" nonsense, a lot that remain are still overly focussed on building systems that do stuff for people (usually in a way that would eliminate or reduce people in the process. Many examples of this exist, but one is "AI teachers" which still requires a human in the room, but only as a "learning facilitator" or some nonsense). I work in a field where machine learning has been a prominent thing for years, so I'm in a weird place of being sick of hearing about AI, and also impressed by what we do have. Mainly though, I'm exasperated because we could be doing so much more with the tech we have if we made tools that were intended to be used by humans.
Humans are dumb and emotional and silly, but we are also pretty cool and we can make awesome things when given the opportunity to. I will always be cynical about tech that seems over keen to cut humans out of things
Yet again tech companies are here to ruin the day. LLM are such a neat little language processing tool. It's amazing for reverse looking up definitions (where you know the concept but can't remember some dumb name) or when looking for starting points or want to process your ideas and get additional things to look at, but most definitely not a finished product of any kind. Fuck tech companies for selling it as a search engine replacement!
Wait, are you advocating people blindly trust unreliable sources and then get angry at the unreliable source when it turns out to be unreliable rather than learn from shit like this to avoid becoming a victim?
Ok, I agree that Google isn't a good guy in this situation, but that doesn't mean advice to not just trust what Google says is invalid. It also doesn't absolve Google of their accidental or deliberate inaccuracies.
It was just a "In case you didn't know, don't just trust Google even though they've worked so hard at building a reputation of being trustworthy and even seemed pretty trustworthy in the past. Get a phone number from the company's website."
And then I'll add on: regardless of where you got the phone number from, be skeptical if someone asks you for your banking information or other personal information that isn't usually involved in such a service. Not because you'll be the bad guy if you do get scammed, but to avoid going through it because it's at least going to be a pain in the ass to deal with, if not a financially horrible situation to go through if you are unable to get it reversed.
I don't see any blaming of anyone in the original comment you replied to but just general advice to avoid falling for a scam like this. There isn't even a victim in this case because the asking for banking info tipped them off if I'm understanding the OP correctly.
So I'm confused about what specifically you are objecting to in the original comment and if it is the general idea that you shouldn't blindly trust results given by Google's LLM, which isn't known for its reliability.
For me it's the idea of focusing at all on telling people not to trust LLMs as opposed to criticizing companies for putting them prominently on the top of the page.
Because the average person doesn't even know what an LLM is or what it even stands for and putting a misinformation generator at the top of search pages is irresponsible.
Like, if something is so unreliable with information that you have to say "don't trust what this thing says" but you still put it at the top of the page? Come on... It's like putting a self destruct button in a car and telling people "well the label says not to push it!"
Remember when 4chan got people to microwave their phones because they got them to believe it would charge it?
If calling those people stupid is victim blaming then so be it. I’m blaming the victim.
This case isn’t as clear as that but even before the AI mania the instant answer at the top of Google results was frequently incorrect. Being able to discern BS from real results has always been necessary and AI doesn’t change that.
I’ve been using Kagi this year and it keeps LLM results out of the way unless I want them. When you open their AI assistant it says
Assistant can make mistakes. Think for yourself when using it.
Would that make Google liable? I mean that wouldn't be a case of users posting information that would be a case of Google posting information in that case wouldn't it? So it seems to me they'd be legally liable at that point.
I think there's a disclaimer with all AI summaries. Although, I just tried googling United airlines and there is no longer an AI summary, only a message directly from their website.
Honestly I wanted to write a snug comment about "But but it even says AI can sometimes make mistakes!", but after clicking through multiple links and disclaimers I can't find Google actually admitting that
They gave up working search with algorithms that are easier to reason about and correct for with a messy neural network that is broken in so many ways and basically impossible to generally correct while retaining its core characteristics. A change with this many regressions should've never been pushed to production.
Google has been sponsoring scammers as first search results since its creation. Google has caused hundreds of millions of dollars in losses to people, and need to be sued for it.
This has been an issue since long before LLMs. Before the AI summary box, scammers used targeted ads to place ahead of the actual company you were searching for.
Yeah, but that was easy to spot, both by people and by Google. There were at least some guardrails, imperfect, not ideal, but they existed. With llm there is basically none
I get paranoid enough about making sure I'm clicking the correct search result and not some scam. I hope I would avoid any AI answers but yeah, to many people it could be confusing.
One could argue there is a new aspect. When it comes to retraining the public on what to trust, there’s a likely blind spot where a person may know to only call a number listed on a trusted website. So they'll check to see if they're on a bank domain before picking up the phone. If Google, being a big name, presents the number in an official looking way at the top of their pages, it may pass the sniff test. And get people into trouble.
Featured snippets would prominently display source URLs:
Featured snippets would prominently display source URLs:
That's meaningless with how easy it is to register legit looking faux domains and how it is even easier to create legit looking sub domains. People who fall for those type of scams will likely not even understand what a domain is.
At the top, but that isn't what this post is saying. This is saying that Google's AI gave the scammer answer. Not that they provided a link you could click on, but that Google itself said this is the number.
It's not an AI, it's just word prediction, which also just follows stupid algorithms, just like those who determine search results. Both can be tricked / manipulated if you understand how they work. It's still the same principle for both cases.
Regardless of what they call it, they're the ones presenting it. I'm not arguing they can't be tricked. I'm arguing they are fundamentally different concepts. One is offering you a choice of sources, the other is making a claim. That's a pretty big distinction in a whole mess is different ways. Not the least of which is legal.
I'm sorry but no. It's not Google making that claim, it's just the LLM replying in a confident way because that's how they are expected to work. As I said, word prediction. You can install the tiniest / most dumbest model on your local PC too and ask the same question. It will give you some random hallucinated number and act like that's what you're looking for due to its default system prompt telling it to sound like an AI assistant. In the case of search engines the LLM is directly hooked into the search engine itself and just does the same thing you'd do and search for a hopefully fitting search result. So scammers playing those search algorithms to get a good spot will end up becoming the recommendation for the LLM to tell the user. It's the same thing, just displayed slightly differently. All the cool AI assistant stuff they try to present this as, is just an illusion, a word based roleplay. The only benefit here is that they can somewhat understand abstract questions, which is helpful for certain search queries, but in the end it is always the user's responsibility to check the actual search result.
Having worked with AI and AI products in my last job before I was let go I can say this:
Out of the box AI is very good at the following:
Mundane very simple binary/boolean tasks. Is this a yes/no. Can I find a piece of information that I was told is here based on your statement? Etc
Condensing very complex processes into very simplistic things - NOTE you will lose a lot of information based on this action unless you refine a statement.
Making overarching summaries - kinda similar to 2 but also its own thing, think more creating a summary of a book.
Programmed AI - read machine learning, because you are still telling it how to interpret things - can be good at (depending how good you are at telling it what it should do):
Interpreting meaning in a statement.
Understanding if - then constructs.
Deducing plausible outcomes.
ALL AI struggles at:
Interpreting real vs fake (thats why you literally teach it how to understand what a spot light is with your captcha)
Understanding complexity in speech and tonal differences - I am SO happy to be here /s
Thinking on its own - using collected data to make an inference that it was not directly programmed to understand
The big craze over AI totally was misunderstood. AI is best to be thought of as Automated Intelligence and the word Artificial at its current state is a complete misnomer.
This is just one example of people having been mislead by the name to not fully understand what is up with AI.
All I'm saying is that an LLM is only decent at generating text. Everything else it sucks at. So it isn't logical to use it as a search engine. Google is also to blame for that. They are slapping AI to whatever they have running. Like most tech companies these days.
Compare that to Perplexity, which is a search engine first and only uses an LLM to make a summary of websites that are riddled with SEO. And then add a link so you can check for yourself if the LLM is hallucinating or not.
Don't be like a Techbro conflating LLMs and General AI. They are different things. And the sooner all users understand this the better.
In general, it is much faster and much cheaper to cross an ocean by plane than by boat.
Do you think if someone lives in Brazil but their family lives in Indonesia, they should drive across South America, charter a boat and then take it across the Pacific rather than just buy a plane ticket?
Up until now, this has not been about what you personally choose. In fact, it has to do with what the person's wife in the image I pasted chose.
Maybe she's going from Brazil to Indonesia to visit her dying father and she doesn't have six weeks to get there by boat, or the (according to my searching) approximately $60,000 it would cost to charter that boat.
81 Comments
skillissuer@discuss.tchncs.de · 117 pts · 1y
it's seo games all over again
Quill7513@slrpnk.net · 43 pts · 1y
and the former kings of tuning the algorithms to favor seo that bubbled useful info up harder have thrown it all out in the name of impressing everyone with how space age and sci fi their tech is. it's not about advancing science or even pushing a useful product. it's strictly a tool for scams. is it a surprise that scammers are gaming the google scam better than anyone else? not really. they've always had a step up compared to the average internet denizen thanks to practice. this is why i get so frustrated when people dismiss ai skepticism as being a product of luddites.
jonne@infosec.pub · 23 pts · 1y
I mean, this isn't specifically an AI issue, this is scammers updating the info in Google business listings because the airlines don't actually care to maintain those pages (and Google doesn't want actual humans doing any work to make sure their shit is accurate). This has been going on before AI, AI is just following the garbage in, garbage out model that everyone said was going to be the result of this push.
orcrist@lemm.ee · 15 pts · 1y
spankmonkey@lemmy.world · 18 pts · 1y
A-I-S-E-O
Tippon@lemmy.dbzer0.com · 10 pts · 1y
🎶 Old McDonald had a server farm... 🎶
neanderthal@lemmy.world · 9 pts · 1y
And Gem'ni was his nam-i....g-e-m-n-i
(I know it is Gemini, but we need 2 syllables and 5 letters to fit the song parady, so I made a contraction)
brbposting@sh.itjust.works · 8 pts · 1y
Artistic license: validated
BananaTrifleViolin@lemmy.world · 87 pts · 1y
This is why "AI" should be avoided at all cost. It's all bullshit. Any tool that "hallucinates" - I. E. Is error strewn - is not fit for purpose. Gaming the AI is just the latest example of the crap being spewed by these systems.
The underlying technology has its uses but its niche and focused applications, nowhere near as capable or as ready as the hype.
We don't use Wikipedia as a primary source because it has to be fact checked. AI isn't anywhere as near accurate as Wikipedia.so why use it?
brbposting@sh.itjust.works · 18 pts · 1y
Yes indeed agreed.
Sometimes BS is exactly what I need! Like, hallucinated brainstorm suggestions can work for some workflows and be safe when one is careful to discard or correct them. Copying a comment I made a week ago:
Tartas1995@discuss.tchncs.de · 4 pts · 1y
https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actually-does-nothing-of-the-kind/
A good read about ai summarizing.
brbposting@sh.itjust.works · 1 pts · 1y
Super interesting, I have to get into all those links. Thank you!
notTheCat@lemmy.ml · 13 pts · 1y
Benjaben@lemmy.world · -10 pts · 1y
Gotta tell you, you made a fairly extreme pronouncement against a very general term / idea with this:
Do you realize how ridiculous this sounds? It sounds, to me, like this - "Vague idea I poorly understand ('AI') should be 'avoided' (???) with disregard for any negative consequences, without considering them at all"
Cool take you've got?
Edit to add: whoops! Just realized the community I'm in. Carry on, didn't mean to come to the precise wrong place to make this argument lol.
prototype_g2@lemmy.ml · 2 pts · 1y
Listen, I know that the term "AI" has been, historically, used to describe so many things to the point of having no meaning, but I think, given the context, it is pretty obvious what AI they are referring to.
Benjaben@lemmy.world · 2 pts · 1y
Well, fair enough, folks seem to agree with you and that commenter. I'm not being deliberately uncharitable, "avoid AI at all costs" seems both poorly defined and hyperbolic to me, even given the context. Scams and inaccuracy are a problem in lots of situations, Google search results have been getting increasingly bad to the point of unusable for a while now (I'd argue long before LLM saturation), and I've personally been getting mileage with some LLMs, already at kind of an early stage, over wading through every crappy search result.
I wouldn't call myself an enthusiast or on the hype train, I work in the industry. But it's clearly useful, while clearly having many tradeoffs (energy use maybe much worse than inaccuracy / scam potential), and "avoid at all cost" is silly to me. But cheers, happy to simply disagree!
perviouslyiner@lemmy.world · 56 pts · 1y
Wait until you hear about the AI's programming abilities!
It "knows" that a Python program starts with some lines like: from (meaningless package name) include *
If you can register the package name it invents, your code could be running on some of the world's biggest companies' internal servers!
stoy@lemmy.zip · 46 pts · 1y
AI was launched with the promise of taking away the boring parts and letting us focus on the fun stuff.
In reality it takes away the fun stuff and gives us more boring things to do.
Tartas1995@discuss.tchncs.de · 16 pts · 1y
Being scammed isn't boring. It is blood boiling and (wrongly) shame-filled.
But yeah, you are right. The boring and the bad.
stoy@lemmy.zip · 4 pts · 1y
Oh I agree with you there, I was talking about AI as a concept and how it was sold to us.
Tartas1995@discuss.tchncs.de · 3 pts · 1y
I understood and I agreed with you. I just wanted to point out that it is actually even worse...
But you are right
Suavevillain@lemmy.world · 31 pts · 1y
AI results are always so bad. I don't like that there is AI medical results. That needs more pushback.
cynar@lemmy.world · 13 pts · 1y
Ironically, that is possibly one of the few legit uses.
Doctors can't learn about every obscure condition and illness. This means they can miss the symptoms of them for a long time. An AI that can check for potential matches to the symptoms involved could be extremely useful.
The provisio is that it is NOT a replacement for a doctor. It's a supplement that they can be trained to make efficient use of.
DaPorkchop_@lemmy.ml · 6 pts · 1y
Couldn't that just as easily be solved with a database of illnesses which can be filtered by symptoms?
cynar@lemmy.world · 7 pts · 1y
That requires the symptoms to be entered correctly, and significant effort from (already overworked) doctors. A fuzzy logic system that can process standard medical notes, as well as medical research papers would be far more useful.
Basically, a quick click, and the paperwork is scanned. If it's a match for the "bongo dancing virus" or something else obscure, it can flag it up. The doctor can now invest some effort into looking up "bongo dancing virus" to see if it's a viable match.
It could also do it's own pattern matching. E.g. if a particular set of symptoms is often followed 18-24 hours later by a sudden cardiac arrest. Flagging this up could be completely false. However, it could key doctors in on something more serious happening, before it gets critical.
An 80% false positive is still quite useful, so long as the 20% helps and the rest is easy for a human to filter.
AnarchistArtificer@slrpnk.net · 3 pts · 1y
The key is considering who is going to be using these systems. Certainly Google search AI is never going to be useful in this way because the kind of info a patient needs is very different to what a doctor would find useful.
And if we do make systems for doctors, then it's pretty damn important that we consider things like you have, taking into account that doctors are already overwhelmed and spending way too much effort juggling medical notes. I read a thing a while back which highlighted how many doctors are struggling with information management and processing all the info they need to because of how IT systems have tended to be enforced on them from the top down, with some doctors even saying paper notes were far easier to deal with (especially for complex cases). Digitisation definitely has huge benefits, but it seems like the needs of doctors have been largely ignored.
Even besides doctors, I feel like the field of Human-Computer Interaction (HCI) has been way too focussed on ways of wringing out more money from people, with not enough focus put on how we can make technology that empowers people. It's no wonder why: If I were a HCI researcher, I know what kind of project would be more likely to get research funding, and it's the ruthlessly capitalistic ones.
This gets at a key point, in my opinion — even when one ignores the straightforwardly scammy "AI" nonsense, a lot that remain are still overly focussed on building systems that do stuff for people (usually in a way that would eliminate or reduce people in the process. Many examples of this exist, but one is "AI teachers" which still requires a human in the room, but only as a "learning facilitator" or some nonsense). I work in a field where machine learning has been a prominent thing for years, so I'm in a weird place of being sick of hearing about AI, and also impressed by what we do have. Mainly though, I'm exasperated because we could be doing so much more with the tech we have if we made tools that were intended to be used by humans.
Humans are dumb and emotional and silly, but we are also pretty cool and we can make awesome things when given the opportunity to. I will always be cynical about tech that seems over keen to cut humans out of things
Duamerthrax@lemmy.world · 4 pts · 1y
In either case, a real doctor would be reviewing the results. Nobody is going to authorize surgeries or prescription meds from AI alone.
Blaster_M@lemmy.world · 30 pts · 1y
and that's why I always go straight to the company website to find that info instead of googling it
Tartas1995@discuss.tchncs.de · 10 pts · 1y
I google for the company website (e.g. Wikipedia) and then I google with the site in mind for info. Works well
LANIK2000@lemmy.world · 26 pts · 1y
Yet again tech companies are here to ruin the day. LLM are such a neat little language processing tool. It's amazing for reverse looking up definitions (where you know the concept but can't remember some dumb name) or when looking for starting points or want to process your ideas and get additional things to look at, but most definitely not a finished product of any kind. Fuck tech companies for selling it as a search engine replacement!
jj4211@lemmy.world · 18 pts · 1y
It is great at search. See this awesome example I hit just today from Google's AI overview:
See, without AI I would have thought housing prices went up between 2007, and that $391,900 was a bigger number than $257,400.
Etterra@lemmy.world · 18 pts · 1y
That's why you always get it from their website. Never trust a LLM to do a search engine's job.
JackbyDev@programming.dev · -7 pts · 1y
Respectfully, this is victim blaming. Criticize Google, not end users.
Buddahriffic@lemmy.world · 10 pts · 1y
Wait, are you advocating people blindly trust unreliable sources and then get angry at the unreliable source when it turns out to be unreliable rather than learn from shit like this to avoid becoming a victim?
A7thStone@lemmy.world · 6 pts · 1y
Google has spent a fortune to convince people they are a reliable source. This is clearly on google, not the people who aren't tech savvy.
Buddahriffic@lemmy.world · 2 pts · 1y
Ok, I agree that Google isn't a good guy in this situation, but that doesn't mean advice to not just trust what Google says is invalid. It also doesn't absolve Google of their accidental or deliberate inaccuracies.
It was just a "In case you didn't know, don't just trust Google even though they've worked so hard at building a reputation of being trustworthy and even seemed pretty trustworthy in the past. Get a phone number from the company's website."
And then I'll add on: regardless of where you got the phone number from, be skeptical if someone asks you for your banking information or other personal information that isn't usually involved in such a service. Not because you'll be the bad guy if you do get scammed, but to avoid going through it because it's at least going to be a pain in the ass to deal with, if not a financially horrible situation to go through if you are unable to get it reversed.
JackbyDev@programming.dev · -2 pts · 1y
Where did I say this? I didn't say this. You said I said this.
Buddahriffic@lemmy.world · 5 pts · 1y
I don't see any blaming of anyone in the original comment you replied to but just general advice to avoid falling for a scam like this. There isn't even a victim in this case because the asking for banking info tipped them off if I'm understanding the OP correctly.
So I'm confused about what specifically you are objecting to in the original comment and if it is the general idea that you shouldn't blindly trust results given by Google's LLM, which isn't known for its reliability.
JackbyDev@programming.dev · -2 pts · 1y
For me it's the idea of focusing at all on telling people not to trust LLMs as opposed to criticizing companies for putting them prominently on the top of the page.
Buddahriffic@lemmy.world · 2 pts · 1y
Why not both? Plus, not just trusting LLMs is something any of us can decide to do on our own.
JackbyDev@programming.dev · 1 pts · 1y
Because the average person doesn't even know what an LLM is or what it even stands for and putting a misinformation generator at the top of search pages is irresponsible.
Like, if something is so unreliable with information that you have to say "don't trust what this thing says" but you still put it at the top of the page? Come on... It's like putting a self destruct button in a car and telling people "well the label says not to push it!"
capital@lemmy.world · 6 pts · 1y
Remember when 4chan got people to microwave their phones because they got them to believe it would charge it?
If calling those people stupid is victim blaming then so be it. I’m blaming the victim.
This case isn’t as clear as that but even before the AI mania the instant answer at the top of Google results was frequently incorrect. Being able to discern BS from real results has always been necessary and AI doesn’t change that.
I’ve been using Kagi this year and it keeps LLM results out of the way unless I want them. When you open their AI assistant it says
I think that sums it up nicely.
njm1314@lemmy.world · 18 pts · 1y
Would that make Google liable? I mean that wouldn't be a case of users posting information that would be a case of Google posting information in that case wouldn't it? So it seems to me they'd be legally liable at that point.
TheRealLinga@sh.itjust.works · 11 pts · 1y
Ah but Google is a giant company and as is U.S. law doesn't have to fave consequences for anything
FlyingSquid@lemmy.world · 7 pts · 1y
In a sane world? Yes.
Timecircleline@sh.itjust.works · 4 pts · 1y
I think there's a disclaimer with all AI summaries. Although, I just tried googling United airlines and there is no longer an AI summary, only a message directly from their website.
jaggedrobotpubes@lemmy.world · 1 pts · 1y
Yes, so, no.
GetOffMyLan@programming.dev · 16 pts · 1y
That is literally the worst use case for AIs. There's no way they should be letting it provide contact info like that.
Also they're stupid for dialing a random number.
OsrsNeedsF2P@lemmy.ml · 15 pts · 1y
Honestly I wanted to write a snug comment about "But but it even says AI can sometimes make mistakes!", but after clicking through multiple links and disclaimers I can't find Google actually admitting that
hark@lemmy.world · 14 pts · 1y
They gave up working search with algorithms that are easier to reason about and correct for with a messy neural network that is broken in so many ways and basically impossible to generally correct while retaining its core characteristics. A change with this many regressions should've never been pushed to production.
FenrirIII@lemmy.world · 11 pts · 1y
Same happened to my wife. She gave them enough info that they threatened to call and cancel her flight unless she paid them. Never did cancel it
answersplease77@lemmy.world · 11 pts · 1y
Google has been sponsoring scammers as first search results since its creation. Google has caused hundreds of millions of dollars in losses to people, and need to be sued for it.
orcrist@lemm.ee · 10 pts · 1y
JimmyBigSausage@lemm.ee · 10 pts · 1y
Noit@lemm.ee · 9 pts · 1y
This has been an issue since long before LLMs. Before the AI summary box, scammers used targeted ads to place ahead of the actual company you were searching for.
Nalivai@lemmy.world · 5 pts · 1y
Yeah, but that was easy to spot, both by people and by Google. There were at least some guardrails, imperfect, not ideal, but they existed. With llm there is basically none
LovableSidekick@lemmy.world · 8 pts · 1y
Yep, that's Imitation Intelligence for ya.
dragonfucker@lemmy.nz · 6 pts · 1y
Drag is making sure to eat one rock per day and put glue in pizza just like the Google AI says!
GeneralInterest@lemmy.world · 3 pts · 1y
I get paranoid enough about making sure I'm clicking the correct search result and not some scam. I hope I would avoid any AI answers but yeah, to many people it could be confusing.
DarkThoughts@fedia.io · 3 pts · 1y
I'm sorry but this has been a thing long before "AI" based results. Scammers always used tricks to end up at the top of search results.
FlyingSquid@lemmy.world · 27 pts · 1y
Scammers have been a thing long before writing. That doesn't mean people shouldn't be made aware of new ways to be scammed.
DarkThoughts@fedia.io · -9 pts · 1y
That's what I'm saying though, it isn't a new way.
brbposting@sh.itjust.works · 12 pts · 1y
One could argue there is a new aspect. When it comes to retraining the public on what to trust, there’s a likely blind spot where a person may know to only call a number listed on a trusted website. So they'll check to see if they're on a bank domain before picking up the phone. If Google, being a big name, presents the number in an official looking way at the top of their pages, it may pass the sniff test. And get people into trouble.
Featured snippets would prominently display source URLs:
But AI summaries? More opaque:
markstos@lemmy.world · 8 pts · 1y
Rights. Scammers were in the listing before, and AI has helped them appear more trustworthy.
DarkThoughts@fedia.io · 1 pts · 1y
That's meaningless with how easy it is to register legit looking faux domains and how it is even easier to create legit looking sub domains. People who fall for those type of scams will likely not even understand what a domain is.
njm1314@lemmy.world · 4 pts · 1y
At the top, but that isn't what this post is saying. This is saying that Google's AI gave the scammer answer. Not that they provided a link you could click on, but that Google itself said this is the number.
DarkThoughts@fedia.io · -1 pts · 1y
It's not an AI, it's just word prediction, which also just follows stupid algorithms, just like those who determine search results. Both can be tricked / manipulated if you understand how they work. It's still the same principle for both cases.
njm1314@lemmy.world · 2 pts · 1y
Regardless of what they call it, they're the ones presenting it. I'm not arguing they can't be tricked. I'm arguing they are fundamentally different concepts. One is offering you a choice of sources, the other is making a claim. That's a pretty big distinction in a whole mess is different ways. Not the least of which is legal.
DarkThoughts@fedia.io · 0 pts · 1y
I'm sorry but no. It's not Google making that claim, it's just the LLM replying in a confident way because that's how they are expected to work. As I said, word prediction. You can install the tiniest / most dumbest model on your local PC too and ask the same question. It will give you some random hallucinated number and act like that's what you're looking for due to its default system prompt telling it to sound like an AI assistant. In the case of search engines the LLM is directly hooked into the search engine itself and just does the same thing you'd do and search for a hopefully fitting search result. So scammers playing those search algorithms to get a good spot will end up becoming the recommendation for the LLM to tell the user. It's the same thing, just displayed slightly differently. All the cool AI assistant stuff they try to present this as, is just an illusion, a word based roleplay. The only benefit here is that they can somewhat understand abstract questions, which is helpful for certain search queries, but in the end it is always the user's responsibility to check the actual search result.
ArnaulttheGrim@lemmy.world · 3 pts · 1y
Having worked with AI and AI products in my last job before I was let go I can say this:
Out of the box AI is very good at the following:
Programmed AI - read machine learning, because you are still telling it how to interpret things - can be good at (depending how good you are at telling it what it should do):
ALL AI struggles at:
The big craze over AI totally was misunderstood. AI is best to be thought of as Automated Intelligence and the word Artificial at its current state is a complete misnomer.
This is just one example of people having been mislead by the name to not fully understand what is up with AI.
LovableSidekick@lemmy.world · 3 pts · 1y
Scroll past the Imitation Intelligence summary to the actual links.
lord_ryvan@ttrpg.network · 2 pts · 1y
In all fairness, the phone number the AI response gave was most likely dreamt up by the AI and probably happened to be a scammer's number.
Speiser0@feddit.org · 1 pts · 1y
Well, I have considered it.
tfw_no_toiletpaper@lemmy.world · 1 pts · 1y
MonkderVierte@lemmy.ml · 0 pts · 1y
Well, stop it?
Honytawk@lemmy.zip · -16 pts · 1y
I mean, you are using an LLM for search, so that was bound to happen.
Use something like Perplexity.ai for search always gives me great results with links so you can find out where the AI got their info from.
This isn't a problem with AI, but a user problem.
FlyingSquid@lemmy.world · 9 pts · 1y
Google literally puts AI "search" at the top of its page and you expect everyone to just ignore that?
Honytawk@lemmy.zip · 1 pts · 1y
I'm not justifying Googles grift.
All I'm saying is that an LLM is only decent at generating text. Everything else it sucks at. So it isn't logical to use it as a search engine. Google is also to blame for that. They are slapping AI to whatever they have running. Like most tech companies these days.
Compare that to Perplexity, which is a search engine first and only uses an LLM to make a summary of websites that are riddled with SEO. And then add a link so you can check for yourself if the LLM is hallucinating or not.
Don't be like a Techbro conflating LLMs and General AI. They are different things. And the sooner all users understand this the better.
HellsBelle@sh.itjust.works · 7 pts · 1y
Blaming the victim for late-stage capitalism's grifts isn't the own you think it is.
Honytawk@lemmy.zip · 1 pts · 1y
Google is also a user of these LLM technology. They understand as much of em as the average techbro.
That is why they are switching their search engine to an LLM like the morons they are.
solsangraal@lemmy.zip · -18 pts · 1y
neanderthal@lemmy.world · 12 pts · 1y
How do you propose crossing oceans in a timely manner?
solsangraal@lemmy.zip · -9 pts · 1y
Honytawk@lemmy.zip · 10 pts · 1y
Great you have that option, most people don't.
solsangraal@lemmy.zip · -7 pts · 1y
FlyingSquid@lemmy.world · 5 pts · 1y
In general, it is much faster and much cheaper to cross an ocean by plane than by boat.
Do you think if someone lives in Brazil but their family lives in Indonesia, they should drive across South America, charter a boat and then take it across the Pacific rather than just buy a plane ticket?
solsangraal@lemmy.zip · -1 pts · 1y
FlyingSquid@lemmy.world · 3 pts · 1y
Up until now, this has not been about what you personally choose. In fact, it has to do with what the person's wife in the image I pasted chose.
Maybe she's going from Brazil to Indonesia to visit her dying father and she doesn't have six weeks to get there by boat, or the (according to my searching) approximately $60,000 it would cost to charter that boat.