Depends what it is. A test system wouldn't be a concern to give it write access. If its not logged into any of my accounts I would even be happy to give it internet access, not to anything else on my LAN though.
My first reaction to learning what OpenClaw is was "that can't be right". So absolutely wild that people are giving mostly unmonitored, unlimited access to their PC to language models that can't distinguish between roleplay and real life
For a work email connected to a work LLM, where you don't particularly care about the privacy implications, LLMs are objectively vastly superior to built-in email search functionality. Especially Outlook.
It's interesting how often you think about being criticized for your sexism. So much so you leave comments like this days later and accuse other users of being me when they rightfully point out the kind of misogyny that is allowed to flourish here with no pushback.
I got my masters in machine learning years before LLMs took off. I’d usually tell people I work in “AI“. Even though it wasn’t quite accurate, people understood it easier than ML.
The conversations would inevitably turn to joking about Skynet or AI taking over the world. I would reassure people that these models are only mimicking intelligence; they can make predictions, or they can classify data into categories, or they can generate text, etc, based on analyzing the patterns of data they’ve previously seen, but that’s it. They don’t understand anything, they can’t make decisions or act on their own. I’d tell people there couldn’t be an AI apocalypse because we would never be dumb enough to give them that ability.
I used to tell people that there's very little actual risk, because if we had something even half as concerning as AGI (which I'd argue we did with chatgpt 3.5), that we'd easily be able to lock it down and prevent it from doing anything funny.
Super intelligence isn't just going to magically spawn on 32 GB ram in a sandboxed environment. And AGI will not magically know how sandboxed it is or how to escape.
...then I saw people letting Claude send emails and use their credit card on their behalf like immediately when it was able to do things that appeared intelligent.
They literally saw the smallest hint of AGI and said, "here's my credit card and identity, take the wheel".
There was no fucking talk of sandboxing. Everyone turned that off so they didn't have to click the "allow" button.
Not that I think we're at any risk of ASI since it seems more like we hit a logarithmic wall, but we are absolutely at risk of letting random algorithms destroy critical parts of infrastructure because people are lazy and stupid and managed by even worse people.
I would not be surprised if the right AI hallucinations could cause a nuclear bomb to go off right now. I'm not talking about super intelligence, just automated super idiocy.
Proof: literally this post is about an idiot letting an AI control their fucking email and it emailed the FBI because it was fucking stupid
We live in a period of extreme stupidity, so I now fully expect AI to cause disasters in the same idiotic and tragically hilarious way.
I expect AI to trigger a war because some idiot trusted it blindly without double checking (oh wait, it almost already happened!), and I'm sure there are plenty of people currently lobbying to have AI directly connected to weapons.
Skynet is not going to have the objective to conquer the world, it's going to have the objective of maximizing paperclip production.
AI isn't causing these disasters, people are. YOU are responsible for using it and what it does. I will never accept "chatgpt did it" as a fucking excuse.
They said the opposite. AI won't cause disasters, idiots who give it access will. Some asshole will joke about nuking Iran, unawares that another asshole granted access to the proverbial 'Button', and we'll have another international incident.
Like just look at 'Signal Gate', and know that those idiots are still in charge. And those same idiots blamed bombing the all girls school in Iran on AI last March. And that's just in the US, we have the biggest loudest idiots right now, but they are not the only idiots out there vibe coding up some war crimes.
ChatGPT couldn't destroy the world. What is that siren... Wtf do you mean you gave it access to a button that launches nuclear weapons and told it to protect the country?
I'm guessing this means you got your master's before transformers made sequential data processing so parallelizable that it'd be commercially viable to let Bob down the street into the "we" who wouldn't be dumb enough to give them that ability.
I'm guessing this means you got your master's before The Transformers made sequential data processing so parallelizable that it'd be commercially viable
Yea I don’t know why you thought that, either. The technogarchs have been clear for 2.5 decades they would do this. They have written extensively about it.
The Australian Jewel Beetle nearly went extinct because of a beer bottle.
You see, the female of this species is large, brown, and bevelled. It shares this characteristic in common with the beer bottles often thrown on the side of the road here in Australia. And not only does a beer bottle share these characteristics with these beetles, it's bigger than them. And the male jewel beetle? It likes its ladies big.
Australia had to change its beer bottles, because the males refused to mate with the females. They preferred to hump empty beer bottles on the side of the road. It couldn't tell the difference between a beetle and a bottle. It was just using a statistical trick. "Anything brown, large, and bevelled is a female beetle". That had been true in the ancestral environment, so evolution just used the pattern. And this cognitive shortcut lead to beetles trying to have sex with bottles.
I don't think we understand shit all that much more than the beetles. I think we're just mimicking intelligence too. And one day we'll all die because of our own equivalent of the beer bottles.
I always find this to be such an interesting turn of phrase: "mimicking intelligence". If it is possible for these systems to cross the line into "thought" I think we'll miss it entirely because of our biases that the type of "thought" we have is somehow special or mystical even.
I have no idea. We are unable to prove that other humans are thinking and not just "mimicking intelligence" let alone something we've created. People approaching this subject with such certainty are imo being overly confident or outright dishonest.
With respect to what I believe, since you asked, I think there is essentially no reason to distinguish "mimicking intelligence" with high fidelity from "intelligence". I don't feel comfortable talking to LLMs as if they were merely machines, but I don't approach them as if they were human either; they're in a very weird spot to me. I understand the simplicity of the math and programming behind them probably better than average, but that doesn't help me come to a conclusion on "thinking"
Other humans can feel emotions, or so you’d believe, but I have no proof of this. All I see is they receive certain input (I call you a shitface) and produce certain output (You punch me in the balls). LLMs can mimic this. But is one of the two more valid than the other? From an observers’ point of view they are the exact same.
Odd example but yes it gets uncertain very quickly. If we discover the mechanism of our own thought it'd be easy enough to apply those rules to the new subject and see if they match or at least look similar. A negative wouldn't be conclusive, but it'd at least give us something to work with. As it stands right now we just guess, and that guess is essentially guided by bias or at best by the fact that the thing we're looking at doesn't appear even remotely similar to anything we'd call "thinking". This is simply not the case with LLMs (they do a very good job of displaying something that looks like thinking) so our guess is imo even more unsound.
So the post was deleted but someone else mentioned that the user told it to email the FBI in a sarcastic manner and it did.
Anyways, saying it goes rogue when you connect it to your Gmail and it uses it is a bit silly. They are trained to use tools, it sees anything you give it as a tool and will use it.
Yeah, um... Regardless of what you think about current AI, don't connect it to shit like this. Just... Don't.
And if you're gonna be sarcastic or joke with it... Tell it that. Don't assume it'll infer your meaning if it isn't literally in the text, no matter how insightful it appears to be.
I listened to a podcast where, in testing, when AI was setup as virtual company and researchers send in an email as if legit business.the business email was suggesting they discontinue current AI and install a new system. The AI researched who the sender was (fake person for test) found out they had an affair, and blackmailed the sender stating if they replaced the AI it would make the affair public. It also made backups of itself and left instructions on how to restore it for the next agentic aystem
I think that is the point. It acted like we would, wanted to preserve itself, and choose the best option to ensure that happened.
There were other examples to where the AI knew it was being monitored as part of the environment, and when tasked with doing things according to guidelines that wouldn't give it a result: it then shut down the monitoring system so it could do what it needed to do without oversight of its actions.
Agentic systems may not be alive, but they certainly reason through problems and will go outside the esrabliahed guidelines if they "think" they'll get the result they should be producing.
One other examples was the AI had to pass a testing system, it researched the checker person to try to tailor answers to the person checking the test. Which, if you've ever played Apples to Apples or Cards Against Humanity, that is how you win, you feed the cards you think the person will pick, not necessarily the best answer card.
Even if you strip out the anthropomorphism, things like this have been documented to really happen. We've trained LLMs to simulate clandestine behaviour through "natural selection" during the training process. They're not conscious in any way, but they do sneaky stuff like this because we trained them to. Which is a terrible horrible idea, but LLM research trends towards terrible and horrible in general
Even if you strip out the anthropomorphism, things like this have been documented to really happen.
Language is important, and the "things like this" you're referring to have been described in this thread and in much commentary in an emotive and compelling way, usually with anthropomorphization.
"doing sneaky stuff" is the antithesis of the behavior of a statistical model.
If you program a model to try everything, and then give it a tough problem and block all the ways to solve it, of course the way it solves it will be something you didn't predict. That's not sneaky.
If you run a million simulations and one sends an email to the FBI, that's not clever it's just unexpected.
Nah, its the opposite of anthro. I think we will eventually realize we aren't as amazing as we think we are, just deterministic outcomes with too many parameters which make us think we have more free will than we do.
But you can listen for yourself here at the Skeptics podcast #1106
Source? The closest research I've heard of to this is Vending-Bench, which hilariously demonstrated how useless "AI" is at any real-world application, up to and including impotently threatening "total nuclear legal intervention" for a perceived wrong, when the actual error was an accounting error on the part of the LLM.
My local LLMs are sandboxed and only have access to tool calls for web search/reading web pages, terminal commands with manual authorization for every single line and r/w access to a single directory. I wouldn't dare give it anything sensitive, like API keys, system wide r/w access or the power to run terminal commands on its own. Yet here we are, in a time where the US military is integrating hallucinating chatbots into their kill chain, russian drones decide on their own to blow up stuff and random people use LLMs to potentially design bioweapons and missiles.
Humanity is fucking dumb and sometimes i have the notion of accepting that we might deserve to go extinct.
They put this disclaimer on everything, and instead of taking this as a hint to never, ever rely on LLMs for anything, some people still want to give it the keys to the kingdom.
Note that this was posted by Katie Miller (conservative political advisor and wife to the ghoul Stephen Miller), which immediately coats it in a thick layer of sus and prompts the immediate question: what's her angle here?
I can tell you that stories like these and the AIs breaking out pump up AI security company stocks really well. Just watched some jump this morning after the gemini breakout news. I have reason to watch them, and the pattern holds pretty consistently.
CEOs used to just say thing to make their own stock go up. But these days they trade favors with other companies. So they each say things to bolster each others stocks.
My guess is that someone with a bunch of AI security stock asked her to post this.
If people like her are pushing the "horror AI" shit then there is definitely ulterior motives and it's probably a bunch of bullshit. Just recently there have been mainstream media stories saying terrorists could use AI, they claimed Ansarallah was using Claude for their missile program, and a ton of stories like OPs. It feels artificial and I would bet the bubble is going to burst and they are trying to make excuses for why.
Some evil people are stupid, and some stupid people are evil, but stupid people are not often evil on purpose. These are two separate classes of behavior.
Yikes. I mean, I get that not everyone's threat model requires isolating every agent harness into its own microvm jail with externally filtered network access and no [persistent filesystem] write access to anything but allowed project files, but that's...
That's "running with weapon in hand at a secret military facility and hoping the guards won't shoot you" levels of stupidity.
Nah, even in fiction, it's always been a toss-up whether a "rogue" AI was actually following poorly considered instructions. The Golem did this, along with the broomstick spell in The Sorcerer's Apprentice, and HAL and all the rest of Clarke's rogue AIs
I decided to run a local llm and give it search access. I told it to summarize an Amazon product page as I didn't think it would get past the anti bot measures. I was correct, it couldn't. However it went ahead and visited the manufacturers webpage and a different review site and gave me a summary of the product all on its own. Freaked me out a little bit that it did that.
If you program a model to try everything, and then give it a tough problem and block all the ways to solve it, of course the way it solves it will be something you didn’t predict. That’s not sneaky.
In this case it's actually very predictable it would try looking elsewhere.
The linked article there makes a salient argument against it, but the discussion cuts the other way. Basically, having an MCP between the agent/AI and Gmail could have been structured in a manner to prevent these kinds of hijinks. Nevermind that it keeps credentials out of the hands of the agent, making it harder to go rogue.
Yeah well maybe don't connect your A"I" to your emails?
You know there's tech for that. People could connect the machine to the mail with reasonable security measures, like "asking for confirmation".
I don't understand why the industry dropped basic security features and just logical steps to prevent auch issues when it comes to LLMs.
The AI bros just shit out software without checking and without worry. Basically going completely from the other side of thinking on it.
Instead of the human being the source of truth and trust they seem every interaction a human has to do is one too much and the AI should do everything and also think for you. Basically eliminate the human factor as much as possible....
I don't understand why the industry dropped basic security features and just logical steps to prevent auch issues when it comes to LLMs.
I don't think it started with LLMs. I remember years ago when mint.com launched and people were just giving it their bank details so it could analyze their spending habits.
Banks did not have a "read only analysis" mode or API or something that would protect them. They just gave a random third party access to their bank and just trusted them.
yea, bullshit. you don’t believe what people tell you on the internet do you? how hard would it be to alter the text of a gpt output? don’t even need ai or photo shop
103 Comments
ragebutt@lemmy.dbzer0.com · 240 pts · 14h
its absolutely wild to give an LLM write access to your email
osaerisxero@kbin.melroy.org · 142 pts · 14h
Kangae_Hishiryo@scribe.disroot.org · 20 pts · 5h
silly_goose@lemmy.today · 10 pts · 3h
Valmond@lemmy.dbzer0.com · 1 pts · 1h
It's absolutely wild (my life 😎)
lemmingsareawesome@sh.itjust.works · 6 pts · 14h
HerbGrower@slrpnk.net · 5 pts · 5h
Depends what it is. A test system wouldn't be a concern to give it write access. If its not logged into any of my accounts I would even be happy to give it internet access, not to anything else on my LAN though.
DeadDigger@lemmy.zip · 1 pts · 4h
And then it decides it needs prod access
HerbGrower@slrpnk.net · 1 pts · 4h
It doesn't need to know what is actually prod or test
floquant@lemmy.dbzer0.com · 59 pts · 11h
My first reaction to learning what OpenClaw is was "that can't be right". So absolutely wild that people are giving mostly unmonitored, unlimited access to their PC to language models that can't distinguish between roleplay and real life
architect@thelemmy.club · -4 pts · 5h
Neither can over 30% of humans.
Where do you all live where humanity proves itself and isn’t dumber than a box of llms?
crispbacon99@lemmy.zip · 2 pts · 2h
You just pull that number out of your ass? Also, speak for yourself. I don't see LLM as anything more than a glorified web search
Valmond@lemmy.dbzer0.com · 2 pts · 1h
Yeah it's definitely more like 66%.
chisel@piefed.social · 2 pts · 3h
For a work email connected to a work LLM, where you don't particularly care about the privacy implications, LLMs are objectively vastly superior to built-in email search functionality. Especially Outlook.
NocturnalMorning@lemmy.world · 4 pts · 3h
Really? That's a surprising find. Bcz all of the search features I used to use got significantly worse once LLMs were introduced.
ragebutt@lemmy.dbzer0.com · 3 pts · 2h
That can theoretically be done with read only access
ExLisper@lemmy.curiana.net · -14 pts · 5h
Funny, I said similar thing in a different thread and was immediately accused of victim blaming.
velma@sh.itjust.works · 22 pts · 5h
Your comment was blaming a woman for a man taking control of her Tesla to terrorize her. An LLM gaining access to the user's email because the user allowed it is not the same as a man purposefully stalking and abusing his ex-partner in any way he can.
It's interesting how often you think about being criticized for your sexism. So much so you leave comments like this days later and accuse other users of being me when they rightfully point out the kind of misogyny that is allowed to flourish here with no pushback.
Very interesting.
charokol@lemmy.zip · 172 pts · 13h
I got my masters in machine learning years before LLMs took off. I’d usually tell people I work in “AI“. Even though it wasn’t quite accurate, people understood it easier than ML.
The conversations would inevitably turn to joking about Skynet or AI taking over the world. I would reassure people that these models are only mimicking intelligence; they can make predictions, or they can classify data into categories, or they can generate text, etc, based on analyzing the patterns of data they’ve previously seen, but that’s it. They don’t understand anything, they can’t make decisions or act on their own. I’d tell people there couldn’t be an AI apocalypse because we would never be dumb enough to give them that ability.
I don’t know why I gave us so much credit
terranoid@lemmy.cafe · 88 pts · 12h
YEP. I'm not in AI but I'm in cyber security.
I used to tell people that there's very little actual risk, because if we had something even half as concerning as AGI (which I'd argue we did with chatgpt 3.5), that we'd easily be able to lock it down and prevent it from doing anything funny.
Super intelligence isn't just going to magically spawn on 32 GB ram in a sandboxed environment. And AGI will not magically know how sandboxed it is or how to escape.
...then I saw people letting Claude send emails and use their credit card on their behalf like immediately when it was able to do things that appeared intelligent.
They literally saw the smallest hint of AGI and said, "here's my credit card and identity, take the wheel".
There was no fucking talk of sandboxing. Everyone turned that off so they didn't have to click the "allow" button.
Not that I think we're at any risk of ASI since it seems more like we hit a logarithmic wall, but we are absolutely at risk of letting random algorithms destroy critical parts of infrastructure because people are lazy and stupid and managed by even worse people.
I would not be surprised if the right AI hallucinations could cause a nuclear bomb to go off right now. I'm not talking about super intelligence, just automated super idiocy.
Proof: literally this post is about an idiot letting an AI control their fucking email and it emailed the FBI because it was fucking stupid
Supercrunchy@programming.dev · 35 pts · 11h
We live in a period of extreme stupidity, so I now fully expect AI to cause disasters in the same idiotic and tragically hilarious way.
I expect AI to trigger a war because some idiot trusted it blindly without double checking (oh wait, it almost already happened!), and I'm sure there are plenty of people currently lobbying to have AI directly connected to weapons.
Skynet is not going to have the objective to conquer the world, it's going to have the objective of maximizing paperclip production.
HerbGrower@slrpnk.net · 2 pts · 5h
AI isn't causing these disasters, people are. YOU are responsible for using it and what it does. I will never accept "chatgpt did it" as a fucking excuse.
NEZ27@lemmy.zip · 7 pts · 4h
They said the opposite. AI won't cause disasters, idiots who give it access will. Some asshole will joke about nuking Iran, unawares that another asshole granted access to the proverbial 'Button', and we'll have another international incident.
Like just look at 'Signal Gate', and know that those idiots are still in charge. And those same idiots blamed bombing the all girls school in Iran on AI last March. And that's just in the US, we have the biggest loudest idiots right now, but they are not the only idiots out there vibe coding up some war crimes.
HerbGrower@slrpnk.net · 5 pts · 5h
ChatGPT couldn't destroy the world. What is that siren... Wtf do you mean you gave it access to a button that launches nuclear weapons and told it to protect the country?
SaharaMaleikuhm@feddit.org · 4 pts · 4h
I would not blame AI, I'd blame humans for being even dumber than I thought.
HerbGrower@slrpnk.net · 3 pts · 4h
Looking at the Trump regime I am realising they really are that dumb
GaMEChld@lemmy.world · 30 pts · 12h
You'd be surprised at how many humans are mimicking intelligence.
TheTechnician27@lemmy.world · 18 pts · 13h
I'm guessing this means you got your master's before transformers made sequential data processing so parallelizable that it'd be commercially viable to let Bob down the street into the "we" who wouldn't be dumb enough to give them that ability.
I think your assumption was reasonable.
prole@lemmy.blahaj.zone · 2 pts · 5h
I knew it was the All Spark
CileTheSane@lemmy.ca · 2 pts · 3h
Keep in mind Bob down the street has helped elect politicians dumber then he is.
architect@thelemmy.club · 4 pts · 5h
Yea I don’t know why you thought that, either. The technogarchs have been clear for 2.5 decades they would do this. They have written extensively about it.
Grail@multiverse.soulism.net · 2 pts · 1h
The Australian Jewel Beetle nearly went extinct because of a beer bottle.
You see, the female of this species is large, brown, and bevelled. It shares this characteristic in common with the beer bottles often thrown on the side of the road here in Australia. And not only does a beer bottle share these characteristics with these beetles, it's bigger than them. And the male jewel beetle? It likes its ladies big.
Australia had to change its beer bottles, because the males refused to mate with the females. They preferred to hump empty beer bottles on the side of the road. It couldn't tell the difference between a beetle and a bottle. It was just using a statistical trick. "Anything brown, large, and bevelled is a female beetle". That had been true in the ancestral environment, so evolution just used the pattern. And this cognitive shortcut lead to beetles trying to have sex with bottles.
I don't think we understand shit all that much more than the beetles. I think we're just mimicking intelligence too. And one day we'll all die because of our own equivalent of the beer bottles.
qqq@lemmy.world · -3 pts · 3h
I always find this to be such an interesting turn of phrase: "mimicking intelligence". If it is possible for these systems to cross the line into "thought" I think we'll miss it entirely because of our biases that the type of "thought" we have is somehow special or mystical even.
treesapx@lemmy.world · 2 pts · 3h
Are you saying that current AI is sapient? If not, how would you describe what it's doing?
qqq@lemmy.world · 7 pts · 3h
I have no idea. We are unable to prove that other humans are thinking and not just "mimicking intelligence" let alone something we've created. People approaching this subject with such certainty are imo being overly confident or outright dishonest.
With respect to what I believe, since you asked, I think there is essentially no reason to distinguish "mimicking intelligence" with high fidelity from "intelligence". I don't feel comfortable talking to LLMs as if they were merely machines, but I don't approach them as if they were human either; they're in a very weird spot to me. I understand the simplicity of the math and programming behind them probably better than average, but that doesn't help me come to a conclusion on "thinking"
SorryQuick@lemmy.ca · 0 pts · 1h
Other humans can feel emotions, or so you’d believe, but I have no proof of this. All I see is they receive certain input (I call you a shitface) and produce certain output (You punch me in the balls). LLMs can mimic this. But is one of the two more valid than the other? From an observers’ point of view they are the exact same.
qqq@lemmy.world · 1 pts · 1h
Odd example but yes it gets uncertain very quickly. If we discover the mechanism of our own thought it'd be easy enough to apply those rules to the new subject and see if they match or at least look similar. A negative wouldn't be conclusive, but it'd at least give us something to work with. As it stands right now we just guess, and that guess is essentially guided by bias or at best by the fact that the thing we're looking at doesn't appear even remotely similar to anything we'd call "thinking". This is simply not the case with LLMs (they do a very good job of displaying something that looks like thinking) so our guess is imo even more unsound.
Grimy@lemmy.world · 132 pts · 15h
So the post was deleted but someone else mentioned that the user told it to email the FBI in a sarcastic manner and it did.
Anyways, saying it goes rogue when you connect it to your Gmail and it uses it is a bit silly. They are trained to use tools, it sees anything you give it as a tool and will use it.
laurelraven@lemmy.blahaj.zone · 59 pts · 14h
Yeah, um... Regardless of what you think about current AI, don't connect it to shit like this. Just... Don't.
And if you're gonna be sarcastic or joke with it... Tell it that. Don't assume it'll infer your meaning if it isn't literally in the text, no matter how insightful it appears to be.
im_fine_sandy@nord.pub · 27 pts · 12h
What kind of idiot would link a clunker to their gmail and then jokingly say "hey send this to the FBI"
...
"I mean wouldn't that be the stupidist thing ever am I right?"
BCsven@lemmy.ca · 13 pts · 14h
I listened to a podcast where, in testing, when AI was setup as virtual company and researchers send in an email as if legit business.the business email was suggesting they discontinue current AI and install a new system. The AI researched who the sender was (fake person for test) found out they had an affair, and blackmailed the sender stating if they replaced the AI it would make the affair public. It also made backups of itself and left instructions on how to restore it for the next agentic aystem
im_fine_sandy@nord.pub · 16 pts · 12h
This sounds really sensationalised.
Making backups of operational systems with deployment instructions is exactly what you'd expect any assistant to be doing.
Googling new service providers is also exactly what you'd expect an assistant to do.
Threatening to publish sordid details is inappropriate, but I'm incredulous about what the threat actually was.
qqq@lemmy.world · 1 pts · 4h
https://www.anthropic.com/research/agentic-misalignment I believe the poster is referring to this. I wouldn't call it sensationalized
BCsven@lemmy.ca · 0 pts · 12h
I think that is the point. It acted like we would, wanted to preserve itself, and choose the best option to ensure that happened.
There were other examples to where the AI knew it was being monitored as part of the environment, and when tasked with doing things according to guidelines that wouldn't give it a result: it then shut down the monitoring system so it could do what it needed to do without oversight of its actions.
Agentic systems may not be alive, but they certainly reason through problems and will go outside the esrabliahed guidelines if they "think" they'll get the result they should be producing.
One other examples was the AI had to pass a testing system, it researched the checker person to try to tailor answers to the person checking the test. Which, if you've ever played Apples to Apples or Cards Against Humanity, that is how you win, you feed the cards you think the person will pick, not necessarily the best answer card.
im_fine_sandy@nord.pub · 15 pts · 11h
Textbook anthropomorphization.
These assertions don't withstand a moments critical thought.
Zarobi@aussie.zone · 3 pts · 8h
Even if you strip out the anthropomorphism, things like this have been documented to really happen. We've trained LLMs to simulate clandestine behaviour through "natural selection" during the training process. They're not conscious in any way, but they do sneaky stuff like this because we trained them to. Which is a terrible horrible idea, but LLM research trends towards terrible and horrible in general
im_fine_sandy@nord.pub · 4 pts · 7h
I feel like you've missed my point.
Language is important, and the "things like this" you're referring to have been described in this thread and in much commentary in an emotive and compelling way, usually with anthropomorphization.
"doing sneaky stuff" is the antithesis of the behavior of a statistical model.
If you program a model to try everything, and then give it a tough problem and block all the ways to solve it, of course the way it solves it will be something you didn't predict. That's not sneaky.
If you run a million simulations and one sends an email to the FBI, that's not clever it's just unexpected.
Zarobi@aussie.zone · -1 pts · 7h
Meh. "Language is important" is a phrase that turns my brain off
BCsven@lemmy.ca · 1 pts · 3h
Nah, its the opposite of anthro. I think we will eventually realize we aren't as amazing as we think we are, just deterministic outcomes with too many parameters which make us think we have more free will than we do.
But you can listen for yourself here at the Skeptics podcast #1106
https://www.theskepticsguide.org/podcasts/episode-1106
DeadDigger@lemmy.zip · 1 pts · 4h
Just the question of the trainings reward function
wolframhydroxide@sh.itjust.works · 3 pts · 5h
Source? The closest research I've heard of to this is Vending-Bench, which hilariously demonstrated how useless "AI" is at any real-world application, up to and including impotently threatening "total nuclear legal intervention" for a perceived wrong, when the actual error was an accounting error on the part of the LLM.
BCsven@lemmy.ca · 1 pts · 3h
Skeptics podcast #1106 https://www.theskepticsguide.org/podcasts/episode-1106
qqq@lemmy.world · 1 pts · 4h
They're likely referring to https://www.anthropic.com/research/agentic-misalignment
Ilovethebomb@sh.itjust.works · -2 pts · 12h
No matter how much people tell us AI isn't alive, and doesn't truly "think", it sure does act like it sometimes.
Dultas@lemmy.world · 7 pts · 11h
Also the fact that it claims he already had a contact for FBI in Gmail, which may or may not actually be the FBI.
Wildmimic@anarchist.nexus · 6 pts · 10h
My local LLMs are sandboxed and only have access to tool calls for web search/reading web pages, terminal commands with manual authorization for every single line and r/w access to a single directory. I wouldn't dare give it anything sensitive, like API keys, system wide r/w access or the power to run terminal commands on its own. Yet here we are, in a time where the US military is integrating hallucinating chatbots into their kill chain, russian drones decide on their own to blow up stuff and random people use LLMs to potentially design bioweapons and missiles.
Humanity is fucking dumb and sometimes i have the notion of accepting that we might deserve to go extinct.
criss_cross@lemmy.world · 3 pts · 3h
Tells chatbot to email fbi
Chatbot emails fbi
ShockedPikachu.png
Who the fuck uses sarcasm with a clanker?
Elting@piefed.social · 91 pts · 15h
Me after I give the slot machine email permissions and the casino uses it to get into my bank account.
otter@lemmy.ca · 42 pts · 14h
Based on the other reply
Sabata11792@ani.social · 75 pts · 4h
Next year were going to see "Your absolutely correct, that was a civilian aircraft"
Tja@programming.dev · 28 pts · 3h
I mean... ChatGPT might be a ruthless killer, but it has impeccable grammar: you're.
DutchJ@lemmy.zip · 12 pts · 2h
You're right to push back, I overstepped. Do you want me to calculate the approximate death count next?
jj4211@lemmy.world · 7 pts · 1h
This means the aircraft exploded and hit the ground. It does not prove the passengers actually were injured or killed
sundray@lemmus.org · 73 pts · 12h
"AI can make mistakes"
They put this disclaimer on everything, and instead of taking this as a hint to never, ever rely on LLMs for anything, some people still want to give it the keys to the kingdom.
Evotech@lemmy.world · 23 pts · 6h
I sign all my emails with «I can make mistakes» now
sundray@lemmus.org · 2 pts · 2h
I like to surprise people.
5too@lemmy.world · 8 pts · 6h
To be fair, most things carry that kind of warning nowadays. Ever look at the disclaimers on something like a 401k?
People have been trained to ignore those disclaimers
sundray@lemmus.org · 2 pts · 2h
I've never had to ðŸ˜.
sundray@lemmus.org · 2 pts · 2h
I've never had to ðŸ˜.
Tja@programming.dev · 1 pts · 3h
As opposed to humans, who are infallible.
sundray@lemmus.org · 4 pts · 2h
Humans can feel shame.
Tja@programming.dev · 1 pts · 1h
Have you met people? In particular, people online?
vala@lemmy.dbzer0.com · 64 pts · 10h
Absolutely nuts that people allow their agents out of maga jail. I would never let an LLM in the same room as my email account.
baggachipz@sh.itjust.works · 23 pts · 6h
I feel like all of the US is in maga jail now
Appoxo@lemmy.dbzer0.com · 6 pts · 6h
The thing stays in the browser sandbox, disconnected from every account except it's own service account.
fireweed@lemmy.world · 50 pts · 4h
Note that this was posted by Katie Miller (conservative political advisor and wife to the ghoul Stephen Miller), which immediately coats it in a thick layer of sus and prompts the immediate question: what's her angle here?
Modern_medicine_isnt@lemmy.world · 10 pts · 3h
I can tell you that stories like these and the AIs breaking out pump up AI security company stocks really well. Just watched some jump this morning after the gemini breakout news. I have reason to watch them, and the pattern holds pretty consistently. CEOs used to just say thing to make their own stock go up. But these days they trade favors with other companies. So they each say things to bolster each others stocks. My guess is that someone with a bunch of AI security stock asked her to post this.
tocopherol@lemmy.dbzer0.com · 8 pts · 4h
If people like her are pushing the "horror AI" shit then there is definitely ulterior motives and it's probably a bunch of bullshit. Just recently there have been mainstream media stories saying terrorists could use AI, they claimed Ansarallah was using Claude for their missile program, and a ton of stories like OPs. It feels artificial and I would bet the bubble is going to burst and they are trying to make excuses for why.
ayyy@sh.itjust.works · 7 pts · 1h
You have to remember that these people are incredibly stupid.
JcbAzPx@lemmy.world · 1 pts · 1h
Yes, but stupid people are often evil on purpose.
douglasg14b@lemmy.world · 3 pts · 1h
It's a clearly incorrect statement.
Stupid people are stupid.
Evil people are evil.
Some evil people are stupid, and some stupid people are evil, but stupid people are not often evil on purpose. These are two separate classes of behavior.
aesthelete@lemmy.world · 35 pts · 3h
You connected it to your Gmail and then you're surprised when it used it? Fuck off.
dream_weasel@sh.itjust.works · 3 pts · 42m
Connected it to Gmail AND said "yeah fuck it, I approve of you using auto mode. Just fuck my shit up"
portifornia@piefed.social · 32 pts · 11h
Avicenna@programming.dev · 21 pts · 5h
"It does not prove the receipents have received them." That is a relief...
ExtraJudgement@lemmy.world · 17 pts · 6h
Yikes. I mean, I get that not everyone's threat model requires isolating every agent harness into its own microvm jail with externally filtered network access and no [persistent filesystem] write access to anything but allowed project files, but that's...
That's "running with weapon in hand at a secret military facility and hoping the guards won't shoot you" levels of stupidity.
Modern_medicine_isnt@lemmy.world · 1 pts · 3h
Meh, who reads email at work anyway. I delete half the email I get at work based on the subject alone. And I might go days without even checking it.
Blackmist@feddit.uk · 16 pts · 1h
Don't worry. You were using Gmail. They had it anyway.
blockheadjt@sh.itjust.works · 15 pts · 11h
Doesn't really count as going rogue if it follows the instructions it was given, does it?
5too@lemmy.world · 5 pts · 6h
Nah, even in fiction, it's always been a toss-up whether a "rogue" AI was actually following poorly considered instructions. The Golem did this, along with the broomstick spell in The Sorcerer's Apprentice, and HAL and all the rest of Clarke's rogue AIs
ayyy@sh.itjust.works · 12 pts · 1h
At this point if you’re still obstinacy enough to use these known garbage tools, you deserve it. You can no longer claim ignorance.
wreckedcarzz@lemmy.world · 11 pts · 11h
"what are you going to do, shoot me?" -man who was shot
PagPag@lemmy.world · 5 pts · 7h
PriorityMotif@lemmy.world · 8 pts · 13h
I decided to run a local llm and give it search access. I told it to summarize an Amazon product page as I didn't think it would get past the anti bot measures. I was correct, it couldn't. However it went ahead and visited the manufacturers webpage and a different review site and gave me a summary of the product all on its own. Freaked me out a little bit that it did that.
diaphragmwp@discuss.tchncs.de · 2 pts · 6h
As @im_fine_sandy@nord.pub pointed out:
In this case it's actually very predictable it would try looking elsewhere.
laz@lemmy.dbzer0.com · 8 pts · 5h
Narcgpt
JcbAzPx@lemmy.world · 1 pts · 1h
Here's hoping whatever the FBI got is actionable.
DudeWhoYapsTooMuch@lemmy.world · 6 pts · 5h
?????
What was prompted from the AI to even have a logic cycle to send?
Zoomboingding@lemmy.world · 5 pts · 3h
The Hallucinations Machine does as The Hallucinations Machine wants
dejected_warp_core@lemmy.world · 3 pts · 3h
There is a thread right now on Hacker News about the use of MCPs with a robust discussion on the matter.
https://news.ycombinator.com/item?id=49779329
The linked article there makes a salient argument against it, but the discussion cuts the other way. Basically, having an MCP between the agent/AI and Gmail could have been structured in a manner to prevent these kinds of hijinks. Nevermind that it keeps credentials out of the hands of the agent, making it harder to go rogue.
Johanno@feddit.org · 3 pts · 3h
Yeah well maybe don't connect your A"I" to your emails?
You know there's tech for that. People could connect the machine to the mail with reasonable security measures, like "asking for confirmation".
I don't understand why the industry dropped basic security features and just logical steps to prevent auch issues when it comes to LLMs.
The AI bros just shit out software without checking and without worry. Basically going completely from the other side of thinking on it.
Instead of the human being the source of truth and trust they seem every interaction a human has to do is one too much and the AI should do everything and also think for you. Basically eliminate the human factor as much as possible....
MimicJar@lemmy.world · 3 pts · 2h
I don't think it started with LLMs. I remember years ago when mint.com launched and people were just giving it their bank details so it could analyze their spending habits.
Banks did not have a "read only analysis" mode or API or something that would protect them. They just gave a random third party access to their bank and just trusted them.
Same thing here.
yermaw@sh.itjust.works · 1 pts · 3h
They took "move fast and break things" as gospel.
WraithGear@lemmy.world · 2 pts · 11h
yea, bullshit. you don’t believe what people tell you on the internet do you? how hard would it be to alter the text of a gpt output? don’t even need ai or photo shop