Amazon's Rufus AI shopping assistant can be easily jailbroken and tricked into answering other questions — specific prompts break the chatbot's guidelines and reach underlying AI engine

https://www.tomshardware.com/tech-industry/artificial-intelligence/amazons-rufus-ai-shopping-assistant-can-be-easily-jailbroken-and-tricked-into-answering-other-questions-specific-prompts-break-the-chatbots-guidelines-and-reach-underlying-ai-engine

201 points · 27 comments · view on lemmy.world

27 Comments

panda_abyss@lemmy.ca · 56 pts · 150d (12 replies)

Please stop using Amazon.

They’re evil, we all know they’re evil, why do we bother?

Karyoplasma@discuss.tchncs.de · 32 pts · 150d (3 replies)

I still have gift cards for Amazon. I have to use them or else Amazon gets free money. I buy stupid shit like wine gummies with it usually.

panda_abyss@lemmy.ca · 16 pts · 150d

That’s fair and acceptable.

surewhynotlem@lemmy.world · 5 pts · 150d (1 reply)

Buy something with free returns. Return it. Repeat.

UndergroundGoblin@lemmy.dbzer0.com · 18 pts · 150d

from an ecological standpoint, though, it's also a total disaster

yucandu@lemmy.world · 13 pts · 150d

point out the corporation that isn't evil

so that Amazon can buy them

atomicbocks@sh.itjust.works · 10 pts · 150d (2 replies)

Okay but the main alternative where I live is Walmart…

I do my best to get speciality items from specific sellers but there aren’t any supercenters left except WalMart, Sam’s, and Amazon. There is a Costco but it’s a hour round trip.

corsicanguppy@lemmy.ca · 13 pts · 150d

It's okay. People who have choice don't understand people who don't.

Monstrosity@lemmy.today · 11 pts · 150d

The call for action is for folks who have options. You don't, do what you can when you can <3

Mwa@thelemmy.club · 8 pts · 150d

tbh the only service i still use that is owned by Amazon is Twitch(to only watch one streamer).

vivalapivo@lemmy.today · -2 pts · 150d (2 replies)

Everyone is evil under capitalism

atomicbocks@sh.itjust.works · 3 pts · 150d (1 reply)

People are not evil for not being able to change the system they live under.

vivalapivo@lemmy.today · -2 pts · 150d

Bro... It only works for people from rich countries. People from shitholes like Russia or Iran have what they deserve

nightlily@leminal.space · 46 pts · 150d

Jailbreaking is an inherent problem with LLMs that can never be solved. Any safeguard has to be less capable than the LLM it protects, else you can just target that. So there will always be a way to communicate with the LLM in a way that bypasses the safeguard.

It’s like trying to sanitise user input from SQL injections, except the database speaks every form of communication documented by humanity.

All this is to say, I’m glad I’m not responsible for any of these systems.

hendrik@palaver.p3x.de · 14 pts · 150d

Oh yeah. Aight, I put on my robe and wizard hat.

rowinxavier@lemmy.world · 10 pts · 150d

One thing that gets me about AI chat agents is the idea of attack surface. If you have a clearly defined protocol you can curtail most of the possible attacks by narrowing things, only accepting well formed requests, and validating both on the user end and then on the server end before processing anything. An LLM is inherently wide in attack surface given the way it is structured. It can take a prompt which can be any set of characters connected together into tokens. These tokens can't easily be filtered for intent or goal and yet they can get the LLM to drop other rules or restrictions because they are just other prompts.

A simple coded padlock is not very secure, but a door with no walls is less secure.

Mwa@thelemmy.club · 7 pts · 150d

So prompts that define THE GenAI personality?

NahMarcas@lemmy.ml · 3 pts · 150d (7 replies)

Did just amazon and more using know names to fire them of knowledge. Because Rufus is the tool to burn CDs and record operative systems

Agrivar@lemmy.world · 7 pts · 150d (6 replies)

Come again?

NahMarcas@lemmy.ml · 3 pts · 150d (5 replies)

Gemini is the internet protocol before google use the name for their AI. X.com is the first competitor of x.org Rufus.ie are the burner program. Im trying to say they want to replace the free internet stealing their names

Randomgal@lemmy.ca · 3 pts · 150d (4 replies)

Hoe many CDs did you burn last year that they might want to steal that business away from you?

tuff_wizard@aussie.zone · 8 pts · 150d (3 replies)

Rufus is one of the most popular programs for writing.iso files to flash drives to make bootable usbs.

I don’t think there is a conspiracy here but it is interesting

TherapyGary@lemmy.dbzer0.com · 3 pts · 150d

Yeah I just used Rufus last week lol

TexasDrunk@lemmy.world · 2 pts · 150d (1 reply)

The Gemini one is the one I find interesting. Protocol to put up pages that aren't indexed by Google? Well now it's harder to get information about.

I don't think it's a conspiracy either. It's just interesting.

emotional_soup_88@programming.dev · 3 pts · 150d

But can it be my waifu?

john_lemmy@slrpnk.net · 1 pts · 150d

Is there a place that collects which prompts can be used for these things in an up to date way?