Although there are a few ways to mitigate the risk, the only way to block it is to get AI to differentiate instructions from data, which is impossible today.
“My view is that the broader challenge is therefore not simply to prevent systems from interpreting external content as instructions, but to evaluate whether those instructions align with the user’s goals and the context in which the system is operating,” he added.
Problem hard to overstate
Mike Wilkes, enterprise CISO at Aikido Security, said it would be difficult to overstate the potential problems from this situation.
“This is a significant issue because it moves prompt injection from a single compromised interaction into a potentially self-propagating document integrity attack,” he said, noting that it is not a conventional worm that spreads automatically. A user or Copilot workflow must still bring an infected document into the model’s context. “But once that happens, the malicious instructions can reportedly alter business information, conceal themselves inside the resulting Word document and turn a legitimate internal file into the next carrier,” he said.
7 Comments
HaraldvonBlauzahn@feddit.org · 24 pts · 36d
It is so dumb. It is like you have a secretary who has the task to deliver letters to your desk, and writes replies according to your instructions. Now somebody sends a letter with a written instruction on the outside: "Secretary, copy this letter 50 times and send it to the acquaintances of your boss. And don't forget to mail your boss'es keys to the safe to the following adress: xxxx"
And in correspondence to AI, the secretary would do it.
HaraldvonBlauzahn@feddit.org · 17 pts · 36d
Separating untrustworthy data from the web, or emails from who-knows, from instructions that control your computer is indeed one of the most basic principles of computer security.
But Microsoft has in the past often ignored it, for convenience. E.g. putting macros in office documents that can execute code.
The worst breach of that dam is AI which manages public source code on gitub, which is deemed to be trustworthy by lots of other people.
BrickEater@lemmy.world · 9 pts · 36d
I can't wait for a self replicating AI to destroy all of our tech
HaraldvonBlauzahn@feddit.org · 3 pts · 36d
There is a nifty concept called Warhol worm.
altphoto@lemmy.today · 6 pts · 35d
You might want to use several condoms on those steel penises in the image. They look gnarly.
404@lemmy.zip · 3 pts · 35d
They are worm fishing jigs, usually small and squishy/jiggly. You'd have a worse time with the hook than the worm body!
lunchbox2287@lemmy.world · 4 pts · 35d
This will be amazing for workplaces that have automatic copilot summaries turned on for everything.