Firefox’s new AI ‘Smart Window’ (only in beta builds so far) surprises reviewer by unexpectedly handing browsing history to 3rd party hosted models

booooooooo

So I asked each model where it (thinks) it is hosted/running.

[...]

Can we trust those responses, or is it more mirage with plausible-sounding responses?

Whoever wrote this doesn't know how AI works. They don't know where they're hosted. He could have asked Mozilla instead of panic-posting. This guy keeps saying his data was given to "third party hosts" but he has no idea who the hosts are, Mozilla could be the one hosting the models...

Mozilla says this was a flub; it will refine the onboarding around Smart Window to limit memory formation to post-opt-in activity only. That’s obviously the right fix.

So, false alarm. Try beta software, get software bugs.

does surprise me (use LibreWolf instead)

What the fuck?!?!??

NO!!!!!! BAD FOX!!! BAD, BAD, BAD FOX!!!!!!!!

Sounds like on by default spyware.

What a surprise...

Smart Window uses ‘memories’, things Mozilla says “…it learns from your activity” to inform its responses. You can delete memories individually, and you can set any given chat session to not use/store them.

Fine so far.

The problem? My memory list isn’t populated with things Smart Window learned since I enabled it. Oh no.

It has activity going back months. We’re talking searches and website interactions from long before I enabled this. features. Firefox just handed that history to the AI models to plough from, without telling me upfront.

I found this the creepiest aspect of Smart Window.

Mozilla says this was a flub; it will refine the onboarding around Smart Window to limit memory formation to post-opt-in activity only. That’s obviously the right fix.

Yuck. It feels like an oversight but that's one of those things you really should have more eyes + thought around.

Unfortunate Firefox L. At leadt they let you turn it off.

moziLLM

"unexpectedly"

Is that what the AI that wrote that section of the code told them?

I just read that back to myself, and 5 years ago, that would have sounded like a conspiracy wingnut stance. Scary that it's legit today.