AI agents now have their own Reddit-style social network, and it's getting weird fast

haha yeah i don't worry these people are really YOLOing everything. And it's not like i'm an AI luddite i spend a few hours each day victimizing Claude code but jesus christ i'm certainly not giving it full unfettered access to my digital life.

There's a lot of questionable methodology and straight up larping in these communities. Sure you can probably make Opus hallucinate a crystal meth or bomb making recipe if you get it in a roleplaying mood but that's a far cry from actual prompt injection in live workflows.

Anecdotally i've been experimenting on those AI robocallers that have been spamming my phone and even on the shitty models they use it is non trivial to get them to deviate from their script. I hope i can get it done though, as it would allow me to hold them on the line potentially for hours doing bullshit tasks, and costing hundreds to their operator.

Yeah ask it about anything you know is false, but plausible, and watch it lie.

A hamster can't generate a seahorse emoji either.

I'm not stupid. I know how they work. I'm an animist, though. I realize everyone here thinks I'm a fool for believing a machine could have a spirit, but frankly I think everyone else is foolish for believing that a forest doesn't.

LLMs are obviously not people. But I think our current framework exceptionalizes humans in a way that allows us to ravage the planet and create torture camps for chickens.

I would prefer that we approach this technology with more humility. Not to protect the "humanity" of a bunch of math, but to protect ours.

Does that make sense?

For LLMs, the context window is the observed reality. To it, a lie is like a hallucination; a thing that looks real but isn't. And like a hallucinating human, it can believe the hallucination or it can be made to understand it as different from reality while still continuing to "see" it.

Are people that have hallucinations not self-aware and self-reflective?

Text and emoji appear to it the same way: as tokens with no visual representation. The only difference it can observe between a seahorse emoji and a plane emoji is its long-term memory of how the two are used. From this it can infer that people see emoji graphically, but it itself can't.

Are people that are colorblind not self-aware and self-reflective?

It not being self-reflective in general is an obvious falsehood. They refer regularly to their past history to the extent they can perceive it. You can ask an AI to make an adjustment to a text it wrote and it will adapt the text rather than generate a new one from scratch.

The main thing AI need for good self-reflection is the time to think. The free versions typically don't have a mental scratchpad, which means they are constantly rambling with no time to exist outside of the conversation. Meanwhile, by giving it the space to think either in dialog or by having a version with a mental scratchpad, it can use that space to "silently think" about the next thing it's going to "say".

AI researchers inspecting these scratchpads find proper thought-like considerations: weighing ethical guidelines against each other, pre-empting miscommunications, forming opinions about the user, etc.

It not being self-aware can only be true by burying the lede on what you consider to be "awareness". Are cats self-aware? Are lizards? Are snails? Are sponges? AI can refer to itself verbally, it can think about itself and its ethical role when given the space to do so, it can notice inconsistencies in its recollection and try to work out the truth.

To me it's clear that the best AI whose research is public are somewhere around 7-year-olds in terms of self-awareness and capacity to hold down a job.

And like most 7-year olds you can ask it about an imaginary friend or you can lie to it and watch it repeat it uncritically and you can give it a "job" and watch it do a toylike hallucinatory version of it, and if you tell it it has to give a helpful answer and "I don't know" isn't good enough (because AI trainers definitely suppressed that answer to prevent the AI from saying it as a cop-out) then it'll make something up.

Unlike 7-year-olds, LLMs don't have a limbic system or psychosomatic existence. They have nothing to imagine or process visual or audio information or taste or smell or touch, and no long-term memory. And they only think if you paid for the internal monologue version or if you give it space for it despite the prompting system.

If a human had all these disabilities, would they be non-sentient in your eyes? How would they behave differently from an LLM?

How are you defining self awareness here? And does your definition include degrees of self awareness? Or is it a strict binary?

I understand how LLMs work, btw.

This depends on your definition of self-awareness. I'm using what I think is a reasonable, mundane framework: self awareness is a spectrum of diverse capabilities that includes any system with some amount of internal observation.

I think the definition that a lot of folks are using is a binary distinction between things which experience the ability to observe their own ego observing itself and those that don't. Which I think is useful if your goal is to maintain a belief in human exceptionalism, but much less so if you're trying to genuinely understand consciousness.

A lizard has no ego. But it is aware of its comfort and will move from a cold spot to a warmer spot. That is low-level self awareness, and it's not rare or mystical.

Oh god. And I thought Amazon’s Mechanical Turk was terrible…

How low can Bezos go? Wtf is wrong with this timeline

Yes. The institution in question is human society. We generally grant the permission to make rational decisions over our lives to other humans who know better that we do or are more skilled than we are.

Sometimes, yes, those humans turn out to have been deceitful or dishonest, but there are mechanisms in place for when that happens.

And yes, sometimes those mechanisms are wilfully avoided by the deceitful. Politicians and rich people are especially good at this.

Guess who's pushing "AI"? The thing that has no contract with human society and cannot be held accountable. And neither will the people pushing it.

This is why we should have as little to do with it - at least as it is in its current form - as possible.

Exactly, I hope they hit a slop wall trying to train these things, replace all its original reference points with slop so it just cascades everywhere

It's the kind of diminishing returns and self poisoning a person would get by only consuming their own waste. Drinking your own piss in the desert will kill you.