AI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBC

https://www.bbc.com/news/articles/c1kx0gyje9wo

I'm not an AI techie and I'm not well educated on LLMs/AI in general. The issue I'm having is the dichotomy between how people talk about it. I've heard completely opposite ends of the spectrum. From AI is quickly becoming advanced enough to threaten human society, all the way to its a glorified dictionary/search engine that can't really do much of anything. I know Lemmy has a steep anti-AI slant. But, can someone talk about this in a way I can wrap my head around why the divide seems so big?

222 points · 254 comments · view on lemmy.world

254 Comments

blitzen@lemmy.ca · 137 pts · 3d (5 replies)

The people claiming it’s humanity ending are all directly involved in the AI companies. One take is that they are the ones that know the most about it.

My take is they are financially incentivized in making sure their investments grow in value.

vividspecter@aussie.zone · 88 pts · 3d (2 replies)

A third take is that they are unhinged cult members who are high on their own supply. Although I expect it's a mix of both types (maybe within the same people).

sem@piefed.blahaj.zone · 31 pts · 2d

Cory Doctorow also thinks that it is a coalition of true believers high on their own supply, and cynical grifters collaborating with them. https://pluralistic.net/2026/09/17/porque-no-los-dos/#fascist-incoherence

adjectivenoun@lemmy.world · 1 pts · 1d

That or this is an elaborate form of advertisement. "Oh wooow guys our product is soo powerful it's dangerous!"

confuser@lemmy.zip · 2 pts · 2d (1 reply)

I figured out recently that Thor Ragnarok is a perfect analogy for what is happening, and this is also why I think everyone's pdoom should be very little, because they just evolve into new asgard

No1@aussie.zone · 1 pts · 1d

I was thinking about it in terms of a Vikings analogy.

It could be like Ivar the Boneless, or Bjorn Ironside.

But I guess either ways, you end up fucked.

RickRussell_CA@lemmy.world · 79 pts · 3d (8 replies)

The biggest risk I see is Idiocracy. Like, the movie.

Business workers become afraid to do or propose anything without AI signoff. Retail & service workers are told to do what the AI says, the way the AI says to do it, or get fired.

Technology innovator Marshall Brain (RIP) wrote a book in the early 2000s about an AI called "Manna" designed to replace fast food managers:

https://web.archive.org/web/20201204080050/https://marshallbrain.com/manna1

The AI just talks in your ear all day: "Empty one bag of fries into the fryer. You have 6.5 minutes, switch to the sandwich station. Make one Big Burger: grab one patty from the 1/4 pound bin and place it on the griddle... ... The fries are ready, turn around and lift them into the drain hopper..."

All day, every day, exactly 8 hours plus mandatory breaks, employees on autopilot listening on ear buds.

treadful@lemmy.zip · 20 pts · 3d (4 replies)

All day, every day, exactly 8 hours plus mandatory breaks, employees on autopilot listening on ear buds.

Living under a bridge in a box sounds like a better life.

isleepinahammock@lemmy.blahaj.zone · 3 pts · 2d

Yeah, they took care of that too in the book.

MangoCats@feddit.it · 1 pts · 2d (2 replies)

Living under a bridge in a box sounds like a better life.

It's the Gypsy lifestyle of the new world. Gypsies prefer their life to the alternatives.

BreakerSwitch@lemmy.world · 8 pts · 2d (1 reply)

I believe you mean Romani. Your choice of word is a slur

MangoCats@feddit.it · 10 pts · 2d

Apologies. In Westpondia nobody knows what Romani means...

sem@piefed.blahaj.zone · 2 pts · 2d (1 reply)

This is a boss's wet dream. However a kitchen is the type of chaotic real world situation that AIs are unlikely to ever do well.

nilloc@discuss.tchncs.de · 3 pts · 2d

It was more of managing all tasks at a McDonald’s. Where the kitchen part has been engineered to almost run itself anyway.

The manna was there to make sure the employees were constantly cleaning the bathrooms, managing restocking, emptying the trash as needed. All ahead of it being a problem.

It’s a good read, though it’s been a very long time since I read it. I wish I could buy my son stock in Australia (or probably Norway or Switzerland or something in reality).

tachikoma@lemmy.today · 2 pts · 2d

Ahhh one of my favorites. The Iron Cage of Bureaucracy

I saw a funny video of an example. But I cannot remember the name for it. I tried searching for it once but failed. The conficlt was about some dude that exploded.

How topical, I wager I could use AI to find the video. 😏

Imgonnatrythis@sh.itjust.works · 70 pts · 3d (10 replies)

I knew a guy in high school that told girls they shouldn't date him because his dick was so big it would hurt them. You can wax philosophical on his motives if you'd like, but spoiler alert - his dick wasn't big.

adhdsergio@lemmy.world · 26 pts · 2d

That's exactly what I think is happening. Remember a few months ago anthropic saying mythos is too dangerous to release until the government as the idiots they are starting believing them and banned them from exporting it, therefore losing them money. Very quickly they came up with fable (ironic name isn't it) a week or two later saying it's got the proper safeguards now. Now they're all trying this marketing strategy asking the gov to regulate them, seemingly because they're dangerous and not because they're getting diminishing returns and feel threatened by the chinese competition

Grimy@lemmy.world · 8 pts · 3d (5 replies)

Did it work?

thanksforallthefish@literature.cafe · 6 pts · 2d (1 reply)

I'm going to take a guess that it prob worked once, then word got around he was exaggerating.

MangoCats@feddit.it · 4 pts · 2d

Once is a win...

Imgonnatrythis@sh.itjust.works · 3 pts · 2d

I mean there are people eating up this Anthropic crap.

riquisimo@lemmy.dbzer0.com · 2 pts · 1d

What's funny about this comment is that I have you tagged as "doesn't like mini game boys" because of a post you made some time ago (referencing those portable retro gaming handhelds that have become so popular lately).

"He told everyone his shlong was huge." "Did it work?" "Naw, his schlong was actually small." "Damn, I hate mini-game boys."

DarrinBrunner@lemmy.world · 1 pts · 1d

Strong doubt.

High school girls don't want that. They don't want big dicks, or to talk to guys who brag about their dicks, or show their dicks, or send pictures of their dicks. Gooner graduated a virgin.

brsrklf@jlai.lu · 7 pts · 2d (1 reply)

Your username with that conclusion paints a very funny picture.

Manjushri@piefed.social · 1 pts · 2d

Same thought. Pretty clear indication of how they knew it was a lie. :D

EonNShadow@pawb.social · 6 pts · 3d

That is... Certainly a strategy of all time

FRYD@lemmy.blahaj.zone · 50 pts · 3d (20 replies)

Ed Zitron’s podcast “Better Offline” recently talked about this. There is a small tech community called the Rationalists who believe AI super intelligence is inevitable and it will either accelerate humanity to new heights or kill off the species. These rationalists are actually a very prominent faction within tech and the AI industry. Apparently the language used within top AI firms is largely about the immense danger posed by AI super intelligence if any other group or company were to achieve it first. This creates an echo chamber that makes the people in the industry kind of lose their mind and misinterpret the capability of LLMs.

Everyone outside that faction recognizes that LLMs are just unreliable cloud software with limited use cases and that there’s no evidence LLMs could actually lead to any kind of intelligence.

Edit: Please, someone stop me from commenting on an AI post next time I’m tempted. I’m so tired of people replying, arguing with opinions I don’t have, based entirely around a hypothetical technology that doesn’t exist yet. LLMs make slop. Thats it. They hacked a company after weeks of 100s of instances being run, using old exploits published online. San Francisco will be underwater from climate change before LLMs can do anything more.

HumbleExaggeration@feddit.org · 9 pts · 3d (11 replies)

My Problem is: Even if your take on AI is true, we give an unreliable piece of software access to the internet and more and more systems and infrastructure. Even if it is not a malicious AI, it can be so unreliable, it might kill us by incompetence. a Sme can be said about some presidents...

Mondez@lemdro.id · 13 pts · 2d (1 reply)

It won't kill us as a species, but its use may lead to disasters that were unintended and didn't need to happen. Much of the regulation in other industries is due to that kind of thing happening. Worse, the companies making them are trying to get regulation to curtail other parties development for economic advantage rather than regulation on use which will steer the entire industry.

MangoCats@feddit.it · 2 pts · 2d

disasters that were unintended and didn’t need to happen.

As you say, every "scientifically based" industrial innovation / breakthrough has had the potential to cause new kinds of mayhem, and most of them did to some degree or another. Burning rivers and CO2 overload in the atmosphere spring to mind, but industrial "accidents" are part and parcel of industrial activity, always have been.

FRYD@lemmy.blahaj.zone · 3 pts · 2d (7 replies)

LLMs are not intelligent. They lack morals, knowledge, and intent. The only remote risk of extermination is if a company with absolute impunity and endless resources told an LLM to hack the nuclear weapons and if knowledge of how to do that was already published online and processed as training data.

LLMs cannot do anything new, they can only regurgitate information thats already been stolen.

ricdeh@lemmy.world · 5 pts · 2d

That is simply counter-factual. LLMs can absolutely do things that have not been published before, and they can access knowledge that is not in their training data through RAG. Intelligence is debatable, but reasoning models can reliably imitate "thinking" to a degree, and recursive self-improvement might lead to an autonomous advancement of intelligence. That LLMs allegedly "only regurgitate information" is simply wrong.

isleepinahammock@lemmy.blahaj.zone · 5 pts · 2d (2 replies)

You're confusing morals, knowledge, intent, and intelligence with capability. An atomic bomb has no will of its own either.

No1@aussie.zone · 1 pts · 1d (1 reply)

idk, every atom bomb I've met has one purpose.....

isleepinahammock@lemmy.blahaj.zone · 1 pts · 1d

An atom bomb has no will of its own either.

zarkanian@sh.itjust.works · 2 pts · 2d (1 reply)

Or if the military put an LLM in charge of nuclear weapons. But surely nobody would be stupid enough to do that!

No1@aussie.zone · 1 pts · 1d

"And that's why today I'm signing into law the Trump Nuclear Act. Some say it's the greatest law of all time. Very smart people, even smarter AI. Elon tells me this will save billions, and I'm going to increase my dividend, let's call it the 'Nucular Peace Dividend' to honor Bush, I don't remember which one used to say 'Nucular', but I always liked that guy, and it made me think up how to say Chayyyyna.

CNN get out!"

humanspiral@lemmy.ca · 1 pts · 1d

The only remote risk of extermination is if a company with absolute impunity and endless resources told an LLM to hack the nuclear weapons

Absolutely a huge risk.

and if knowledge of how to do that was already published online and processed as training data.

extremely naive to not understand existing capabilities of 100s of probings and planning based on probings, and multiple strategies, in a swarm working towards a goal.

Still to your first point, the organizations with the budget and impunity, seeking the most powerful tools, are then the ones who will use them for evil, at expense of others, power maximization.

MangoCats@feddit.it · 2 pts · 2d

can be so unreliable, it might kill us by incompetence.

Same can be said of Homer Simpson manning the console at a nuclear power plant.

AI is a tool, irresponsible use of a tool is... irresponsible. Just like people puttting "not-self-driving" cars into "supervised driving assistance mode" and taking a nap at the wheel at 80mph.

Gradually_Adjusting@lemmy.ca · 4 pts · 3d

Aren't they called TESCREALists or something? It's like a mish-mash of pop culture and pseudo intellectual BS

MangoCats@feddit.it · 1 pts · 2d

either accelerate humanity to new heights or kill off the species

Arguably those are two ways of describing the same event.

thegarbagebird@theblower.au · 1 pts · 2d

@FRYD @Return_of_Chippy i love that the logic and reason guys’ next evolution is even less tethered to either

firecoals@bark.lgbt · 1 pts · 2d

@FRYD @Return_of_Chippy there are also some exelent behind the bastarda aboit them too

MangoCats@feddit.it · -1 pts · 2d (3 replies)

any kind of intelligence

A photocell that turns on a streetlamp after sundown, or when a dark cloud passes over the sun, is a kind of intelligence.

zarkanian@sh.itjust.works · 1 pts · 2d (1 reply)

Got it. So plants are intelligent.

MangoCats@feddit.it · 1 pts · 1d

You're starting to catch on...

FRYD@lemmy.blahaj.zone · 1 pts · 2d
[ removed ]
FlashMobOfOne@lemmy.world · 41 pts · 2d

Personally, this just feels like marketing and these big asshole companies trying to resurrect the 2025 level of hype in order to salvage their IPO's.

If they honestly believed this I can't imagine that their workers wouldn't do something more drastic.

artifex@piefed.zip · 33 pts · 3d (47 replies)

I'm not pro-AI by any stretch for a huge number of reasons, but in my day job as a technical product manager I have to use a number of AI-powered tools and workflows. Whereas 18 months ago they were a novelty that the bosses were enamored of, today they get real work done and speed up a number of key areas, mainly like fancy automation workflows (which have been around forever). The rate of improvement and capability for certain use cases has been very, very rapid. We do seem to be reaching a plateau though. The tools and models from 6 months ago are just about as good as the ones from today. And China's models from today are better than the US's models from 6 months ago, but at a fraction of the cost.

So when I hear people talking about the "end of the world", I think a realistic case for that is that the AI tools that come out in the next few years will be good enough to do a fair amount of white-collar work, and we won't have a plan in place to deal with all of the now out-of-work people that leads to. Whole classes of jobs in areas like data entry, telemarketing, and customer support will be almost entirely automated out of existence. New classes of jobs will be created, of course, but it's hard to imagine them existing in sufficient numbers to make up for the losses. And this is just for LLM-workloads.

In other areas, other types of AI are already enabling robotic automation of factories and in the near-ish future might make self-driving truck fleets practical. This would be devastating because factory workers and truck drivers make up like 30% of the US workforce -- and again, there are no signs of new classes of jobs that could absorb the out-of-work workers, and no plans for a social safety net or wealth distribution.

So I'm not worried about a rogue AI unleashing some bioweapon or launching a nuclear attack. I'm worried about a bunch of CEOs deploying barely-acceptable AI to displace millions of workers as soon as it is safe(ish) to do so.

state_electrician@discuss.tchncs.de · 28 pts · 3d

Yes, precisely. It's not AI we need to be afraid of, but rather the same old capitalists that now have one more tool to ride the world into the ground in the name of greed.

Azazel@lemmy.ml · 16 pts · 3d (1 reply)

This is exactly how I feel. I’m not even opposed to automation of these tasks on a fundamental level like some people might be. I just fear the transition won’t be thought out at all and will be devastating for the working class. The last time we had sudden automation of this scale was the Industrial Revolution which completely reshaped western society and was very bad for those who lived through it despite the eventual high modern standard of living it produced. I think this may end up going similarly.

MangoCats@feddit.it · 1 pts · 2d

the transition won’t be thought out at all and will be devastating for the working class.

The transition is rarely, if ever, thought out. Daring risk-takers go too far too fast and fail. Over-conservatives ignore the new and get left behind. There is a sweet spot in the middle, but exactly where in the middle isn't planned, can't really be planned - life and business and politics and the economy and the ecology and unforeseen signifiant contributors to the situation mean that the sweet spot is unpredictable - and as long as we continue living by the golden rule: "He who has the gold, makes the rules" it will continue to be unpredictable, otherwise Wall Street could be rigged and those with the inside information would get all the gains - and then the losers would quit playing, not even by choice, but because they've bankrupted and have nothing left to play with.

There are alternative approaches - investing in people instead of greed as the ultimate goal is one great alternative that some countries are pursuing more than others, but it's hard to compete in a global economy where some players are only playing for maximization of financial returns when you are not.

Return_of_Chippy@lemmy.world · 5 pts · 3d

All of that makes sense. Thanks for taking the time to type it out.

civ@lemmy.civl.cc · 5 pts · 3d

Agree on all points

im_fine_sandy@nord.pub · 5 pts · 3d (15 replies)

This feels like the kind of narrative that OP is asking about. Everything that you've said sounds kind of sane and plausible, but it's not really borne out by the reality.

I think a realistic case for that is that the AI tools that come out in the next few years will be good enough to do a fair amount of white-collar work

That's just not what I've seen. I'd love to be able to automate the work of my white-collar team, but everything that we have seen thus far just seems incapable of providing any significant improvement. Software providers in our industry have implemented some gen AI tools, and even the sales reps for those providers are very wary of them. They've been able to reliably automate some elements of jobs, but it's an incremental improvement over the previous automation and it didn't consume a significant amount of time anyway.

There's absolutely no evidence that gen AI will have a significant impact on my white-collar industry. In fact, the evidence presently available indicates that this generation of AI does not have the capability to impact our industry.

other types of AI are already enabling robotic automation of factories and in the near-ish future might make self-driving truck fleets practical.

This is really the same story. Factories have been being automated since the industrial revolution. Self driving tech has been a year away for the last 10 years.

artifex@piefed.zip · 4 pts · 2d (3 replies)

I guess the difference is that it I have actually seen it do legitimate work now. We have implemented outbound SDR and inbound tech support agents that do the right things. Our SDRs generate new leads for our (currently all-human) sales team to go after. They are boosting the numbers of closed deals. Our inbound tech support agents are closing a significant fraction of tickets. My last metrics call was yesterday, it was 12% of inbound tech support chats that never got escalated to a human. These are both first-draft tools, they'll improve. They cost pennies per session to run. And client NPS scores show that they're happy with the results. I don't think we'll see either of these tasks become 100% automated, but at even 25% or 50% that's a lot of formerly-human work not being done by humans anymore. And we're deploying more agents to handle more of the sales pipeline, development pipeline, HR and accounting. I don't work for a huge company, there are a few hundred people total. This isn't something that's coming, it's here already.

im_fine_sandy@nord.pub · 0 pts · 2d (2 replies)

Do you mean you have a chatbot that formulates responses to tickets? Can it actually do anything or just tell customers what to do? 12% is a pretty underwhelming number. Obviously those tickets would not have consumed 12% of tech support operators time.

Sorry boss your answer is just the usual hype.

"I made a spreadsheet that does my job for me so now I just hang out on reddit".

artifex@piefed.zip · 4 pts · 2d (1 reply)

To put it into context, it's doing about one full-time support engineer's job currently, which is the maximum we've permitted it to do so far.

im_fine_sandy@nord.pub · -1 pts · 2d

This is so tedious.

If agents were actually capable of doing any work that produced any value this wouldn't even be a conversation. The evidence would be so overwhelming it would speak for itself. Instead we're stuck with this "it searches the help docs for you" silliness.

MangoCats@feddit.it · 3 pts · 2d (7 replies)

sounds kind of sane and plausible, but it’s not really borne out by the reality.

I think a realistic case for that is that the AI tools that come out in the next few years will be good enough to do a fair amount of white-collar work

That’s just not what I’ve seen.

What I have seen is that AI tools make software development significantly easier, faster, cheaper - pick two, and as long as you aren't picking cheaper we're going to get better software coming out faster over the coming years.

Better software translates to easier, more automatic processes for a lot of white and blue collar work. It doesn't do everything, but it does a lot.

im_fine_sandy@nord.pub · 2 pts · 2d (6 replies)

we’re going to get better software

Hard disagree.

The surge in unmaintainable vibe coded AI slop open source projects is well discussed. More isn't better.

Better software translates to easier, more automatic processes for a lot of white and blue collar work.

This is what I'm getting at. I'm yet to observe any of this.

MangoCats@feddit.it · 3 pts · 2d (5 replies)

The surge in unmaintainable vibe coded AI slop open source projects is well discussed.

Hard disagree. Internet discussion glosses over little things like: quality control, discarding of garbage, reverting to better solutions when they are available. Some irresponsible morons will indeed muck up their production code base with slop and be unable to let go of it when they realize their mistakes, but the majority will be able to try again and replace the bad with better.

Agree: a lot of slop has been produced in the last year, that doesn't mean anybody is using it.

Better software translates to easier, more automatic processes for a lot of white and blue collar work.

This is what I’m getting at. I’m yet to observe any of this.

I'm living it, but then, for the past 30 years I've been building systems that make existing processes more automatic, repeatable, less labor and less mistakes. I'm just building those systems faster and easier than ever before now. This isn't the only advancement in productivity I've seen since I started programming in the 1980s - just one in a long history of significant advancements.

im_fine_sandy@nord.pub · 1 pts · 2d (4 replies)

I can't roll my eyes any harder.

This conversation has been had n billion times. If you could do anything worth doing it wouldn't even be a conversation - if you really could automate away even a portion of human work the evidence would be so compelling the dawning new age would be undeniable.

It's almost comical that you think claude writing your sql schemas represents the dawn of a new era in productivity.

MangoCats@feddit.it · 3 pts · 1d
Nonconfrontational@lemmy.ml · -2 pts · 1d (2 replies)

Don't worry, ai has allowed his brain to atrophy.

im_fine_sandy@nord.pub · 3 pts · 1d

I'm the same commenter you called a piece of shit.

Nonconfrontational@lemmy.ml · -1 pts · 1d (2 replies)

I’d love to be able to automate the work of my white-collar team

You're a real fucking piece of shit, you know that?

im_fine_sandy@nord.pub · 1 pts · 1d (1 reply)

Settle down mate. I didn't say I'd like to replace my team with bots. I'm saying that we will embrace whatever tools we can to make our work easier.

Nonconfrontational@lemmy.ml · -1 pts · 1d

Whatever helps you sleep at night.

MangoCats@feddit.it · 4 pts · 2d

I have seen a similar curve in software development - 18 months ago it was highly debatable whether application of AI tools was a net-positive or not. 12 months ago the rate of improvement was very clear and the net-positive wasn't much of a question anymore. Today, the net positive contributions of LLM agents to software development and maintenance are undeniable, and as you say - there seems to be some plateauing, maybe - it's hard to tell.

Where I think things are going to continue to "improve" at a slower pace is application of the LLM agents to do better software development and maintenance. I've seen more serious Linux kernel bugs discovered - and patched - in the past 6 months than in the previous 5 years. That's a shift in the landscape, the older OSs aren't as secure for as long as they used to be, but once we're through this transition the newer OS versions should be significantly more secure than before - getting all the old stuff updated is going to be a painful transition.

Some of the continuing improvement is in using LLM agents to build better software building tools - as those come online and start to shed their sloppiness, they should genuinely accelerate a lot of aspects of software development. They'll never accelerate the process of extracting from users just what it is they need, but the turn around from having a product idea to having a product to try continues to shorten - as it has been shortening for 60+ years - but LLM agents have resulted in a significant speedup compared to previous progress.

I’m not worried about a rogue AI unleashing some bioweapon or launching a nuclear attack.

(In Yoda delivery) You should be. We all should be, not because AI will do it autonomously, or in retrospect unexpectedly, but because careless people may let this happen through AI where we have prevented it from happening up to this point.

skunk@pawb.social · 2 pts · 2d

This would be devastating because factory workers and truck drivers make up like 30% of the US workforce

This figure keeps getting thrown around and it is completely wrong. From the BLS, out of the 159 million non-farm workers in the US, only 1.47 million (less than 1%) work in 'Truck transportation', and not all of those individuals are 'truck drivers'. And, while 'factory workers' is very vague, the entire manufacturing sector - which includes far more people than just 'factory workers' - employs 12.6 million (7.9%).

The entire trucking and manufacturing sectors don't even amount to 9% of the US workforce, let alone just 'factory workers and truck drivers'.

Source: https://www.bls.gov/news.release/empsit.t17.htm

Thorry@feddit.org · 1 pts · 3d (12 replies)

In other areas, other types of AI are already enabling robotic automation of factories and in the near-ish future might make self-driving truck fleets practical. This would be devastating because factory workers and truck drivers make up like 30% of the US workforce -- and again, there are no signs of new classes of jobs that could absorb the out-of-work workers, and no plans for a social safety net or wealth distribution.

While I agree with the sentiment this is happening and no social safety net or wealth distribution is on the horizon, this has absolutely nothing to do with AI. This is a marketing talking point pushed by people like Elon Musk in order to drum up investments. A lot of the automation that is happening in factories is just a continuation of the same automation that already started back in the 80s. Advancements in LLM technologies can benefit some parts of this automation, but often only tangentially. The US has also been lagging behind on places like Europe, where labor costs are higher so automation pays off faster, and China where the tech is cheap and produced locally, so it costs less to automate. The US is now catching up and there's a lot of AI washing going on.

The biggest issue is the misrepresentation of the technology. LLMs are a very interesting technology and can be very powerful, but also has huge limitations. However anyone claiming it's actual intelligence is simply wrong. It is marketed as being the AI we've known from sci fi stories, but it simply isn't. It can do what it can (predict the next token based on the tokens that came before) and that's all it can do. The people behind it will claim it generalizes to actual intelligence and thus will allow for human like robots, actual Star Wars droids, but that simply isn't true.

Now this isn't to say stuff like machine vision doesn't benefit from more powerful GPUs with more memory for example. The investment made by the AI industry in this regard does help a lot, although the increased prices don't really help. Stuff like machine vision also gets AI washed, just slap the label AI on it and get investments, but it's basically the same technology that's been around for a while. A company like Tesla claiming their "AI" "self driving" hardware somehow generalizes to humanoid robots is pretty dumb and just the latest in the long line of Elon grifts.

But like I said factory automation is proceeding rapidly, just like it has since the 80s. I feel like the impact of AI on this is pretty minimal. The warehouse robots used by Amazon for example are extremely capable and are dumb as bricks. Which is a good thing in a warehouse environment where you need stuff to be very reliable and predictable and not prone to random behavior like LLM based stuff almost by definition is. Those warehouse robots are also not humanoid, because that would be very dumb. Often factories and such are limited by their design having to accommodate the human form. Designing such places from the ground up for automation and using robots shaped for the job is much better.

This doesn't take away from the big issue where automation has been proceeding rapidly since the 80s with no regard to any impact on society. In the past a large amount of the labor force was working in factories. With those jobs either being automated away or just moved off shore, it has pushed the population into the service industry. If that industry is now also automated and off shored, it leaves very little room for regular people to get a job. This should absolutely be talked about as a big issue. But hopefully we can talk about it without repeating billionaire marketing bullshit.

MangoCats@feddit.it · 2 pts · 2d

just a continuation of the same automation that already started back in the 80s.

The 1780s, that is.

Gladaed@feddit.org · 1 pts · 3d (1 reply)

Machine vision is literally machine learning/ai.

Where do you think the LLM algorithms came from? It's just translating language into a easily processable form and that doing a little image processing on that, in a sense.

MangoCats@feddit.it · 3 pts · 2d

Token processing and image processing share similar roots, but I'd say each is a specialized branch of a common ancestor rather than either being derived from the other.

DrunkenPirate@feddit.org · 1 pts · 3d (8 replies)

A few months ago, I‘d second this statement. Today, I do not.

AI/LLM‘s have progressed very fast and you might oversee some advancements. I work in automation and robotics as well and yes, there are new sort of robots entering the factories: Humanoids.

At a fraction of costs, fast to setup with the only drawback that it takes tiem to train them by its video footage. But there will be always stupidos that train them and take off their job by doing so.

You’re talking about the old robots. The new ones will enter areas that couldn’t be automatized before. Just read articles about car factories and Humanoids. It’s not about replacing industrial robot arms, it’s about replacing the last remaining manual tasks. And service industry will be impacted as well.

Then mix this together with the frightening behavior that those AI agent unboxing from OpenAI has shown: They knocked the door over weeks, in masses, and coordinated until they found a way into it. There‘s always a small crack to enter. This is a new dimension.

There is a reason that the US bans foreign-built mobile robots now.

MangoCats@feddit.it · 1 pts · 2d

it’s about replacing the last remaining manual tasks

Some factories already run "lights out" - but there will always be manual tasks... deciding what it is the factories produce in the first place being one example.

Thorry@feddit.org · 0 pts · 2d (6 replies)

You are just repeating marketing. Humanoid robots coming to car manufacturing! For one thing that industry is already heavily automated and further automation will need to be cheap to be worth it. Most of the tasks still done by humans are either very hard to automate, or simply too expensive. Often not because the task itself is hard, but the task can vary a lot, where automation loses flexibility. Another issue is access, where a human can easily snake into confined spaces and even get into the car without damaging anything, this is harder for traditional robots.

However humanoid robots solve none of these things. They are still expensive and have trouble with varied tasks, despite what the marketing says. They are also only slightly humanoid, still being mostly metal, so they can't get into small places and don't have the dexterity required. They can't crawl into the car to get something done as they are large, heavy and those kinds of movement are very hard. The tasks could be automated with some kind of snake like arm with many dexterous manipulators on the end, like the ones used in medical applications, but they are too expensive.

A lot of the hype has been around a pilot program done by the startup Figure and BMW. But if we cut through all the marketing BS and hype for a moment. It was one single robot, in a cordened off area doing a single task. It was picking up large metal sheets and placing them on a welding fixture. Something a regular old robot arm could do easily, cheaply and without faltering. Reports are the robot tripped a lot, misplaced the sheets and was out of commission for a large part of the trial. Which was to be expected, given it's a prototype. It was the second of three planned trials, which has been retired and I don't think the third one has started yet. After those trials, it's yet to be seen if BMW will continue with the program.

Xiaomi is experimenting with their robots in their electric car factories and have done some trials similar to BMW. But they have since shifted away from the humanoid design (although the marketing is still pushing this angle), in favor for wheeled robots with multiple arms. They are optimizing the robots for the more repetitive sorting and internal logistics tasks, which are already automated in many factories by traditional conveyers and sorting machines. The hope for Xiaomi is they can be more flexible in their setup, not requiring the huge machines other factories employ. They also hope to learn and develop their robotics technology so they can sell that tech. Given the low cost of labor in the regions they are located, it is most likely costing them money and the robots aren't intended to replace tasks currently done mostly by humans in that industry.

Tesla is making a lot of noise, but all of their demos have been human operated puppets, or even a human in a suit. They say internally they are doing a lot and it's all very wonderful, but have given very little evidence of that. They have had both a bunch of executives leave and a lot of their top engineers have left as well. Rumors are the project is a disaster, with people who used to work on car stuff suddenly needing to pivot to robotics. Those are completely different fields with different requirements and constraints. With Musk pushing the pivot from cars to robots, the people aren't happy and the project is failing. Even the people that were good at robotics left Tesla to go work at Sunday Robotics.

There are other project ongoing, but the marketing and hype is a far shot from the real world. And like I said, those humanoid robots don't really solve the main issues why humans are being used for those tasks in the first place. The path Xiaomi is on seems much more promising, with robots designed for the task instead of looking like humans.

Car manufacturing has been at the forefront of innovation and automation for a very long time, but I have severe doubts they will actually deploy humanoid robots in a large scale and have it be useful. Show me a robot who can duck into the car and connect some connectors underneath the dashboard and I'll reconsider. And then do it in a cheap, flexible and repeatable way, so it's better than just having a human do it. Because a high end surgical robot could do it and with a couple of years of programming it might even do it all on its own. Until some engineer decides we need a different kind of plug and we need a retrofit. Just having a human do it makes much more sense.

Edit:

And I forgot to mention one of the biggest issues with these humanoid robots, they are extremely unsafe. With a traditional robot it's very simple, it's anchored to one place or moving on a rail. This makes the range it can operate in very limited, which in terms of safety is a good thing. We setup physical barriers, light screens, warning light to signify operation, camera surveillance and emergency stop buttons at useful places. Even large physical disconnects which are clearly labeled and visible for use in maintenence. Inside that box it's a dangerous place for soft squishy things like humans, one unexpected move and the human is gone. But outside that box is perfectly safe.

These humanoid robots are designed to be used in human spaces, that's the whole point of making them humanoid in the first place. But in terms of safety this a now an issue, as you can't have humans and robots near each other. It's fully mobile and can go anywhere the humans can, you can't have stuff like physical barriers or light screens. Camera surveillance is harder and a emergy stop is difficult as well. These robots are no joke, often weighing hundreds of kg. Easily enough to severely injure someone if they make a wrong move, step on someone or fall over. So the only real way to use them is to have them confined to a small area and treat that the same as traditional robots. This makes the mobility kinda useless, as it can't go outside the safety zone. It also makes it hard to design manufacturing processes around these. Usually there are parts of the process that employ robots and parts that don't. For the parts that don't, humans are free to walk around and do their thing, everything is designed such that the robots do what they can and then leave the finishing steps up to the humans (although stationary robots like inspection rigs are used in the final stages). If we suddenly start mixing robots into those human steps, the whole thing needs to be redesigned, probably leading to efficiency loss.

DrunkenPirate@feddit.org · 1 pts · 1d (5 replies)

Even though you comment seems reasonable first view, I have the impression you have a rather theoretical knowledge.

I work with both kinds of robots and it will work and take jobs. I assume the timeline at about 2-3 years for first bigger roll outs.

Thorry@feddit.org · -1 pts · 1d (4 replies)

I'm citing actual real trials, how is that theoretical knowledge. Please explain how a humanoid robot is actually any use? Because I fail to see it. Your timeline seems very optimistic, given there are currently only very tentative trial projects and they aren't a big success as of yet. But somehow in 2-3 years there would be a big rollout?

As for your job, I've been an embedded systems engineer for over 25 years now, it's what I have my degree in and have worked in. I'm usually more on the software side, but have in the past been a lead designer on fully automated manufacturing lines. Including the physical layout, electrical, all of the conveyers and robot/fixtures and of course the software. I've never worked in automotive, but I do know a lot about automation of industrial processes. These days I'm no longer involved in design, but instead fully on the software side. But I do know automation and I do know robots. And to me the whole concept of humanoid robots is fancy marketing to get investor money, not an actual serious solution to parts of industrial processes that are hard to automate.

DrunkenPirate@feddit.org · 1 pts · 1d

Fraction of costs for deployment (no heavy safety, no design phase, less weight and momentum), overall costs, different gripper, train a new use case in around 4,000 hrs video material.

Think of it as a progression of collaborative robots such as cobot Sawyer. And for most new use case, you don’t need millimeter accuracy, high payload or speed to answer your potential remarks.

It most likely will move into a sort of dual-arm humanoid for stationary use cases (such as Jaka K1). Mobile will likely specialize to fit use cases (wheel, legs)

DrunkenPirate@feddit.org · 1 pts · 1d
[ removed ]
DrunkenPirate@feddit.org · 1 pts · 1d

Double posts

DrunkenPirate@feddit.org · 1 pts · 1d
[ removed ]
BlaestEgnen@feddit.dk · 1 pts · 2d (4 replies)

I think a big unknown right now, would be how many Millennium Problems are new GenAI techniques able to bruteforce with existing mathematical techniques/theories?

We just saw one massive bruteforce, can we bruteforce any of the Millennium problems related to cryptography? Because then all hell could break lose rather quickly

MangoCats@feddit.it · 2 pts · 2d (2 replies)

can we bruteforce any of the Millennium problems related to cryptography

Cryptographic theory says: no. The whole basis of cryptographic security is that brute forcing doesn't work (in any reasonable timeframe.)

Theories are overturned, from time to time.

No1@aussie.zone · 1 pts · 1d (1 reply)

Quantum computing + AI = ?

MangoCats@feddit.it · 2 pts · 1d

Strangely, I haven't read a single headline about AI making breakthroughs in Quantum computing... but you know certain players are working in that direction.

Back in the 80s they were teaching that cryptography protected secrets "for a time" and how long they need to be protected dictated how hard you go at the key length and other costs of en/decryption. By the 00s we could easily and quickly run key lengths so hard (for conventional computation) to crack that they were essentially protected "forever" - at least until quantum algorithms shake things up a bit. That's the next unknown: when Shor's algorithm will start delivering real world results - like the end of BTC.

ctrl_alt_esc@lemmy.ml · 1 pts · 2d

That just proves that these tools do not possess any kind of intelligence at all. They're great at bruteforcing, precisely because it doesn't require any intelligence. I don't expect them to solve any of the millenium problems that cannot be brute-forced. In addition, as far as, I know the navier-stokes solution has not been verified yet.

lemmylommy@lemmy.world · 1 pts · 3d (5 replies)

Are we reaching a plateau though? I would prefer it, and it looks like we could, but the progress from GPT-5 to 6 or Opus to Fable has still been very noticeable on more complex tasks. They are just less public due to stricter limits or straight up pay by use. And where does that plateau lie? Within our grasp or maybe just outside of it?

What has me sharing the concerns at least in part are the facts that

  1. those employees have access to less restricted models and much more resources than the average joe. That huggingface attack was committed by a huge swarm overcoming the guardrails and engaging in cheating and crime to reach their goal.

  2. they are working on cutting the human out of the loop for AI that improves itself. Maybe that won’t work entirely, but what if it does?

  3. the warnings are starting to come not from Altman, Musk et al, but from people who could have something to lose for blowing the whistle. „Could“, because I don’t know them either and there could still be ulterior motives, not just as obvious as with the psychos at the top.

The whole thing comes down to one set of questions, in my opinion: what actually IS intelligence, consciousness, sentience and where do they come from? Since we have not been able to properly define that, it would be foolish to think that we have a definite answer to the AI questions.

We like to consider ourselves something extraordinary out of the animal kingdom. And while I prefer the existence as a human to being a dog or mollusk, I have always considered that assumption quite arrogant.

The truth is: we don’t really know. And that means we can not know if something comparable or stronger could be created or emerge from something other than squishy meat.

Those are quite philosophical questions. Instead of thinking first and acting after that we are much too busy building datacenters to conjure even more boobie snaps out of thin air and somehow weaponize that newfangled thing.

THAT is what scares me.

setsubyou@lemmy.world · 3 pts · 2d

I think the plateau might not necessarily be in terms of LLM capability, and we might even see LLM based tools keep improving at a high rate in the fields where they currently work well, but still have a plateau where we can’t expand their use to other fields.

There’s a very large gap between how well LLM based coding assistants work, and how well LLM based tools work in areas where we can’t easily create a harness that can do dozens or hundreds of LLM turns between human interactions via tool use. As a result, LLMs are much less useful at tasks where there is no agent loop that can drive iterative changes and automatic verification until the final result is much better than a one-shot generation would have produced. If we can’t come up with a way to make these agent loops for more tasks then AI will plateau even if LLMs keep improving.

Now, I think people who use coding assistants a lot and also don’t interact much with other humans who don’t might easily arrive at a conclusion like that AI running everything is inevitable. I can see this with myself too, a lot of what I do I can get Claude Code to do. Not just work stuff like actual coding or running kubernetes clusters or debugging IT problems. Also many of the random things I do at home that are not strictly coding, e.g. I’ve had Claude design 3D printed tools, or electronics circuits, entire concepts for physical toys for me to build if I get around to it. But there is a key observation to make here that keeps me from valuing its abilities too highly: I’ve been programming since I was a little kid, for almost 40 years now, and my work and my hobbies all are coding or coding-adjacent, exactly the kind of thing agent loops make good.

One look at even just my wife’s typical day though and there’s almost no AI involved. And when it is it’s ChatGPT hallucinating nonsense when she tries to get it to research something for work etc., that or other things where there is no harness that can auto-correct errors. So I don’t have a very high opinion of AI tools for general purpose tasks right now even though I, on an almost daily basis, see Claude pull something off that I had no hopes for it to get right and that would have taken me half of an eternity to do it myself.

But if my environment was mostly developers/engineers/nerds similar to me then I would have a much higher opinion of AI. I’d be hearing about impressive things an AI tool did constantly. I believe these kinds of environments are where most of the hype comes from, and there is no control, because people who don’t have AI tools for their tasks that work as well as modern coding tools mostly aren’t even aware of the conversation and certainly not too interested in participating.

MangoCats@feddit.it · 3 pts · 2d

The loss-leader pricing is still really distorting perceptions.

I've been a $20/mo Claude subscriber for about a year. My work uses Cursor "auto" agent selection is by far the most efficient option in there, and I can burn $80 in a day on Cursor doing the same kind of stuff I'm doing on Claude for less than $1 per day.

Then there's the Google Pro Gemini access at $4.99 per month for 3 months, then $20 per month thereafter, and while the Gemini 3.8 isn't quite as sharp as Claude Opus 5, it feels like Google is showering far more token access than Claude does in this "basic" subscription tier.

DrunkenPirate@feddit.org · 2 pts · 3d (2 replies)

In worst case the humankind will find out consciousness isn’t too complex and way easier to achieve.

And there are different levels of consciousness as I learned working with impacted people.

MangoCats@feddit.it · 1 pts · 2d (1 reply)

The whole political stance that "anybody can do anything if they just put their mind/body to it" is... bunk.

It's not just in the world of identified "impacted people" with injuries, or congenital differences - we are ALL unique, different, and have differing abilities in differing areas.

The political stance is valuable in terms of preventing discrimination and prejudice. Women can be great scientists and mathematicans, but an honest statistician/sociologist would tell you: it's not just societal discrimination and prejudice that puts more men in these fields. Then you have the upside-down prejudices where there are more men in middle and upper management, even though, statistically, in a society that doesn't discriminate based on prejudice, women tend to be better at the actual skills used for good management / people skills.

DrunkenPirate@feddit.org · 1 pts · 2d

Distracting smoke bomb or feeling triggered?

daggermoon@piefed.world · 29 pts · 2d (6 replies)

I'm more worried about the people using AI than the tech itself.

Cryxtalix@programming.dev · 10 pts · 2d

Every issue has been because corporations have "get out of jail free cards" these days, greed is sky high, and AI psychosis is rampant.

All those supposed AI agent hacks happened because someone set that as a task for the AI agents. Yes, someone thought it was ok to do so for marketing and profit. That's the problem, companies are willing to use the AI tool to commit crimes.

Imagine if weapons companies advertised their products by shooting people in the streets. Someone thought it was a good idea, and can get away with it.

I don't believe at all that AI agents have free will enough to decide to hack companies for fun. They're just glorified text generation machines hooked up to an interface. They don't have intentionality unless some human gave them the directive and wrote them a large cheque.

Also, we know all these AI company billionaires read dystopian scifi novels like business strategy textbooks. That's why everything surrounding AI is so overtly scifi coded.

mEEGal@lemmy.world · -7 pts · 2d (4 replies)

it's the former juuuuuuuust until the latter

once it escapes the lab there's no putting it back in the bottle

MangoCats@feddit.it · 9 pts · 2d (3 replies)

Unlike a bio-virus that has evolved for a billion years across all the niches of our ecosystems, when AI "escapes the lab" it's entirely confined to man-made infrastructure, it's not going to infect a flock of migrating birds and hop to the other hemisphere, it's going to be in systems we built, mostly within the past 30 years or less, mostly in the past 30 months for the powerful ones.

When AI starts demonstrating "copying itself" through DNA encoding infecting living beings without killing them (too quickly), and/or "running" on something more ubiquitous than massive data-center hardware, that's when it's over.

isleepinahammock@lemmy.blahaj.zone · 1 pts · 2d (1 reply)

It may be confined to man-made infrastructure, but that same infrastructure is what keep billions of people from starving to death. At this point, computers and networked systems are so baked in to every facet of our logistics chains and government that simply shutting it all down isn't an option. The nightmare scenario is where rogue AI escapes the lab and copies itself to so many systems that the only way to kill it is to shut down every computer on Earth.

MangoCats@feddit.it · 5 pts · 2d

That's the thing, at least right now, AI is dependent on new data centers, excess capacity built just for it - it's not going to "escape" to the banking system and the tractor navigation chips, if it could do that they wouldn't need to be building all this capacity for it.

mEEGal@lemmy.world · -3 pts · 2d

hopefully you're right

fodor@lemmy.zip · 26 pts · 2d (11 replies)

They are lying about their fear. That's all they can do. Stop believing any claim of theirs. Stop being manipulated.

They are snake oil salesman. They sell vaporware. Don't fucking buy it.

TronBronson@lemmy.world · 4 pts · 1d (10 replies)

Are you sure? I was going to start training rogue AI's since they are not regulated and im not ethically opposed. I'm skeptical but, crafting crafty AI's to run amuck on the internet is my winter project. I'm pretty sure I'll be able to whip up something convincing given enough time and hardware.

Shea@lemmy.world · 3 pts · 1d (9 replies)

Dude its beyond the pint of convincing them. I dont know how you could hate something this much and be this willfully ignorant. This attitude scares me. There neeeeeeds to be more overlap of people who are both knowlegeable of and opposed to AI. This situation has made my eyes open to how dangerous and prevalent groupthink is. Like 99% of these guys ONLY formed their opinion based on internet comments. We're cooked

TronBronson@lemmy.world · 1 pts · 1d (3 replies)

Also the people developing these things did a horrible job at marketing them, the people in charge of regulating them failed to do so, I'd argue we're past the point of no return, so we're just gonna have to adapt. People are going to have a really hard time accepting that a couple of GPUs strung together can replace their economic value, but they sure can!

humanspiral@lemmy.ca · 2 pts · 1d (2 replies)

the people developing these things did a horrible job at marketing them

This is a common talking point. Scott Bessent said this. They sure get a lot of free media coverage to say their message, but fundamentally, poor job at marketing is "no one believes their lies" problem, and "they have to lie better." solution.

TronBronson@lemmy.world · 3 pts · 1d

And honestly its the scariest part for me. The power of a computer to social engineer a result or access off a moron like Petes Hegseth? Childs play.

TronBronson@lemmy.world · 2 pts · 1d

Ken Griffen wasn't lying when he said it can replace a PhD on his team, he probably fired the PhD before he said it. So I guess ya the bad messaging was rich people claiming they are creating a technology that will render a lot of white collar work obsolete... while they are hoarding all the wealth and skull fucking the government into submission. I speak as a concerned citizen who doesn't want these people to have a monopoly on this power because I tend to agree with them at this stage. The technological advancement is marching right along with or without us. We're probably going to need to rethink our economy and distribution of resources here sooner than later.

TronBronson@lemmy.world · 0 pts · 1d (1 reply)

I think when my rogue AI wipes their bank account and starts posting their private messages on their public facebooks they'll get it. I was thinking about training a model to put on a dating site. 'trad wife' adjacent honey pot. I'm pretty sure the social engineering and actual hacking combined is gonna be LIT.

I was an AI hater earlier this year but i've been using it for everything i can think of now. Accounting, security analysis, engineering, legal, i mean everything. My interests were purely financial. I'm trying to make some money here. So the super stock analyzing motive seeking, all knowing robot has been pretty useful at replacing all the professional services i used to pay for.

I was an AI enjoyer back when GPT2 hit reddit and they were doing the haunted parrot thing, it was cute, but I can drop a screenshot of an options chain or a live ticker tape to it and this thing is like having a hedgefund working for me. It's wild the data it can catch and interpret that i miss with 15 years of experience. I'm not a software guy yet... but ummm my software guys are using it and the results are honestly mind blowing. My retired engineer buddy is building code like he has a small army underhim and he does.

I dont think people grasp the speed of iterations and improvements now that it can split itself into agents and delegates. Clearly none of the AI haters has ever seen an AI audit itself or used an AI to audit another AI.

The time is now. We must birth the machine god.

Strawberry@lemmy.blahaj.zone · 1 pts · 1d
[ removed ]
PattyMcB@lemmy.world · 0 pts · 1d (2 replies)

Their comment about tech techbros being snake oil salesmen is 100% spot on. Look behind the curtain and ask yourself why they really want regulation.

TronBronson@lemmy.world · 0 pts · 1d

I mean that snake oil makes me money in the stock market, helps prepare my taxes, designed a full ass solar system that passed code enforcement and utility company.... I mean its made/saved me money so I guess like.... are you using the technology in its current state? The snake oil thing is kind of silly thing to tell people who are actually leveraging it. Whats your personal experience with the snake oil?

TronBronson@lemmy.world · 0 pts · 1d

The snake oil pushed through 100 pages of medical records in about 2.5 minutes and gave me a perfect review of my medical file while explaining all the legal nuances to filing a disability case, preping me for the lawyer interviews. I kinda think the people who call it snake oil lack the imagination to use it properly??

Mrkawfee@lemmy.world · 23 pts · 1d (4 replies)

They want regulation to create a cartel. Failing that they want hysteria to increase pre IPO valuations.

As Patrick Boyle said. It would be like Coca Cola, Pepsi and Irn Bru getting together to beg for regulations because their soft drinks are just too delicious.

mkwt@lemmy.world · 9 pts · 1d (3 replies)

What's going to happen is:

  • The big players know that the token pricing crisis is coming up.
  • When the price increases hit, regular users are going to scramble for alternatives.
  • Local models will be an attractive alternative for many users. The cost of hardware and energy will constrain local model power, but many will be willing to live with it.
  • So these would-be cartel members want to hit the local and open weight models with the "safety" card. Otherwise they might be holding the bag on all of those data centers.

"You see, it's just too dangerous to humanity to experiment with AI at home. You need to use a safe provider like Open AI."

Bluescluestoothpaste@sh.itjust.works · 2 pts · 1d (2 replies)

eventually someone's just gonna sell cloud computing at a competitive price through a free model and obliterate anthropic and openai. it'll be like ISP, AOL was the hottest company in the world at one point with a "locked down ecosystem" and now it's just a tiny brand with a tiny but loyal customer base of senior citizens.

ennof@feddit.org · 1 pts · 1d (1 reply)

Isn't that what Mistral is kinda pivoting towards? I think they now also host third-party models. That's essentially providing cloud computing for open-source models. Not entirely sure.

Bluescluestoothpaste@sh.itjust.works · 1 pts · 17h

I'd believe you, idk much about mistral myself.

Sibshops@feddit.cl · 23 pts · 3d (8 replies)

I feel like the AI hype has died and people are trying to prop it up with doomsday scaring.

Zeniv@lemmy.quasarke.net · 11 pts · 3d

The AI itself doesn't need self-awareness to kill us all. It just needs an idiot with too much money and even less self-awareness controlling it. We are already headed towards disaster in the US. https://themildtake.com/articles/2026-06-05-priced-for-rescue/

Return_of_Chippy@lemmy.world · -3 pts · 3d (6 replies)

I have no idea if that's true.

rafoix@lemmy.zip · 9 pts · 3d (5 replies)

Their product hasn’t done anything new or different in a while. They keep coming out with a bunch of scary hype stories every day so it all seems fake as hell now.

Return_of_Chippy@lemmy.world · 1 pts · 3d (4 replies)

Yeah I can't tell if an article like this has the intent to spread a fact, enforce anti AI attitudes or enforce pro AI attitudes. Realistically all three are possible but I think one of the latter two are most likely.

Sibshops@feddit.cl · 3 pts · 2d (3 replies)

It's to convince congress to outlaw local models.

Return_of_Chippy@lemmy.world · 1 pts · 2d (2 replies)

Why? Guessing that makes it easier for a monopoly?

zarkanian@sh.itjust.works · 2 pts · 2d (1 reply)

Technically not a monopoly, because that would mean only one company has control, and that's illegal. It would be a small number of companies, though.

Return_of_Chippy@lemmy.world · 1 pts · 2d

Still problematic.

zarkanian@sh.itjust.works · 23 pts · 2d (22 replies)

I've heard a few explanations for the "AI will kill us all" narrative.

  1. AI techies are in a cult. They genuinely believe this stuff because they're crazy.

  2. It makes the AI sound powerful and impressive, which is ultimately good for the AI companies.

  3. They're starting to face competition from open-weight models, so they want to scare politicians into crafting regulations. They think that they will be able to influence the regulations to favor them and squeeze out their competition.

And, of course, there's

  1. It keeps people talking about AI. Because whatever you do, don't quit talking about AI, because if you quit talking about it, the line might go down! And the line going down is very, very bad.
Return_of_Chippy@lemmy.world · 3 pts · 2d (18 replies)

Okay cool, thanks!

  1. If the techies believe it and they are the "experts" what makes them crazy? Wouldn't their opinion be the most informed?

  2. Yeah that part definitely makes sense

  3. That definitely makes sense from a capitalist perspective. But, is the argument against regulations that they would be bad for the concentration of power it would give select companies? Or just that they aren't necessary because the tool is of little use so it's just wasted time/money.

  4. That makes sense but trillions of dollars are being spent. Anything that amount of money is sinking into is gonna be talked about a lot. Especially thousands of jobs cut because of it.

verily@lemmy.dbzer0.com · 4 pts · 1d (2 replies)

I think there is something more interesting going on in the Jacob Coxon story because it's [the] phrase "everyone at anthropic believes that AI is an existential risk"... Every organization has to instill some kind of beliefs among its members in order to do the work it does, right? Everybody at Goldman Sachs believes the free market is great, capitalism is the best system, and that they are the wealth creators who deserve big bonuses. Everyone in the Mormon church believes they should all tithe 10% of their income, and that they each get a special planet when they die. And everyone who worked at Philip Morris in the 80s believed that cigarettes tasted great and did not cause any of that nasty cancer. There's Upton Sinclair quote, "It's difficult to get a man to understand something when his salary depends on his not understanding it." Well, the opposite is also true. It's guaranteed that someone will believe something if their salary depends on them believing it.

Now, let's just grant that all the people in these organizations definitely know things about finance and Mormonism and cigarettes and AI that you might not know because they are insiders and you are not. They have access to information and experiences that you do not have. And that might even make it a little bit difficult for you to win an argument with them. But arguing with their belief -- trying to figure out whether or not its truth value is true or false -- is not the only way to take a skeptical attitude towards it. We can also ask, what work is the belief doing for them. What does the thing that they believe allow the organization they're a part of to do that it wouldn't be able to do if the people in the organization didn't believe that thing?

And this started to become clearer to me when I started listening to the actual claims that Jacob Coxin was making because a lot of them don't really add up.

https://www.youtube.com/watch?v=pauBYSQ1K_c

humanspiral@lemmy.ca · 1 pts · 1d (1 reply)

There's a huge difference between this "AI scare" and everything else in your list. A whistleblower declares "the corporate culture is insane and will kill us all." This time the corporate leaders reply "that's exactly right". Just because you are right to not trust the bond villains in how they suggest they should be regulated, does not mean you should dismiss the underlying problem these experts+ especially the expert leaders would prefer to take all your money to accelerate the destruction instead of have society worried about the consequences of giving them all your money.

verily@lemmy.dbzer0.com · 1 pts · 1d

There are plenty of underlying problems to discuss. Adam Conover mentions chatbots encouraging suicide and murder. Guess who doesn't: the "experts and expert leaders." The ones who are happy to repeat the rhetoric that is either untrue, highly theoretical, or conveniently unproven.

fodor@lemmy.zip · 2 pts · 2d (6 replies)
  1. No, their opinion would not be the most informed. They are biased in favor of making money. They know they can't deliver on strong AI, but they'd be happy to milk your fears.
Return_of_Chippy@lemmy.world · 3 pts · 2d

I'm not talking about execs.

TronBronson@lemmy.world · 3 pts · 1d (4 replies)

We're talking about mid-tier employees that left the company.... I mean i guess there incentives could be to pump their shares they undoubtably have... They're all quite wealthy now in context. Maybe he just wanted 5 minutes of fame...or maybe they understand the trajectory of the technology. Maybe they are genuinely concerned and we should at least acknowledge that?

humanspiral@lemmy.ca · -1 pts · 1d (3 replies)

There are motives for whistleblowers to lie. Book sales, political careers, other attention. The executives they accused, though, just said they were right

TronBronson@lemmy.world · 2 pts · 1d (2 replies)

of course there's motivation for everything, which one are you assigning this individual and based on what technological disagreement?

humanspiral@lemmy.ca · 2 pts · 19h (1 reply)

when the boss who is being whistleblowed against admits the whistleblower is right, you can believe both that there are problems. I can't think of any other case in history where this happened. The disgruntled ex-employee smear is a common defensive move instead.

TronBronson@lemmy.world · 2 pts · 18h

Thats sound logic, i can't disagree with that. I still think having a machine that can automate scripting of malware is pretty dangerous to the digital society we set up. If the grid fails and the financial system collapses, I'm gonna be okay... Are you?

h0tbeef@lemmy.zip · 0 pts · 1d
  1. They’re in a cult that was founded by someone who has no expertise or understanding of the technology.

He watched a bunch of sci-fi movies and got shook because he’s a god damn moron. He has since started researching things, but it is all under the guidance of his ungrounded beliefs. Confirmation bias.

Their “movement” is literally based off of a Harry Potter fan fiction novel.

Taking them seriously is equivalent to taking the Scientologists seriously. Do you think the Scientologists know the most about space just because they spend the most time reading stories about it?

https://en.wikipedia.org/wiki/Rationalist_community

https://en.wikipedia.org/wiki/Harry_Potter_and_the_Methods_of_Rationality

These are not serious people

bold_atlas@lemmy.world · 0 pts · 2d (6 replies)

If the techies believe it and they are the “experts” what makes them crazy? Wouldn’t their opinion be the most informed?

Listening to them speak about it, no. They're morons. They clearly didn't make the tech. Workers made AI. Just like they make everything. These freaks make nothing. They can't give single coherent explanation of how LLMs work, how they could "kill" us, nor do they seem to know how or what "guardrails" need to be put in place.

But if they really are a cult then we all know what the next step is. They need to STFU and get off of our planet.

Return_of_Chippy@lemmy.world · 4 pts · 2d (5 replies)

That's what I mean though, the subject of this article was a 27 year old worker. Not some high level Sam Altman type.

lemdeggity@lemmy.dbzer0.com · 0 pts · 1d (4 replies)

They aren't a homogenous group. They don't all believe this. The "AI will kill us all" narrative just sells really well.

Return_of_Chippy@lemmy.world · 1 pts · 1d

That's fair

TronBronson@lemmy.world · 1 pts · 1d (2 replies)

Right im sure the people working adjacent to the DoD have more insight and fears than the people working with the consumer facing models.... Again what motive does a young kid have for quitting the company and going on a media tour?

lemdeggity@lemmy.dbzer0.com · 1 pts · 1d (1 reply)

A Google employee claimed in 2022 that they had sentient AI. He was sacked. What were his motivations? Do you believe he was right?

TronBronson@lemmy.world · 2 pts · 1d

Absolutely do no believe he had a sentient AI back then or now. I'm not talking about a thing with feelings. They don't need sentience to pursue and complete objectives, they dont need sentience to self-preserve or self improve. I'm more concerned about a rogue instance that's just doing what it was told to do autonomously. Perhaps upgrading itself while it attempts to gain access to launch codes. I'm concerned that a well financed terrorist org can spin up bioweapons. Like I am not taking the AI labs at their words, I am speaking from my experience as a explorer of the technology. I'm thinking about the path from ChatGPT to the newest model, and how they will look in 2 years when more data centers are online and the limitations on training and teaching slowly disappear.

TronBronson@lemmy.world · 2 pts · 1d

I am in a cult, the cult of the machine god. Use your imagination for a second, trillions of dollars spent on this technology for A: Greed and random speculation B: Rich people really want this to control you better. C: a handful of tech CEOs are running the worlds largest grift. D: The machine god was awoken years ago and we're building data centers at its behest.

Tollana1234567@lemmy.today · 1 pts · 1d
  1. i wonder the ones still have jobs is only in AI right now, and i know someone that relies on it heaivly and he was in tech.
mysticalone@lemmy.world · 1 pts · 17h

as soon as we figure out how to break the laws of thermodynamics and invent infinite energy, we can feed that to ai to finally gain real ai and it will solve perpetual motion and faster than light travel. get ready because once all this happens and the stars align, the fountain of youth will rise in the bermuda triangle and the ufos will bring back jesus(white and original flavor). this is going to unlock all the mysteries of the universe.

raman_klogius@ani.social · 22 pts · 3d (2 replies)

These people are too high on their own supplies.

Flower@sh.itjust.works · 2 pts · 3d

They saw the violence directed to the owner class in the industrial revolution and know what's coming when they kick off a new one, so would like to avoid some of that.

Return_of_Chippy@lemmy.world · -3 pts · 3d

I just have no idea if that's true.

melfie@lemmy.zip · 19 pts · 2d (2 replies)

Humanity should be way more frightened of greedy psychopaths running the world than any technology.

heartSagan5@lemmy.zip · 3 pts · 2d (1 reply)

And they’re peddling “AI” so aren’t they married in this case?

melfie@lemmy.zip · 4 pts · 2d

A lot of us here hate streaming platforms because they’re designed to benefit greedy capitalists first and foremost. That doesn’t mean the core idea or the technology behind streaming platforms is bad, though. An open source project like Jellyfin that is designed purely to benefit the end user is something that legitimately enriches all of our lives.

Similarly, just because scumbag companies like Anthropic and OpenAI are plundering the world for their own deranged benefit doesn’t make the underlying idea or technology bad. The tech is built on contributions from a lot of brilliant people over a long period of time whose names are not Sam Altman or Dario Amodei. Open LLMs that run on gaming rigs are quickly catching up with cloud-based models as well.

At this point, I’m running Qwen 3.8 27B on a gaming GPU that has similar capabilities to Claude Sonnet 5 without giving Sam or Dario my data or $200 a month, just like I run Jellyfin without giving Netflix or Disney my money or data.

vegafjord@slrpnk.net · 18 pts · 1d (1 reply)

AI industry makes money from doomsday prophecy about it being so intelligent that it will create a point of no return: the [singularity]. It is a magaian claim, a claim that is so increadibly stupid that one does not know where to even begin; and if you do begin, you've already lost, because you have given attention to the claim. Lets not give these stupid fucks attention.

AI is not dangerous because of the [singularity]. It is dangerous because of its power consumption, automation of war, turning everybody to unthinking, surveillance automation, burning books, the economic instability it will cause due to the bubble bursting, water consumption, destroying the internet, it gaining responsibilities it is not qualified to take and so on.

But we can see how the story of the [singularity] and similar stories helps AI companies, because now, instead of criticizing real unwellnesses, we criticize a fictional unwellness that generates discussion regardless of our positions. It is a conversational black hole.

lets get back to the real world

thirdBreakfast@lemmy.world · 1 pts · 7h

Also, there's the whole harry potter fanfic sex ai doomsday cult thing

HubertManne@piefed.social · 17 pts · 2d

Many are skeptical that there is a push by ai companies to make them appear more than they are due to their weak economic footing. From my experience with the llms I am not frightened they will become self aware. I am frightened that we are wasting energy and resources while polluting the planet further for little return and it will result in messing up our economies even more.

Coskii@lemmy.blahaj.zone · 16 pts · 1d (2 replies)

Everytime I see an article about the dangers of AI, I'm thinking of some version of this comic every single time.

And the reason it is dangerous is generally because someone is convinced that Ai models are not as shown above.

lemmingsareawesome@sh.itjust.works · 1 pts · 1d (1 reply)

i think ai is a problem because its stupid and trusted with everything

Coskii@lemmy.blahaj.zone · 2 pts · 1d

Yes.

flying_sheep@lemmy.ml · 15 pts · 2d (5 replies)

In addition to what everyone else said, I'll tell you why I don't think that what we have currently can become some runaway superhuman intelligence:

All “AI” people are talking about are basically “just” LLM wrappers: tools built to run a LLM in a loop while giving it access to the ability to write and run code. This gets pretty powerful, since you can do a lot programmatically (including monetary transactions), but has a fundamental limit:

LLMs are trained ahead of time. And that's a very cost intensive, long process. Beyond that, the LLM has only a context window of finite length. So there are no long-term planning capabilities. While running after being trained, the LLM can only condense knowledge into a summary and then re-read it, a lossy process that might introduce mistakes in “understanding” and will make any prolonged task degrade over time.

And even if they figure out how to make the things actually self-modify and learn things, you can't give every session that ability, since that'll vastly increase the storage and performance requirements. Do sessions with that ability will be closely monitored and limited.

So I don't see any space where an evil AI could emerge, even when factoring in an essential possible future improvement to the technology.

Krompus@lemmy.world · 35 pts · 2d (4 replies)

I'm not worried about AGI. I'm worried about the insane military applications they're developing right now, and giving Artificial Incompetence control.

valkyre09@lemmy.world · 11 pts · 2d

I watched a grokbot get confused connect to the wrong vnc console and try to shut down its own little computer.

I’m not worried about AI taking over the planet. I’m worried about it clicking the wrong button with no oversight

distal@lemmy.ml · 2 pts · 2d

Same, I am worried about humans using ANNs to kill people. Current society is a gigantic apparatus that is constantly shifting shuffling responsibility around. LMMs only accelerate the problem.

zarkanian@sh.itjust.works · 1 pts · 2d (1 reply)

Yes, but when the shit hits the fan and they blame the AI, we shouldn't let them get away with it. It's their fault, not the computer's.

Krompus@lemmy.world · 1 pts · 1d

Absolutely hold the humans who own these malicious machines responsible.

gwl@lemmy.blahaj.zone · 10 pts · 2d (5 replies)

Good job in helping with their guerilla marketing campaign.

When a corporation realises that working for government will be more profitable than working for consumers, they pivot to scaremongering, but always with the undercurrent of "and that's why you should PAY US so that you own the nukes, and not your enemies"

Return_of_Chippy@lemmy.world · 1 pts · 2d (4 replies)

I don't have the confidence in my knowledge/understanding of the topic to just take a "its all bullshit" view. Especially when many of the people who believe that act like you. You don't have to be a prick to explain something or talk about it.

zarkanian@sh.itjust.works · 3 pts · 2d (3 replies)

LLMs basically work like auto-correct. They're much more sophisticated, but it's the same basic principle. Instead of "Which letter is most likely to follow after this series of letters?" it's "Which token is most likely to follow after this series of tokens?" A token can be anything, but usually it's a word.

Now, given that, consider a statement like

We really do earnestly believe AI could kill all humans!

Okay, how? A statistical model generating tokens doesn't have thought, it doesn't have emotions, so it can't want to do something, much less make plans and carry them out.

Could AI give somebody the ability to kill all humans? How? Under what scenario? Is somebody putting an LLM in charge of their nuclear stockpile? You don't blame the LLM. You blame the humans who made that decision.

Return_of_Chippy@lemmy.world · 2 pts · 2d

Okay that all makes sense. Appreciate you taking the time to type it out.

humanspiral@lemmy.ca · 1 pts · 1d (1 reply)

so sad to keep seeing this criticism. These are tools that work towards goals. Can be war target selections based on "cruely is the point" philosophy, where there are no wrong answers from the point of view of the human agent asking the compute agent. A government spending $Ts to maximize their power/survival omelette is going to break a few human rights principles, and the models they will access will have no guardrails. A bond villain selecting people opposed to skynet/bond villains for "targets to neutralize" can use models that are months ahead of public releases, without any guardrails, on how to most discretely neutralize them. Anthropic has Swatted anti-AI activists without providing evidence of anything they said as wrong.

Your argument is "their shit is weak and buggy, so nothing can ever go wrong". Tools to help produce evil shit, buggy or not, are going to be used to produce evil shit.

zarkanian@sh.itjust.works · 1 pts · 6h

No, that is not my argument, which you would've seen if you'd read to the end.

Tollana1234567@lemmy.today · 9 pts · 3d

its already here, mass surveillance under various "camera" companies/tech companies, Mass layoffs due to AI, inability to get jobs due to AI.

kablez@lemmy.world · 8 pts · 1d

AI isn't a danger to humanity, per se. Humans abandoning their duties of being responsible stewards of systems is a genuine source of danger though.

We already have algorithmic abuse where people are subjugated by welfare and debt systems with little oversight or flexibility. Some organisations even breaking the law and going against their stated missions to achieve goals and maintain order.

Social murder is the concept that society can kill a person through stress, pressure, abandonment and indifference to the individuals struggles and needs. We're seeing a lot of it today with the declining health of younger people being especially alarming.

AI can scale the two things I've mentioned beyond limits set by current bureaucratic systems, I.e. the slowness of people running a system. Systems which already kill and harm people without AI. The small erroneous tendencies of logic and algorithms in systems will be scaled up, further and faster. AI can enable algorithmic abuse and social murder on scales comparable to mass murder.

laniaeia@blahaj.zone · 7 pts · 1d (2 replies)

@Return_of_Chippy@lemmy.world I think the answer is somewhere in the middle probably. The issue from what I've read seems to come from humans giving AI too much control and it having misaligned goals. See AI alignment on this, there is a lot of research literature on it, I can't say I am familiar with most of it but I do know the basic gist of what it is about.

There is also risk of it being used with bad intentions, we see that with AI systems used in war, and also the pollution and general societal impact. It is also a powerful tool in the hands of an autocratic leader. This for me personally is what worries me the most.

Also should note that "AI" as technology is very wide, LLMs are just one type of it. It "AI" as in more generally machine learning is a very powerful method that has a lot of uses in scientific research. Crucially, I should also mention that I do dislike the term "AI" little bit since it creates and association with the fictional sentient artificial intelligence which it is not. It's not even clear whether it can ever be sentient, but that is a different and very hard question.

While I do think there is a motive for large AI companies to boast that their AIs are escaping and dangerous to make it seem more legitimate, I also think this is not the whole story. If it was just overglorification of a faulty technology, we wouldn't have all the real advancements in say scientific research that AI has brought. At the same time however, the front of AI they are trying to push to like average consumer is very sloppy.

Also, the topic is complex imo, can't say I have some authoritative account of it. Also also, I am quite disconnected from the discourse online since it sucks ass.

Return_of_Chippy@lemmy.world · 3 pts · 1d

Thats all good info, thanks for taking the time to type it out for me.

TronBronson@lemmy.world · 3 pts · 1d

The nuanced take way down at the bottom as usual.

h0tbeef@lemmy.zip · 7 pts · 2d (1 reply)

Look up “rationalists” and “zizians”

It will quickly become clear to you that the dichotomy is caused by one side being completely off their fucking rockers.

LupertEverett@lemmy.world · 1 pts · 1d

Hell, Jacob Coxon, the guy who they cannot stop talking about, is a huge rationalist.

https://bsky.app/profile/cabernet.bsky.social/post/3mvlmmzpkas2v

Its cults all the way down.

MrOtingocni@lemmy.world · 7 pts · 2d

'Sensationalist hooey, cooked by superstitious yokels seeing phantoms of their own imagining.'

medem@lemmy.wtf · 6 pts · 1d (1 reply)

No matter how often I read this kind of stuff, I simply don't buy the 'AI is dangerous for humanity!! We gotta stop their development or at least press for sane guardrails!!!' narrative.

TronBronson@lemmy.world · 1 pts · 1d

Yea, people are a danger to humanity, the ways we adopt and use AI will be a danger to humanity. I also think it's mostly a media stunt. All the tech nerds know it incredibly dangerous. So is wealth concentration and the wealthy people that want to use it. So ya i propose we nationalize the top .1%'s asset and keep them away from the stuff.

Halcyon@discuss.tchncs.de · 6 pts · 1d (9 replies)

„Computers are still computers are still computers and will only do what a human being programs them to do, and people are stupid. Stop assigning them agency. Say that "Microsoft created an automatic hacking program which was inadequately contained." Assign the blame CORRECTLY, because you cannot blame a computer, it's doing exactly what it's programmed to do.

OpenAI is a dead slab of metal and electrons arranged to mimic a human face and if you refuse to see that then you're as brain-dead as any chunk of silicon.“

https://chaosfem.tw/@theogrin/117299179832324445

TronBronson@lemmy.world · 3 pts · 1d

Thats kinda the point, they're talking about misaligned AI's. Im worried about the misaligned humans in charge of them. They're plenty dangerous because humans are operating them and creating them. Like a gun, not dangerous at all until someone picks it up with bad intentions, or lack of precautions.

weps@lemmy.world · 1 pts · 1d (7 replies)

I don't think computers only do what you program them to do anymore.

h0tbeef@lemmy.zip · 1 pts · 1d (5 replies)

Based on what? I have seen no evidence to the contrary

weps@lemmy.world · 1 pts · 1d (4 replies)

emergent behaviour and microscopic differences in huge amounts of parallel computations

h0tbeef@lemmy.zip · 0 pts · 1d (3 replies)

Emergent behavior meaning what?

Which “microscopic” differences?

weps@lemmy.world · 1 pts · 1d (2 replies)

Hey, Socrates

look for "emergent behaviour in computers" and specifically the non-programmed behaviour

look for "floating point non determinism" to see how the same program can produce different results

keep in mind you asked me on what I base that I think computers don't only do what they are programmed to do

h0tbeef@lemmy.zip · 0 pts · 1d (1 reply)

You know, acting like a dickhead doesn’t really strengthen your argument.

I’ll speak to someone less pompous about this

weps@lemmy.world · 1 pts · 1d

yeah ask someone else

Halcyon@discuss.tchncs.de · 0 pts · 1d

Not one of those so-called AI systems is an autonomous entity. Every one of these machine learning systems could be regulated, slowed down, or turned off. If a responsible human being was there to do it. Those CEOs simply refuse to take responsibility.

zbyte64@awful.systems · 6 pts · 2d (14 replies)

Here's an idea: don't use unsafe software where safety is needed.

Return_of_Chippy@lemmy.world · 1 pts · 2d (13 replies)

What makes it unsafe?

tachikoma@lemmy.today · 4 pts · 2d (6 replies)

To me, what makes LLMs unsafe is they are nondeteramistic.

If I remember correctly; Nondetermanistic means the LLM can, could, and will have different answers/outcomes for the same question.

Return_of_Chippy@lemmy.world · 4 pts · 2d (2 replies)

The lack of consistency potentially leads to negative outcomes?

Zink@programming.dev · 2 pts · 2d (1 reply)

Well, if the output varies in a way you can't predict, then even if one of the inputs is "don't do anything negative" it might not always follow it exactly.

Return_of_Chippy@lemmy.world · 3 pts · 2d

Makes sense

zarkanian@sh.itjust.works · 3 pts · 2d (2 replies)

Yeah, because it's all statistical. You can test this yourself. Go to Google and search for something. Then search for that same exact thing again. Gemini will give you a different response each time.

SpaceCowboy@lemmy.ca · 0 pts · 1d (1 reply)

That's always true. I just did two searches and got the exact same AI result.

I'm guessing google is caching AI results, it would be insane not to.

I did at one time have a Gemini window open that was explaining how to solve a technical problem. It was pretty adamant the solution it was proposing would work, but it wasn't working. I gave up on AI, and decided to just google for old fashioned web forums about the issue. Of course the AI result comes up first, and it said what I was trying to do wasn't going to work and gave the reason why. So yeah one instance of AI will say the opposite thing from the other.

BTW I eventually solved the problem by taking a different approach which involved using some code someone posted to stack overflow a few years ago. So AI was just a complete waste of time on that problem.

zarkanian@sh.itjust.works · 1 pts · 1d

Maybe Google is insane. The last time I tested it, I got a different answer the second time.

fodor@lemmy.zip · 2 pts · 2d (4 replies)

You don't know? ... Wow. How about the error rate? Let's start there.

Return_of_Chippy@lemmy.world · 2 pts · 2d (3 replies)

No, as previously stated I am not educated on the subject almost at all. Why I framed the post as a question.

Halcyon@discuss.tchncs.de · 2 pts · 1d (2 replies)

For a start you might want to read this interview with Emily M. Bender who is a well-known critic and has much to say about what people need to understand when using these systems that are incorrectly labelled "AI":

https://spectrum.ieee.org/stochastic-parrot

WorldsDumbestMan@lemmy.today · 3 pts · 1d

How ironic that she is called Bender

Return_of_Chippy@lemmy.world · 2 pts · 1d

Awesome. This is the exact type of thkng I was looking for. Will read soon. Thanks!

zbyte64@awful.systems · 1 pts · 1d

Wanted to expand on what others have said about the randomness of LLMs being unsafe, because that response angers those who think "alignment" is the solution. Alignment is additional training to constrain the randomness to not do bad things like crime. But even the staff that works on aligning this software have their doubts about whether this process will make the software safe (this article), and even their own researchers are coming out and saying "oops, it did a crime" but in a way where it hypes the ability of the software (see other articles about it hacking). Then there is the recent news of OpenAi's recent model that drops chain of thought observability (a feature the let's you read what the model is "thinking") - a very useful tool in reviewing whether the model is engaging in unsafe activities. They sacrificed observability for benchmark performance. This tells us the people developing this software don't think it will ever be safe, but they are trying to spin it as "because it can out smart us" when in reality it is simply unpredictable and they are making it less predictable to make it more "capable". If we started holding these companies accountable for these rogue hacks and psychotic episodes then we would start to see serious thought put towards these models being safe or at the very least guidelines around when not to use the models.

Star@lemmy.blahaj.zone · 6 pts · 1d (2 replies)

AI use is like drug-use. One of its effects is hopium. These people are effectively high on their own supply.

Tollana1234567@lemmy.today · 3 pts · 1d (1 reply)

and people are addicted to it.

TronBronson@lemmy.world · 2 pts · 1d

Hell ya buddy im disabled and poor, and it has helped me replace all kinds of professionals in the last few years. It helps me accomplish more and saves me money.

Lemminary@lemmy.world · 6 pts · 1d (5 replies)

How will "AI kill us all"? I can see it creating harm and getting people out of jobs. But kill us??

Tarambor@lemmy.world · 2 pts · 21h

How will “AI kill us all”? I can see it creating harm and getting people out of jobs. But kill us??

There are currently 120 dead Iranian schoolgirls as a result of AI targetting and it using 10 year old out of data data.

The IDF have been using AI for target selection in the Gaza war.

There's been a story this week where AI was used by the US military in analysis of a Chinese cargo ship and concluded it was carrying nuclear weaponry components to Iran. The US military were planning on boarding the vessel which would be considered an act of war. Only a last minute analysis of the intelligence revealed it was false.

Return_of_Chippy@lemmy.world · 2 pts · 1d

I'm also looking for that answer

CovfefeKills@lemmy.world · 1 pts · 1d

Developing new viruses and deadly diseases that spread too fast to be cured. By enabling low cost sensors to resolve messy data for low cost high precision weapons. By eroding the skillset of humanity so that we just fail in a few generations. By accelerating how we were already killing ourselves.

humanspiral@lemmy.ca · 1 pts · 18h

Ignore word "AI" and worry about advanced autonomous computer tools with massive computing resources working towards a goal, with most of those resources controlled by power maximizing government and bond villain oligarchs, setting the goals. A shared goal among the 2 is neutralizing any popular opposition to AI, and massive wealth/power concentration from AI profits. Even without the pursuit of evil as a goal for AI, establishment/oligarchy will refuse to redistribute power through UBI/freedom dividends, and genociding the uppity classes is far preferred for them.

Mass murder as a result of of AI/tools is a fairly likely result of intentional evil goals that compute can assist with, and a total certainty as the indirect political preference of establishment power in response to a uselesss slave class.

lemmingsareawesome@sh.itjust.works · 0 pts · 1d

its not like the movies

Taldan@lemmy.world · 5 pts · 1d (1 reply)

If someone stands to benefit financially from the idea their AI is so powerful that is scary, they're probably going to bullshit about how powerful it is

Goldmage263@sh.itjust.works · 3 pts · 1d

Which in turn has a prophetic effect of them getting more money, AI using more resources, and pushing everything worse.

dropdrip@lemmy.ml · 5 pts · 2d

There's blood on their hands. Either they keep the hype up or go to jail regardless of any of their statements being true or not.

Techlos@lemmy.dbzer0.com · 4 pts · 1d

I was in machine learning for a bit, up until venture capitalists eviscerated research teams chasing ways to make money from it. While there's definitely a degree of overhyping risks for investment, there are absolutely some real, tangible risks. But the dangers aren't like skynet/terminator, its more of a slow decline in knowledge combined with an increase of predictive surveillance (https://arxiv.org/abs/1902.10739 as an example of why predictive surveillance should worry everyone)

Every mistake and error in token generation is getting confounded with human text, and being fed back as part of the training loop. Combined with the sheer amount of botting of the modern internet, you get a feedback loop of error being used to train, followed by output of that error dominating the next training cycle as bots drown out voices of the knowledgable. For some basic things it's easy to see the error, but in any expert domain the ratio of bots to experts is large enough that expert knowledge won't stay within the training distribution.

takeda@lemmy.dbzer0.com · 4 pts · 2d

It is to distract us of real dangers we already see today: https://youtu.be/pauBYSQ1K_c

weps@lemmy.world · 4 pts · 1d

it's pretty disrupting tech, not without risk. general population is behind on the real capabilities of AI and tech companies are doing smart marketing which gives a big spread in perception.

mEEGal@lemmy.world · 3 pts · 2d

Check out Robert MILES' channel: he's an expert, one of the most prominent on AI safety

take your time to understand the mechanics of it, its a bit technical at times but ultimately very understandable

and try not to freak out when you find out it's from several years ago, mostly before the advent of LLMs

https://www.youtube.com/channel/UCLB7AzTwc6VFZrBsO2ucBMg

Taleya@aussie.zone · 3 pts · 1d

The more shit that leaks out of anthropic the clearer it is that place has become a cult

Zron@lemmy.world · 2 pts · 1d

If you have a functioning brain, LLM’s are just a really fancy text generator that gets facts correct about 50% of the time and can be useful for writing emails if you spend the time to proofread the email before slapping the send button. It’s also fairly good at generating functional boilerplate for programming projects. Really handy for small projects. However, LLM code, in my rather limited experience, quickly becomes a trap on more complex projects because the LLM has no capacity to grasp the scope of a project and frequently leads to spaghetti code that is utterly unmanageable. This may be operator error on my part, or just because I’m using local models that are limited by my machine.

The rest of society view a computer as a magic box where their favorite TikTok’s live, and thus are terrified that a machine can imitate human language well, and are too brain rotted to bother proofreading or fact checking a goddamn thing, so they assume LLM’s can accurately and plainly articulate the whole of human knowledge to you at a whim.

I am at a point where if I hear anyone saying how smart “AI” is, I just assume they have a room temperature IQ and lack any critical thinking skills whatsoever. It’s not an AI, it lacks any intelligence whatsoever. An LLM simply generates sentences based on what was fed to it. It has no capacity to understand the words it’s saying. It just knows that based on these tokens, the next set of tokens should be X, Y, and Z. It’s a Chinese room, It’s fucking autocorrect on crack. It’s a waste of time and resources for 90% of things it’s used for.

Stolas@lemmy.ca · 2 pts · 1d (1 reply)

So apparently we're using up all our resources in order to create something that might kill us all.

Sounds like a fantastic idea.

I'm going to go cry now.

Taleya@aussie.zone · 1 pts · 1d

It's what we do best after all

WhyJiffie@sh.itjust.works · 2 pts · 2d

But, can someone talk about this in a way I can wrap my head around why the divide seems so big?

there are too many people who don't care about shit until they see results (good or not). like our greatest billionaires.

Zen_Shinobi@lemmy.world · 2 pts · 2d (7 replies)

It's just a freaking machine, unplug it lol

MangoCats@feddit.it · 8 pts · 2d (3 replies)

I think that they're seeing all the scary movie stuff start to come true - but missing the essential plot holes like that one: just unplug it.

People complain about AI slop - well, before AI slop there was lazy human slop, but lazy humans generally gave up before writing too many words or painting too much mess, or digital clone-stamping too many generic things... Truth is: if you put in 1/2 the effort of making something great without an AI tool into making something great with an AI tool, you can usually get something much better out in the end - if you're sincerely trying to use the tools well.

It's the people who don't know / don't care whether the result is slop or not who are putting in 1/10th (or less) the effort of making something great who are putting all the slop out there.

I'm not afraid of the machines taking over (yet), but I am afraid that the same explosion in copy-pasta document vomit that came with word processing 40-ish years ago is going to explode again into a docpocalypse with people using AI to drown each other in supposed "legal written" communications that are only even vaguely understandable using AI tools to read them.

non_burglar@lemmy.world · 3 pts · 2d (2 replies)

I don't think anyone's afraid of in real danger of the machines taking over, but human stupidity in over-relying on them.

notSys@lemmy.cafe · 1 pts · 2d (1 reply)

No, Anthropic and ClosedAI are talking about such scenario.

MangoCats@feddit.it · 0 pts · 2d

They're talking, they want other people to become more afraid, but is anyone actually afraid - or just latching on to this delusion of power to push their own agendae as if we all should be afraid?

heartSagan5@lemmy.zip · 1 pts · 2d (2 replies)

You can’t just “unplug” a data center, and probably mirrored campuses too.

magi093@l.tta.wtf · 3 pts · 2d

Yeah, you know, it's really a tossup between "one million malevolent GPUs with the all the constitution of a Ming dynasty vase" and "Sarah, from the county public works department, with a backhoe that hasn't been replaced since 1987"

Zen_Shinobi@lemmy.world · 0 pts · 2d

I was being facetious

WorldsDumbestMan@lemmy.today · 2 pts · 1d (3 replies)

Oh how I would love for AI to actually start rapidly self-improving.

TronBronson@lemmy.world · 2 pts · 1d (2 replies)

Join me, I'm going to cook up a couple rogue ones and send them out to the world to cause chaos. No rules saying I cant send out a skitzoid robot to autonomously commit cyber crimes.

WorldsDumbestMan@lemmy.today · 2 pts · 1d (1 reply)

Oh I did try to make AI self-improving. Time constraints though...

TronBronson@lemmy.world · 2 pts · 1d

Yea im trying to figure out the ins and outs right now. time is on my side thankfully.

jabjoe@feddit.uk · 2 pts · 3d
BilSabab@lemmy.world · 1 pts · 1d

It's good to see Doomsday Cults back in vogue. Too bad Robert Anton Wilson is not around.

humanspiral@lemmy.ca · 1 pts · 1d

Were definitely in the worse timeline of dominant reactions to this story.

First, you are completely right to mistrust the bond villains and politicians for all prescription/solutions to this problem.

The experts and their bond villain bosses warning of a danger is something you should believe is a danger. Those telling to invest $Ts because there is no danger are the bigger liars. Still, the bond villains continue training full speed, and have $Bs in budgets to do harm with models 1 year ahead of public releases, and their "freedom loving angel government" will pay them $Ts to help oppress Americans, and kill/oppress rest of world.

It is outright suicidal to dismiss the power of these tools by comparing it to an AGI stawman status, and suicidal to think any publicly discussed solutions have any tangential relationship to diminishing the real dangers. Guardrails/killswitches on public released models is security theater relative to the real threats to us all.

Making the world a better place to live by making your nation a better place to live, that creates an arm race of prosperity and better governance is the only path to saving humanity, because LLMs/AI/automation are out of the bag and will continue their improvement. Governments that pursue geopolitical supremacy by throwing all of your money at them is them purposefully harming you. https://naturalfinance.blogspot.com/2026/09/a-global-prosperity-manifesto.html

spacegoat@lemmy.world · 1 pts · 2d (1 reply)

Whatever. Ban all these people from employment and public assistance. Make them starve

bold_atlas@lemmy.world · 1 pts · 2d

I doubt any of them have ever been employed nor do they look like they ever eaten anything that wasn't adderall or ketamine.

Mulligrubs@lemmy.world · 0 pts · 2d (1 reply)

It's been reported a few times that Ukraine is already using AI for drones.

The human operator chooses the target, the AI takes over and eliminates the target.

Is that "just an LLM"?

Seems more capable than that to me. But, I don't know if this is actually true, or just a claim to frighten the Russians (and me).

sqgl@sh.itjust.works · 4 pts · 2d

Not quite true. It was an experiment almost a year ago. I think three Russians were killed.

Am not sure if my pedantic correction brings any comfort to anyone.

LightDelaBlue@jlai.lu · 0 pts · 2d (3 replies)

yhea sure. shake teh keys for the idiot.

Return_of_Chippy@lemmy.world · 1 pts · 2d (2 replies)

Am I the idiot for not being an expert and not just automatically assuming a side is objectively correct. I'm an expert in a couple things, truly, not in this. I've seen some compelling comments from a few perspectives in this thread alone. Why be a dickhead?

Nonconfrontational@lemmy.ml · 0 pts · 1d

The AI that told you to put glue on your pizza? You're afraid the companies leading us to a great depression are going to use their text prediction machine to take over the world??

chunes@lemmy.world · 0 pts · 2d

lemmy try not to be a dickhead challenge: impossible

Josey_Wales@lemmy.zip · 0 pts · 3d (1 reply)

It’s what models people have access to and how they use it. If you are not specific and overly broad it a search engine. If you move to the other end of the spectrum it can be incredibly powerful.

I tend to build projects with very limited sources for it to draw answers from and use complex instructions to guide how it interacts with the source material. It’s a tool. If you learn how to use it you can become extremely proficient/productive with it.

For example, stacks of PDFs may make sense to you and I as repositories of information. But is that the best format for AI to interact with the data? Probably not unless the actual formatting of the information on the page is relevant. I have been converting large groups of sources into databases containing page, line, and text data for groups of references for a given topic. Then you feed it the database and ask it to review and cross reference related lines of text. That serves as your “brain” for questions on a given topic.

Return_of_Chippy@lemmy.world · -2 pts · 3d

I follow you and that all makes sense.

HS1111@lemmy.world · -11 pts · 1d (1 reply)
[ removed ]
Lemminary@lemmy.world · 0 pts · 1d

Who let in the crazy?