MIT report: 95% of generative AI pilots at companies are failing

https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/

94 points · 9 comments · view on lemmy.world

9 Comments

audaxdreik@pawb.social · 19 pts · 1y (2 replies)

I don't know why I expected a Zitron-esque lambsating from fortune.com, but reading the article is disappointing,

But for 95% of companies in the dataset, generative AI implementation is falling short. The core issue? Not the quality of the AI models, but the “learning gap” for both tools and organizations. While executives often blame regulation or model performance, MIT’s research points to flawed enterprise integration. Generic tools like ChatGPT excel for individuals because of their flexibility, but they stall in enterprise use since they don’t learn from or adapt to workflows, Challapally explained.

Sure. Let's blame anything but the AI 🙄

UnexpectedBehavior@lemmy.world · 2 pts · 1y (1 reply)

This is a hill I'm going to die on. The article is mostly right. It's not AI that fails, it's people using it wrong. If you had a manual shift car and drive 3000 Miles in 1st gear you'd probably had a slow journey at high cost and now a broken car. Would you blame the car or the driver?

AI is not going to replace us, people how know how to use it right will. Firing 80% of your developers and solely use AI will be your demise. Training your devs in AI usage and develop strategies for integration into existing code will multiply your productivity.

AI is just a fucking tool that is not going to go away. Use it right and it will benefit you

audaxdreik@pawb.social · 8 pts · 1y

I'm a pretty big AI-detractor and my concern is not that AI will replace us, it's that it will do serious damage to any number of systems by being forced into them. This isn't a bottom up training problem, it's a top down problem. It's an implementation problem. Even if I allow your argument for a moment that it can be effective when wielded in specific situations, it does not belong in all places. I think the scope of where AI can be effective is extremely more limited than what is currently being pushed.

If you had a manual shift car and drive 3000 Miles in 1st gear you’d probably had a slow journey at high cost and now a broken car. Would you blame the car or the driver?

I see analogies like this being made all the time and they're fallacious, they just don't hold water. I have not seen proof that AI (as it currently exists) is that sort of technology. Imagine seeing a NINETY FIVE PERCENT (95%) failure rate anywhere else and thinking, "Clearly it's not the tools." This article is delusional.

ThirdConsul@lemmy.ml · 5 pts · 1y (3 replies)

They seem to be focusing very much on NOT using LLMs yourself, but buying an SaaS offering providing LLM instead.

95% are failing, and:

Companies surveyed were often hesitant to share failure rates,

Oh. So it's at least 95%?

Edit:

Source is MIT Nanda project - isn't that an university project? I can't access it, can anyone share? I'm curious about the methodology and data set

Zerush@lemmy.ml · 2 pts · 1y (2 replies)

Here you can access some material, but the article in question is for members only, it's a document stored in Google Docs

https://nanda.media.mit.edu/

You can find other pages with this article, but all these pointing to Fortune as source.

ThirdConsul@lemmy.ml · 2 pts · 1y (1 reply)

Let me clarify what I wrote: I can't access the google stored document because they send it only to approved people. Do you have access to it and can you share?

Zerush@lemmy.ml · 1 pts · 1y

Well, I have an paleolitic Google Account

I don't believe that it's enough to access the documents
I don't know if it is still valid in case de remember the password
I prefer to suck my elbow before activating the account again for it

I hope you understand me

gpowerf@lemmy.ml · 2 pts · 1y

AI has all the signs of another dot-com bubble. The hype is outpacing reality, with investors and MBAs convinced that slapping “AI” on anything will make it better, or even worse replace people altogether. The truth is, AI is a tool, not magic. The bubble will burst, yet the underlying technology will endure and keep reshaping how we work.

Zerush@lemmy.ml · -10 pts · 1y

Wrong Title, what fails isn't the AI, but a bad management of the AI by pure company interests of people which don't have any tecnical skills (apart of the poor reliability of some of the employed AIs)

Summary by Andi

A new MIT report reveals that 95% of enterprise generative AI pilot programs are failing to deliver meaningful financial impact in 2025, with only 5% achieving rapid revenue acceleration[^1].

The research, titled "The GenAI Divide: State of AI in Business 2025" from MIT's NANDA initiative, analyzed 150 leadership interviews, surveyed 350 employees, and examined 300 public AI deployments. The core problem isn't the AI models themselves, but rather a "learning gap" between tools and organizations[^1].

"Some large companies' pilots and younger startups are really excelling with generative AI," said Aditya Challapally, the report's lead author. "Startups led by 19- or 20-year-olds have seen revenues jump from zero to $20 million in a year. It's because they pick one pain point, execute well, and partner smartly with companies who use their tools"[^1].

Key findings include:

Vendor-purchased AI tools succeed 67% of the time, while internal builds succeed only one-third as often[^1]
Over half of AI budgets go to sales and marketing tools, yet the highest ROI comes from back-office automation[^1]
Companies are reducing headcount through attrition rather than layoffs, particularly in customer support and administrative roles[^1]
"Shadow AI" - unauthorized use of tools like ChatGPT - is widespread in enterprises[^1]

The report identifies several critical success factors:

Empowering line managers, not just central AI labs, to drive adoption
Selecting tools that can integrate deeply with existing workflows
Focusing on specialized vendor solutions rather than building in-house
Targeting back-office automation for highest returns[^1]

Looking ahead, pioneering organizations are testing agentic AI systems that can learn, remember, and operate independently within defined parameters, pointing to the next evolution in enterprise AI[^1].

[^1]: Fortune - MIT report: 95% of generative AI pilots at companies are failing