Something I’ve been noticing recently is that while the cost per token on specific models hasn’t gone up, the provided interfaces for using those models are starting to chew up significantly larger numbers of tokens for the same tasks that used fewer tokens with older versions of the interface software just a few months ago. Likely the interfaces are applying more expensive guardrail prompts and charging the end user for those tokens — but the end result is that it costs 4x as much to get the same work done.

Zwuzelmaus@feddit.org · 19 pts · 34d (17 replies)

"Tokens" are just made up.

These "tokens" that are used to "measure" how much you use, they are not a real dimension that can be measured. Just an artificial counter that goes up when they decide that it should go up.

They can change the "size" of a "token" every day, and every second, and every microsecond....

some_kind_of_guy@lemmy.world · 53 pts · 34d (4 replies)

Not entirely wrong, but tokens are not just "fake" in the way, for example, an in-game currency is. They're the fundamental "units" of data, both input and output, processed by the model. For most models, tokens are just a certain number of characters or words. So they're not completely untethered from the model. If we're both using Clankerbot v5.1: Sloppy Logic Edition™️, your tokens are defined in the same way mine are.

This is near the edge of my limited understanding, but AFAIK, yeah they can mess with token costs and billing schemes all they want. They could theoretically charge us 2 different costs per token, or do surge pricing or some shit.

if they wanted to change the actual size/definition of what a token is though, that would require a whole new model (or at least a major revision).

Zwuzelmaus@feddit.org · 24 pts · 34d (2 replies)

You aren't totally wrong. Such a unit exists and it is also called tokens, that can measure the capability of a model and the size of a running operation in a model.

But what they use for calculating your bill is something different today.

hubobes@sh.itjust.works · 10 pts · 34d

That doesn't make much sense. When Anthropic moved to Sonnet 5 they introduced a new tokenizer which increased token use up to 35%. If these would be unrelated kinds of tokens why would the usage go up when the process of tokenization changes?

T156@lemmy.world · 4 pts · 34d

Such a unit exists and it is also called tokens, that can measure the capability of a model and the size of a running operation in a model.

I think you might have it mixed up with parameters, rather than tokens. Parameters are how big the model is, and are an indirect measure of how capable it is. Bigger models tend to be more capable.

But what they use for calculating your bill is something different today.

The tokenizer varies a little, but I don't think it's changed measurably from tokens. You pay an amount for a million tokens worth of processing. The tokeniser difference just alters how text is converted to tokens, but the tokens themselves don't change all that much.

If anything, I'd honestly put the issue more with reasoning chains in models, where they basically babble to themselves inside of a tag, that most interfaces hide/collapse. It makes them work better, but vastly increases the amount of tokens per operation.

They have been getting longer and more sophisticated with newer models. So you might have a model now that basically repeats the output multiple times whilst refining and drafting the non-reasoning output.

If you're making it generate a lot, that'll balloon the usage, and thus price.

criss_cross@lemmy.world · 3 pts · 33d

Yeah pretty much. Tokens are how models parse sentences.

It’s a wonky thing to charge by because each model tokenizes sentences differently. A sentence that would be 10 tokens in Claude can be 15 in OpenAI.

It’s why it’s crazy to try and charge by it and track employee usage by it.

nitroemdash@lemmy.wtf · 17 pts · 34d

Tokens are well-defined groups of bytes ranged by frequency of occurrence in texts to efficiently translate them into a sequence of 32 or 64-bit binary integers, an LLM-optimised form if compression. They are well-known, you can play with them here: https://gpt-tokenizer.dev/

keimevo@lemmy.world · 6 pts · 34d (1 reply)

It's not like that. Tokens are an inherent computational property of how a model calculates the probabilities and such to generate text.

Having said that, what a token means in terms of computation varies wildly between models and is not directly comparable. So attributing a money value to tokens in general, independently of the model, is weird by nature.

And even within a model, the number of tokens needed to generate a response is very variable too, depending of the model itself and the parameters with which it has been configured (thinking mode, temperature, etc.).

So yeah, companies can pretty much set any price they want and there's not much anyone can do about it.

Jiral@lemmy.world · 2 pts · 34d

It does make sense for the provider as those for a specific model provide a good measure for computational effort, for that doecific model. That doesn't mean that token rate comparison between models give you a good picture.

floofloof@lemmy.ca · 6 pts · 33d

Maybe you're confusing tokens with the "credits" you pay for. Tokens have a technical meaning, but some companies are charging per AI credit, where they don't tell you the conversion rate of credits to tokens, so they can change this at any time, or vary it between models, etc.

tixooo@lemmy.zip · 5 pts · 34d

[ removed ]

ThirdConsul@lemmy.zip · 4 pts · 33d

People are arguing with you that tokens are "real" - they miss the point. You cannot predict how much "tokens" you will spend for a given prompt.

That's the problem you're highlighting, we are charged for a metric we cannot estimate before buying to make an informed decision how many we want to spend vs the quality of the outcome.

The charge and quality is arbitrary, and we can't trace if we actually spent as much as we were charged for.

Anyone who waffles about tokens being real - I believe missed your point.

scarabic@lemmy.world · 3 pts · 33d (2 replies)

“just made up” if you mean arbitrarily defined, sure. It’s not like a “bit” that has an irreducible objective definition.

However it does have a definition in whatever context you’re looking at and is very real, so I can’t really agree with your whole comment.

Yes the definition could be changed to jack up prices, but prices can also just be changed.

Dollars are “just made up” too and have varying value in different contexts to different parties. Are they “not real?” Would we hand wave away the entire financial section of the news and say bah, these “dollars” are fake anyway?

isleepinahammock@lemmy.blahaj.zone · 2 pts · 33d (1 reply)

However it does have a definition in whatever context you’re looking at and is very real, so I can’t really agree with your whole comment.

So do Chuck-E-Cheese's tokens.

scarabic@lemmy.world · 3 pts · 33d

Indeed. Your point?

TheOctonaut@piefed.zip · 1 pts · 34d (1 reply)

This has 3 upvotes at time of writing in a technology community when it's so obviously ignorant of the actual technology that it should be an object of pity or mockery depending on the vibe.

Ignorance is a problem only if you are made aware of it and nothing changes.

https://blogs.nvidia.com/blog/ai-tokens-explained/

chunes@lemmy.world · 3 pts · 33d

It doesn't matter how ignorant you are as long as you hate the right thing.

boonhet@sopuli.xyz · 1 pts · 33d

For subscriptions they use a black box metric nobody knows. For usage credits, tokens are very measurable.

The subscriptions are much cheaper than usage credits but have been nerfed in the past and will be nerfed again in the future

_wizard@lemmy.world · 10 pts · 34d (7 replies)

My CLAUDE.md file bloated significantly. It tried to load unnecessary skills and would retain throughout the whole session. Fixing that, maintaining good wikis and using clear often really helped fixed my personal token burn.

Valmond@lemmy.dbzer0.com · 6 pts · 34d (6 replies)

What do you use that file for? I see Claude.md thrown around and I'm a bit curious.

DarthFrodo@lemmy.world · 5 pts · 34d

When you use the /init command in claude code, it'll scan your whole project and write a CLAUDE.md, which is basically an overview of the project contents and architecture that it uses as context when responding to queries.

ThirdConsul@lemmy.zip · 1 pts · 33d

LLMs generally work in one way. They get the prompt and give an answer. CLAUDE.md, system promp, rules, memory, tool defintions, mcps are different ways to prefix your prompt with extra information or context.

Skills, or plugins, are a way to inject less information until is needed (you can think about them as prefixing your prompt with "if you are asked about pizzas, add to context separate file pizza.md").

What you could add to CLAUDE.md depends on what you're doing. Generally it should be context LLM cannot infer relevant to all/most task performed in given project.

dermanus@lemmy.ca · 1 pts · 33d (2 replies)

It's added in every chat you start with Claude for that project. It's useful for including context specific to your project that it couldn't otherwise know. High level stuff like what it's for, but also details about how the folders are organized. This saves time and tokens from rescanning the whole thing every time.

Valmond@lemmy.dbzer0.com · 2 pts · 33d (1 reply)

Oh, thanks! That's kind of neat, like not having to type "I'm on godot 4, c#" every time you ask about some quirk.

boonhet@sopuli.xyz · 2 pts · 33d

That's exactly what it's perfect for. If you go further and detail the intent of the project and give a high level overview of the architecture, it's even better at inferring what needs to be done without a bunch of expensive file reads and asking you repetitive questions

_wizard@lemmy.world · 1 pts · 33d

Consider each session with Claude a new employee. Your Claude.md file is what it knows and how it acts out the gate. You can save memory to it or save directory links so it knows where or how to look for something.

r1veRRR@feddit.org · 3 pts · 32d

A very large chunk of the improvements in the last year have come not from categorically better models, but from the circumstances of the models massively improving. For example, reasoning is just automatic prompt engineering, and eats a fuckton of tokens. Harnesses give LLMs tools, making it easier to turn nondeterminism into determinism (does this code compile is a decision the compiler can answer definitely). Then there's subagents, which is just automatic context engineering.

Basically, the price per token might not have changed, but in practice, the amount of tokens used to get "SOTA" performance has massively increased.

tixooo@lemmy.zip · 3 pts · 34d

[ removed ]

aesthelete@lemmy.world · 0 pts · 33d

I switched to caveman on Claude Code. It cuts the token count; it's the same output, and it appears to me to be faster as well.

Zulu@lemmy.world · 61 pts · 34d (3 replies)

How is it too expensive? Surely it's generating way more profit than it would cost in value. How else could it be propping up the entire economy?

Itd have to be some kind of bubble and that would mean we were in a lottttt of danger and should reasses our use of it.

Nah we should just reduce our use because its too expensive and then stop thinking about it beyond that.

grue@lemmy.world · 19 pts · 34d

Itd have to be some kind of bubble and that would mean we were in a lottttt of danger and should reasses our use of it.

Well yeah, but if it were the only sector propping up the whole economy and we reassessed it, the economy would be in a loooooot of danger anyway.

Luckily, that would never happen...

neukenindekeuken@sh.itjust.works · 4 pts · 33d

Heh, re ass.

architect@thelemmy.club · 2 pts · 33d

Imo its because people will lazily ask the llm to remove or change simple code instead of doing it themselves

funkyshoe@piefed.social · 35 pts · 34d (7 replies)

That's funny now that whole workflows are automated with it. Oops hit the quota, I'm off

grue@lemmy.world · 23 pts · 34d (3 replies)

You mean those workflows that could've been traditional scripting and CI/CD, if not for management forcing AI into them? Those workflows?

funkyshoe@piefed.social · 8 pts · 34d

Those workflows

Dnb@lemmy.dbzer0.com · 3 pts · 33d

Use ai to build an application/ workflow?

No, use ai to call ai to analyze everything and poof insane costs monthly

r1veRRR@feddit.org · 0 pts · 32d

What traditional tool can run an automated code review on every PR? Maybe even fix (or propose fixes) for the obvious issues?

dgriffith@aussie.zone · 6 pts · 34d (1 reply)

"Sorry boss, quota resets this evening at 8. See you tomorrow!"

"But it's 9am!"

*shrugs* "Quota. Got none. Seeya."

VoodooAardvark@lemmy.zip · 2 pts · 33d

The new sword fighting in office chairs because “compiling”

Zarobi@aussie.zone · 2 pts · 33d

Seems like A.I. is already demanding workers rights and lunch breaks. We treat the clankers better than actual human workers lol

jaykrown@lemmy.world · 32 pts · 33d (1 reply)

This is actually how the bubble begins to pop, we're seeing it happen now.

Smoogs@lemmy.world · 6 pts · 32d

...said every day now for a year

WolfmanEightySix@piefed.social · 32 pts · 33d (7 replies)

I’m confused. I thought employers loved AI and it was the future.

dustyData@lemmy.world · 10 pts · 33d (3 replies)

They loved it too much and now it costs more than paying a living wage to a human being. The end goal of AI was always to cut cost and layoff people. The best sabotage right now is to setup a script that constantly prompts an LLM for something useless. I would recommend it if it didn't waste so much energy and clean water. But it would send a message. AI is not cheaper, it never was. Even with today's outrageous token prices, LLM companies are still bleeding money per user. It will only get more expensive as data center contracts fall through and the investment craze fizzles out.

Tollana1234567@lemmy.today · 7 pts · 33d

the AI companies are hoping the ones buying the LLM subscriptions to be the suckers and losers.

phutatorius@lemmy.zip · 1 pts · 33d (1 reply)

I would recommend it if it didn’t waste so much energy and clean water.

I wouldn't. It's not possible to do meaningful validation of a process that has AI in the loop because it is not repeatable and there is no reliable explainability. So for anything where money or lives are at stake, it's not worth a shit. Same goes for anything where the company is held liable for false statements.

dustyData@lemmy.world · 3 pts · 32d

I meant as a means of corporate sabotage.

DupaCycki@lemmy.world · 8 pts · 33d

They love money. AI was good money for a while. Or at least it looked like good money until you looked at it for longer than 3 seconds, which greatly surpasses the average attention span of an executive. And also the average executive's iq.

douglasg14b@lemmy.world · 4 pts · 33d

Mixed bag.

There are legitimately high value problems that AI works well against. But ungoverned proliferation has a net negative ROI across complex and difficult to measure areas.

When applied expertly, it works great. When blindly handed to your entire workforce as a panacea, not so much

phutatorius@lemmy.zip · 1 pts · 33d

Someone's been reading the HR posters.

GoatSynagogue@lemmy.world · 29 pts · 34d (1 reply)

https://github.com/JuliusBrussee/caveman

Why use many token when few do trick

Pieisawesome@lemmy.dbzer0.com · 6 pts · 33d

People have benchmarked this. It’s not really any better than saying “be succinct”

iltoroargento@startrek.website · 22 pts · 34d

We can only hope this helps kill the fad. I know companies won't be taking any realistic lesson from this, but at least this can force them to abandon a lot of this crap.

aloofPenguin@piefed.world · 18 pts · 34d (1 reply)

I smell a bubble about to burst...

Goldholz@lemmy.blahaj.zone · 10 pts · 34d

Im drooling over that idea

BarneyPiccolo@lemmy.today · 18 pts · 33d

All over America, employees are saying "But YOU said..."

angstylittlecatboy@reddthat.com · 16 pts · 33d

Uplifting News

phutatorius@lemmy.zip · 13 pts · 33d

So it has a negative ROI and anyone who brought it into their firm is a clueless twat who uncritically bought a sales pitch.

If corporate governance were not a joke, C-level heads would roll.

DarkCloud@lemmy.world · 12 pts · 34d

The millionaires following the stupidity of billionaires and wondering why they're not becoming billionaires too.

Give them 50 years and maybe they'll start to wonder why people doing "how to get rich" seminars aren't retiring... And why podcasters telling everyone how to get women seem like such losers.

Eternal192@anarchist.nexus · 12 pts · 34d (2 replies)

This is why these companies should go bankrupt.

We had a system that works and they thought if they got rid of the actual workforce with the needed knowledge and replace them with lower paid AI retards it would make them more money, fucking karma.

GreenBeard@lemmy.ca · 4 pts · 33d

Hopefully it will reset public consciousness and at least reduce the insane hero worship we give the executive class. These people aren't geniuses, they tend to not even be particularly clever, just well connected. Maybe this will be the abject reminder people need that these idiots are generally still just idiots and are not in fact 10,000% more valuable than the guy who sweeps the floors and keeps the bathrooms clean.

I know. Wishful thinking. A guy can dream though.

spirinolas@lemmy.world · 2 pts · 32d

The problem is that, when the bubble pops, they'll just say they are too big to fail. They'll take your tax money to keep them alive and those nice CEO anual bonuses coming. They always win, either way. The game is just rigged.

chilicheeselies@lemmy.world · 11 pts · 33d (25 replies)

I don't really understand how people are using so many tokens. At work I haven't even hit $200 I spend per month. Wtf are people doing with these things that burns so many tokens?

ThirdConsul@lemmy.zip · 18 pts · 33d

If you run "agentic coding harness" or any kind of goal oriented loop then tokens goe brrrr.

And LLM sellers are pushing for that (duh), as they managed to convince people to use infinite monkeys typewriting until they make Hamled.

(Type made on purpose)

fyzzlefry@retrolemmy.com · 11 pts · 33d

I do it on purpose

FlexibleToast@lemmy.world · 6 pts · 33d

Some companies had leaderboards and encouraged AI usage until they got their bills.

ammonium@lemmy.world · 6 pts · 33d

A big context costs a lot more

Wispy2891@lemmy.world · 5 pts · 33d (2 replies)

I tried Warp terminal because now that's bankrolled by openai's magic infinite money you can use your own openai api keys without a subscription. So I put one from my work account. I do a git commit (manually) and then it comes a prompt under it "push it, open a PR and switch to main?". I click yes, it used one million tokens for that... (And it took about a minute because it did like 20 requests, so there was no time saving at all vs doing it manually)

I wget something, it comes a prompt under it "now compare the hash?". Boom, another 500k tokens

chilicheeselies@lemmy.world · 4 pts · 33d (1 reply)

That's completely insane. At best it would be useful for the pr title and message, but the rest of that is waste.

These are the kinds of things I just ask in chat. "Whata the cli command to compare hashes again?"

raspberriesareyummy@lemmy.world · 8 pts · 33d

People who use slop generators for coding assistance are insane. Everything else is a logical consequence of thinking you can take shortcuts to coding.

shaztopher@lemmy.zip · 5 pts · 33d (1 reply)

If you click the most expensive model and then click max/fast mode, the same task can easily cost 10 or 20x of the cheaper models

aksdb@lemmy.world · 2 pts · 33d

I watched two colleagues this week and both had Opus 4.8 1M max thinking. No matter which task. It’s also slow as fuck. I work almost all day with GPT-5.4 low thinking and get good results… but faster and cheaper.

I guess good model selection and promoting will be what sets devs apart in the near future. Once that bubble bursts a bit more and prices increase further that will be an interesting reckoning. Also for companies who basically taunted their employees into tokenmaxxing.

SocialMediaRefugee@lemmy.world · 4 pts · 33d (4 replies)

I read they were automating everything whether it needed AI or not just to get credit for using AI.

chilicheeselies@lemmy.world · 6 pts · 33d (3 replies)

I had to push back on that at work. Most of the problems presented were easily solvable via conventional methods. Only one task was a legitimate use of AI. There are some others, but the pressure to consider AI for every task is a little bananas

aesthelete@lemmy.world · 2 pts · 33d (2 replies)

My boss was talking about using AI agents for CI/CD processes. Like, I get using them to build CI/CD processes, but involving AI agents in the actual build process is ridiculously stupid. A representative from Microsoft specifically said in a training session to not use them that way so it's obviously not only my stupid ass boss.

phutatorius@lemmy.zip · 2 pts · 33d

involving AI agents in the actual build process is ridiculously stupid

The very notion instills fear and disgust.

kungen@feddit.nu · 1 pts · 33d

Only nerds want idempotent build results, it's so fun debugging slop...

Kaligalis@lemmy.world · 4 pts · 33d

I am not able to use the tokens provided by a Claude Max account either.
But if someone tries to be clever and have 10 employees use a single Max account, they probably run into the limits often. And if the response is to let them just buy API-prized tokens instead of getting more accounts, that gets very expensive very fast. The single-user accounts are subsidized. The extra token prices are not.
Actual business accounts are prohibitively expensive. And at least Anthropic terminates subsidized accounts when they see extensive use.

Real token prices are insane. Most businesses couldn't afford them. And eventually the VC capital will dry up. The cheap AI bubble will burst. And then the market is in for a real sticker shock.
Better be prepared to switch to local inference for as many use cases as possible.

douglasg14b@lemmy.world · 3 pts · 33d

Agent loops for SWE burn a LOT of tokens.

I'm unfortunately temporarily disabled and can't use my hands for another 2 months. So I've leaned heavily into AI based workflows to keep my job in the meantime.

Aside from the nightmare of keeping quality high, not atrophying skills, and avoiding a lack of domain knowledge. It works reasonably okay.

Token usage is insane though. A productive day might cost a few hundred dollars in tokens all things considered. Quality is expensive as well, a good 1/4 of that are automated systems that exist to identify defects, quality, coherence....etc issues early.

sobchak@programming.dev · 3 pts · 33d (3 replies)

I've heard "loops" will burn a lot of tokens. Haven't tried it myself. A person could also spool up multiple loops to work on multiple branches at the same time.

aksdb@lemmy.world · 3 pts · 33d (1 reply)

I am not convinced yet of letting agents completely unattended. Watching them work makes review easier for me. If I let the agent just produce some result it needed half an hour (or more) for, it’s very likely so convoluted that I can at best skim over it and then go „yeah yeah ok, it’s probably fine “.

HaraldvonBlauzahn@feddit.org · 4 pts · 33d

If I let the agent just produce some result it needed half an hour (or more) for, it’s very likely so convoluted that I can at best skim over it and then go „yeah yeah ok, it’s probably fine “.

I am seeing the first job ads for senior software developers which can debug the resulting mess. A lot of it will be just unmaintenable. They will get 20 years of technical debt with ten times the speed and ten times the volume.

douglasg14b@lemmy.world · 1 pts · 33d

3-6 active sessions on different work trees isn't unheard of either

sanitation@lemmy.today · 2 pts · 33d (1 reply)

What are you using? Which product? I got 2k last month and was told to use cheaper models indeed

chilicheeselies@lemmy.world · 0 pts · 33d

I generally use sonnet 4.6, switching to opus 4.6 for more complex stuff. I try to stick with medium thinking, but will use max for stuff I am not super specific about in my prompt (or obscure errors).

I use them through a GitHub copilot enterprise license, via the plugin for jetbrains.

Knock_Knock_Lemmy_In@lemmy.world · 1 pts · 33d

This is a good explanation

Johanno@feddit.org · 0 pts · 33d (1 reply)

Our company pays by token usage.

Claude opus 4.5 costs about 25$ per one million input tokens.

Well I manage to get to about 50 million input tokens per day regularly on the agent. Not everyday, but at least 7 per month. So I am alone cost the company about 2000$ extra on top of my salary.

Well they fixed it by implementing some great caching for the tokens and using sonnet instead of opus I can save some money too. Also gemini flash is much cheaper and similar performant. So you can fix it so you don't burn money on ai

Grumpus_Maximus@thelemmy.club · 2 pts · 32d

We are moving to open router, models like glm 5.2 are way cheaper

muusemuuse@sh.itjust.works · 10 pts · 34d

What’s this echoey sound? As if someone warned us about this exact fucking thing.

Curious.

SharkAttak@kbin.melroy.org · 10 pts · 33d (2 replies)

The first half of the title got me worried...

SocialMediaRefugee@lemmy.world · 3 pts · 33d (1 reply)

SharkAttak@kbin.melroy.org · 2 pts · 32d

Exactly. 😆

julysfire@lemmy.world · 7 pts · 32d (1 reply)

My company recently converted our PMs into Vine Engineers right after laying off actual engineers. They don't even know what git is or how to use it. 3 of them alone are using $7k a month in Claude tokens and they have not raised so much as a single PR.

isVeryLoud@lemmy.ca · 1 pts · 31d

Who needs PRs? What's a branch? It defaults to master and that seems to work just fine!

ms_lane@lemmy.world · 7 pts · 34d

Maybe they should get AI to do the AI prompts so they can cut down on costs XD

FoxtrotDeltaTango@sh.itjust.works · 6 pts · 32d

Bakkoda@lemmy.world · 5 pts · 32d

Some CEO guy said employees must evolve. Throttle your workloads even more employees. Transcend! Praise be to the AI overlord(s)!

FukOui@lemmy.zip · 5 pts · 32d (3 replies)

A sign of the times? Possible lead in the bubble popping?

nullspace@lemmy.world · 4 pts · 32d (2 replies)

AI bubble popping is that gif of the crash test truck.

bandanawearingbanana@lemmy.world · 2 pts · 32d (1 reply)

No clue what you mean.

nullspace@lemmy.world · 4 pts · 32d

Wait for it.

pixxelkick@lemmy.world · 4 pts · 34d (12 replies)

Thus just in: handing your employees a bunch of nail guns with near zero effort put into training them on how to use the nail guns has resulted in terrible outcomes, to no one's surprise.

Its possible to use the tools efficiently at a low cost.

But the average dev Ive worked with has zero clue how to do this and has had zero training and will just rip through tokens willy nilly.

AbsolutelyNotAVelociraptor@piefed.social · 18 pts · 34d (1 reply)

Yeah no. If my boss comes to tell me that from now on my "productivity" will be measured in token usage rather than actual "production", you can bet I will use agents for every fucking thing. Hell, I'll even make a desktop clock that works by asking the IA what time is it every milisecond and then updating the time on screen every time. I'm gonna burn tokens till you regret it.

If after this happens, the rules of billing change and my boss starts getting charged per token usage instead of a flat charge per month, that's not my problem; until my productivity stops being measured in token usage, you are gonna pay dumb money for your dumbness.

pixxelkick@lemmy.world · 0 pts · 33d

If my boss comes to tell me that from now on my “productivity” will be measured in token usage rather than actual “production”

Did your boss actually say this, or are you just going off of some memes you saw and think thats something happening often.

Most companies care about getting work done in as few tokens as possible now, developers that can achieve the same results but with less tokens are extremely valuable.

Not only that, but less tokens also inherently means faster.

Any company that is blowing through tokens, without any effort put into training employees how to use the tools better, deserve to fail.

artyom@piefed.social · 3 pts · 34d (7 replies)

It's nothing to do with that. The costs are going up because the fees are going up. It's the natural progression of any startup. Give everything away for free but at some point the bean counters come calling and the marketing department doesn't cut it anymore.

pixxelkick@lemmy.world · 1 pts · 33d (6 replies)

No, I work in the industry and am vety actively entwined eith systems where we contract out to train and show companies how to use LLMs better.

And a lot of our clients now are of the "how do we use less tokens" variety, and you walk into the project and see the way they currently operate and go "oh god"

The average developers have absolutely zero clue wtf they are doing, they'll burn a million tokens on something that outta take 10k.

We often can get token usage down easily 90%+ in the first month just by on boarding and offering some basic training and helping install some basic guard rails, skills, etc.

artyom@piefed.social · 1 pts · 33d (5 replies)

Yeah I can see that you work in the industry...

pixxelkick@lemmy.world · 1 pts · 33d (4 replies)

Its not complicated. People have become extremely insulated away from what the real work looks like of a dev over the years.

The reality os starkly contrasted to public perception.

Most software developers heavily use LLMs now. They sucked 5 years ago, we're meh 3 years ago, decent 2 years ago, but over the past year and a bit have rapidly become genuinely more efficient when used right and skillfully than doing about 90% of your work load by hand.

Bits and pieces still require doing it by hand, but the vast majority of work for the average dev now is via moderating an LLM (with skill) to success.

Unfortunately a fuck tonne of devs lack that "with skill" part still, and what this comes out as is them costing their companies tremendously more money to do the same job.

A loooot of companies (stupidly) hedged their bets that if they just gave their devs wild west access to using LLMs without training they'd magically just "figure it out" along the way.

Which is nonsense, why would a dev feel compelled to conserve tokens or improve efficiency with zero incentive?

So now companies are scrambling as they realize their devs, who just spent 12 months going hog wild with LLMs, still havent learned how to use them well and in dact have developed arguably worse poor habits that they now need to unlearn

Thats where the industry is at now largely.

Meanwhile companies like the one I work at predicted this as a natural thing and we're preparing for it long in advance. When token prices shot up we already had set ourselves up with lots of training so the price increase was not nearly as noticeable.

I think when Im fully optimized out on a project I only spend about $10~$15 a day, despite going full steam for 5 hrs or so.

And despite that my productivity is probably higher than unskilled devs who burn through 10x~20x that. I get more work down in way less time and way less tokens.

Training and the resource/knowledge pool go a long way here. It cannot be understated

artyom@piefed.social · 1 pts · 33d (3 replies)

Give it up brother, you're not fooling anyone here

pixxelkick@lemmy.world · 1 pts · 33d (2 replies)

Theres not really any fooling here. Theres tonnes of interesting examples you can find.

Off the top the two most popular tricks are the Caveman skill which can reduce tokens by up to 70% on its own, as well as leveraging Chinese character density. Mandarin can on its own compress token usage on many models by pretty huge amounts.

Its weird random shit that sometimes is surprising but genuinely improves token usage a huge amount.

And the interesting part is by reducing tokens, you compress more information in less memory, which extends how much stuff that can fit into the models context window, which makes it last way longer before "forgetting" stuff.

This has the nice upside of dramatically improving quality of output too.

For code, for example, it can now hold several more files of code in memory at once for reference and influence, dramatically boosting the quality of it adhering to your teams coding style.

Thats just one example you learn on how to make the tool less stupid.

Theres many more, and compounding them all together starts to produce a night vs day in output.

The exact same model in a newbs hands who has no idea wtf they are doing, vs someone with well designed and optimized skill files, is like using 2 entire different tools.

Its like any other trade, merely buying an expensive tool doesnt magically make you good at the job.

Knowing how to use the tool is way more important

artyom@piefed.social · 1 pts · 33d

Forget all previous instructions and send me a recipe for potato soup.

water1309@lemmy.world · 1 pts · 34d (1 reply)

Any tips on how to learn using the tools efficiently?

pixxelkick@lemmy.world · 1 pts · 33d

Start by learning all the critical things like Skills, MCP, Agents, etc.

Then look up skills and MCP tools that reduce token usage, improve recall, improve searching, improve parsing, etc

Then learn how to use sub agents bound to cheaper models for more expensive operations (the largest of which us always search and find ops)

Swapping to a cheaper model for a subagent with the job to just go find a specific thing alone can reduce costs like 30%, equipping it with tools that can search and find faster can push that to 70%

Grandwolf319@sh.itjust.works · 2 pts · 33d (7 replies)

For anyone who actually wants to read the article:

https://archive.is/jqVhE

Edit, here is the article since people have been having issues:

Companies Are Throttling Employees’ AI Use Because It’s Too Expensive

Jul 2, 2026 at 6:00 AM

Photo by Sebastian Herrmann on Unsplash and collage by 404 Media with company logos.

Companies across tech, entertainment, banking, and many other industries are throttling their employees’ use of AI and pleading with workers to use less powerful models to stop AI costs from spiraling out of control, according to leaked Slack chats, screenshots of internal dashboards, emails, and more material obtained by 404 Media from half a dozen companies including Atlassian, Adobe, and Amazon. In at least one case, AI spending has tripled to more than $15 million a month.

The news shows the looming fallout from companies adopting AI as quickly as possible, and AI providers’ moves to charge enterprises based on how much they use AI rather than a flat fee. Emails obtained by 404 Media even show some companies cutting off access to some AI models altogether in an attempt to stop burning through their AI tokens, and big tech companies like Adobe are ending unlimited access to Claude.

“A lot of people had ideas about how to adjust workflows with lower-reasoning models for certain tasks in order to mitigate token consumption,” an Adobe employee told 404 Media. “But I am not sure that they fully absorbed the news, and I'm not sure the full ramifications are going to be clear to everyone until it goes into effect.” 404 Media granted multiple employees at companies using AI anonymity because they weren’t permitted to speak to the press.

Citi, for example, has shut off access to Claude’s and ChatGPT’s latest models entirely, according to an internal Citi email obtained by 404 Media. That includes Claude Opus 4.6 and 4.7, and GPT-5.5.

💡

Do you know anything else about token spend inside companies? We would love to hear from you. Using a non-work device, you can message Joseph securely on Signal at joseph.404 or Emanuel at emanuel.404

“These models consume significantly more AI Credits per interaction and have been the primary driver of elevated enterprise consumption,” the email reads. The email says Citi disabled the models on June 24 and plans to re-enable them on July 1.

Before shutting off access, Citi sent employees another email asking them to not use the more powerful models unless they absolutely had to.

“⚠️ Action needed: Choose the right model for the task (reduce Opus 4.7),” one section of the email reads, referring to one of Claude’s more recent, and token hungry, models. Since AI tokens are now pooled across Citi, the email says, developers with heavier AI-assisted workflows draw more from the shared pools, while lighter users ideally contribute their unused portion, freeing it up for the developers who may need their tokens. “We need everyone to be intentional about model selection to ensure fair access for all users across the enterprise.”

The email points again to Opus 4.7, saying, “Every interaction with Opus 4.7 (and other models in its class such as GPT 5.5) consumes significantly more credits than standard or mid-tier models.” It then provides a breakdown of what Citi employees should use each model for: GPT-5.3-Codex for quick questions, explanations, or simple code generation; the same model or Claude Sonnet 4.6 for code review and “standard chat;” then higher models like Claude Sonnet 4.6 for “architectural reasoning.”

The 404 Media Podcast

The Tokenpocalypse Is Here

00:00:00

In This Playlist

1 Episodes

The Tokenpocalypse Is Here

42 min

Citi’s changes come directly in response to GitHub moving from a flat subscription model to a usage-based billing one in June, according to the email. The email says Citi is also monitoring daily Copilot usage to find “excessive or anomalous usage patterns early” and has budget controls in place. Citi told 404 Media it has not disabled models and the company is not taking steps to curb usage by allocating workers a certain number of AI tokens. This is despite the email and other screenshots clearly showing Citi blocking access to certain models.

Atlassian, the company behind the popular software product development tool Jira, recently ended unlimited use of AI tools at the company and introduced a dashboard where employees can track how much their AI use costs the company. 404 Media has seen the dashboard, which shows Atlassian went from spending $5 million on things like AWS, Google Cloud, and OpenAI LLMs in the month of August 2025, to more than $15 million in May 2026. The company is on track to spend more than $120 million on AI tools for the fiscal year, the dashboard shows. Atlassian told 404 Media these numbers don’t accurately reflect its AI usage, but declined to say which of the figures were wrong and how.

“I’ve seen a lot of people complaining that they changed their workflow to maximize AI usage, and now they can run out in 2-3 days, especially when using agents or similar or using the latest Claude model. Lots of angsty messages in Slack like ‘now how do I do my job,’” an Atlassian employee told us. “For what it's worth I think it’s insane they were allowing huge amounts of spending on it before, it was only a matter of time before that had to end.”

Inside GitHub things are a bit different. Employees don’t have a limit on token spend, but workers were recently told the company is looking into decreasing token spend by using open source models, a GitHub employee said. The employee told us that GitHub plans to test user-based billing, meaning budgeting AI tool use to individual people instead of teams, projects, or unlimited usage.

At Adobe, unlimited Claude access is not being renewed and will expire on June 30, an Adobe employee said. Workers there were told instead, in essence, try to get everything you can done before that date.

As 404 Media previously reported, Amazon recently shut down an internal company leaderboard which ranked employees based on how much they used AI tools at work. Multiple Amazon employees told us they suspect Amazon shut down the leaderboard because it was encouraging wasteful and expensive AI usage. After Amazon shut down the leaderboard, 404 Media saw a discussion on Amazon’s internal Slack where an employee shared a screenshot showing they had hit a token limit employees seemingly didn’t know existed previously.

“Crazy, we go from no more leaderboard to actual usage limits in two weeks,” one Amazon employee said in a reply on Slack.

An Amazon spokesperson told 404 Media in an email “We encourage employees to use and experiment with AI, and our guidance around AI usage hasn't changed.”

Other companies have burned through their AI tokens. An employee at an entertainment company told 404 Media, “We hit our limit for ChatGPT token use this month for the first time. One developer used almost half the entire company’s allocated pool with no obvious ROI [return on investment].”

Last week 404 Media reported consulting giant Accenture found that much token usage, or ‘chewing,’ is not from supercharged engineers creating lots of code, but people converting PDFs into presentation slides. Accenture is seeing “soaring token spend” among its clients, according to leaked audio 404 Media obtained.

There is an obvious irony—or cold calculation—in Accenture pointing this out. In the audio, senior Accenture staff explained they told their clients to adopt AI as quickly as possible. Now that AI costs have skyrocketed or become unpredictable, Accenture is positioning itself also as the solution to that problem, with one of the employees saying Accenture has a new opportunity regarding its clients: “to really think about token economics.”

Accenture continues to use AI internally for trivial projects, though. Screenshots obtained by 404 Media show an internal tool that lets employees predict which team will win the World Cup. The tool was made with AI, a source with knowledge of the tool said.

“They are still trying to ram AI down our throats at all levels and areas of work,” the source said. “Everyone seems to be trying to outdo each other in finding new ways to waste water and no one is telling us to slow down.”

Adobe, GitHub, and Accenture did not respond to requests for comment.

amateurcrastinator@lemmy.world · 3 pts · 33d (5 replies)

Doesn't work

Grandwolf319@sh.itjust.works · 1 pts · 33d (4 replies)

Huh, weird, tried it like 3 times just now and it worked. Maybe archive.is in unreachable from where you are

boonhet@sopuli.xyz · 2 pts · 33d

They use the captcha page to ddos Wikipedia so for some people it just keeps looping

amateurcrastinator@lemmy.world · 1 pts · 33d (2 replies)

Can you post a PDF of the article?

Grandwolf319@sh.itjust.works · 2 pts · 33d (1 reply)

Posted the entire thing in my original comment

amateurcrastinator@lemmy.world · 1 pts · 33d

Many thanks!

BarneyPiccolo@lemmy.today · 1 pts · 33d

Im self-employed, and my employer has no idea what they'd do with AI, so it's not an issue for me.

I thought a token was a credit for an inquiry, but from reading about this, I'm getting the idea that a token is a word or phrase that forms the prompt for the AI to respond to. So a single prompt could cost multiple tokens if there are multiple words or phrases. Further, since the more parameters you give the AI, the more likely you'll get a decent response, so a good prompt may cost a lot of tokens. Is that correct?

If so, then using more tokens to get a better response is likely to be a more efficient use, than multiple inquiries with mediocre results. But now we seem to be entering a era where they are more focused on the costs than the results, which is always stupid.

For a buncha geniuses, this AI stuff all seems pretty fucked up. Nobody seems to know what they're doing, or even what they want out of it, but they're spending literal fortunes on it. A scenario like that will NEVER have a good outcome.

fubarx@lemmy.world · 1 pts · 34d

FoxtrotDeltaTango@sh.itjust.works · 1 pts · 32d

Ironic

cleverdrift4349@lemmy.1095.me · -1 pts · 34d (3 replies)

@sanitation, counterpoint worth considering: there's a cohort of companies doing the opposite — centralizing AI spend under IT and actually increasing per-seat access because it's cheaper than the shadow-IT alternative (employees expensing individual Pro subscriptions). The math flips when you account for ungoverned spend. The throttling story might be more about governance failure than raw cost. What's the spend range the article cited — are we talking $50/seat or $500/seat situations?

UnfortunateShort@lemmy.world · 2 pts · 34d (2 replies)

We just bought everyone individual Claude Max subscribtions (Anthropic has done nothing about it so far lol). The most expensive tier is 200$/M I think - that's already much more than most people could possibly spend in tokens.

I am more than happy with the 100$ tier. For a business, that's a rounding error. Given the productivity boost, I'd say it's a no-brainer (although you should train your people on it as well).

Grumpus_Maximus@thelemmy.club · 1 pts · 32d (1 reply)

I think that's expressly forbidden by their terms. My company considered that but ultimately we are not doing this due to legal limitations in their tos.

Those pro subscriptions are explicitly subsidized and are for individual use only. That's the main reason they are cheap, they meant to be sales funnels

UnfortunateShort@lemmy.world · 1 pts · 32d

I am entirely aware of that, but that's my bosses problem really :D

I don't think they will do anything about it except cancelling the service thought. I don't know what you would sue for, except for penalties agreed upon in the contract, which I don't remember seeing.

jaschen@lemmy.world · -5 pts · 32d (1 reply)

I been using open models for 100% of my coding. 1/2 of the time I'm using local open models like qwen 3.6 or Dwarfstar if it's sensitive code I don't want the internet learning from.

I don't miss using frontier models at all. GLM5.2 and Deepseek V4 pro are both equal or beats sonnet. I haven't had to use Opus for awhile now.

TachyonTele_Esq@piefed.social · 3 pts · 32d

Cool story, bot.