How anyone thinks they are productive with llm tools is baffling

Generally I equate positivity about LLMs with people’s technical ability. I find the more they say AI is good the worse programmer they are.

every time I use it, i waste hours

Yes, exactly this. It looks good, I ask for it to tweak something. It tweaks, but now something else needs adjustment. Then it comes back unusable.

It ends up taking the same time as doing it myself. There’s some value perhaps in either the novelty or engagement that keeps me focused but it’s not more efficient.

When it does work, I’m always worried it is an illusion I’ve missed something. Like how you send an email and immediately see the typo.

People who love it, love it because they don’t need to or care about having accuracy and precision in their work. Sales and marketing, management, etc. Business idiots.

you're obviously prompting it wrong, and/or not using the latest models /s

There are right tools and wrong tools depending on the application.

There are right ways to use said tools and wrong ways...like you wouldn't use a phillips head screwdriver on a flat head.

I guarantee your company's provided tool is Copilot or OpenAI based, which is already bottom of the barrel for usefulness.

My experience is that it can work reasonabky well, but you have to waste absurd amount of tokens and have the 1m token context window.

I only do gamedev, which means a little bit more simple scriots, but it could handle even more involved systems.

If i force it to first document and explain the wholw architecture and data flow.

Did it help? Yes, but it still did take only a slightly faster. I could do it in a day more, probably.

It also cost like 50$ in tokens, in today's prices - where every AI company is loosing trillions of money, so the costs will get a lot worse. And if I tried to conserve tokens, it's shit. You have to feed it 10$ of data to be useful.

Add to that the fact that it also causes skill attrition, so once the expensive-cost future arrives, you probsbly won't be able to afford it, and good luck getting your skills back after that.

Our company wants us to use it, and the average token consumption is like 100$ per day in consumer prices. How is that even considered for such a minor gain?

So, I'll pass.

Id say you dont know how to use the tool. 'Write tests here, develop.feature X' is not a good way to use llms. Using 80k tokens and keep using same a Session is context rotting. There are a lot of boring, everyday tasks in my job that got faster. Many others that meh. Use AI, dont be driven by AI.

I know this community is all about fuck AI, but this is just straight echo chambering.

But honestly your post sounds like you're just not using it right? You can get pretty good results with it with enough guardrails. Just because you can't get the results you want doesn't mean that no one can.

That said, fuck AI. It's all a bunch of bullshit, but denying real results just means you're sticking your head in the sand and that's not how you fix this problem.

holy fucking shit man. this community has a clear astroturfing problem.

I recently used it to install Nvidia l40s drivers on redhat 9 and pass it through to my Frigate instance. Took me a few minutes. Would have been a lot of reading to find the exact answers manually.

I cut my LLM usage to almost zero because of environmental and political reasons, but it was helpful enough to wish it could be sustainable and not another tool in the dystopian take on the world.

Oooh buddy, is isn't even young engineers using these to destroy their designs. I was at a building construction conference recently where one of the presentations was about how AI is going to "give us so much time back" as designers. He then told us about how the AIs hallucinate math still, and that the AI companies are not liable for their output. After the presentation, I and another person asked him a question about who exactly the liability will lie with and how someone could protect themselves from the liability without spending all the time we "save" meticulously checking the outputs. His response was to generate thousands of outputs for the same task and then only check "the best versions." Okay, so how will we know which are the "best" without meticulously checking thousands of them?

Anyway, afterwards, I asked my colleagues from all around the country who were at the conference for their opinion on AI and the presentation, and most of these 50-60 year old men told me they regularly use it in their work already. So be prepare for things constructed in the past few years to be incredibly dangerous facilities to be in or near.

An idiot intern is better.

Well, 100% because the intern WILL eventually learn. That's the entire difference. It won't be about adjusting the prompt, or add yet another layer of "reasoning", or wait for the next "version" with a different code name an .1% larger dataset. No, you'll point to the intern they did a mistake, try not calling them an idiot, explain WHY it's wrong, optionally explain how to do it right, THEN the next time they'll avoid it or fix it after.

That's the entire point of having an intern : initially they suck BUT as you train them, they don't! Meanwhile an LLM, despite technical jargon hijacked by the marketing department, they don't "learn" (from machine learning) or train (from "training dataset") or have "neurons" (from "artificial neural networks") rather it's just statistics on the next most probable world, sounding right with 0 "reasoning".

For programming, at least it's a good way to speed up things that you know how to do but take some time to type, or you don't remember the syntax of. But relying on AI any more than that usually means you'll be adding free technical debt and debugging time or becoming dependent on it.

Yeah, don’t generate code with it. Treat it like StackOverflow. It does pretty good at that.

Claude and super powers / planning have changed my mind more on AI feature development. Iterating on the spec and making it as unambiguous as possible gives good results when you clear context and have it implement the plan. Even if it starts to stray you can just do a git reset and start a new session with the spec, adjusting it a bit, because time wise you probably haven't invested much.

It also depends on the code base, if the code base has very clear separation of concerns, good documentation, and good contracts between layers then claude can handle it pretty well. If the code base is full of spaghetti code with multiple ways to do the same thing then AI will struggle with it. In our large legacy monolith repo it doesn't do well, in our micro service repos it does great.

Also time wise it may not seem like a benefit if you just set it and wait for it to complete, the productivity advantage comes from running a couple sessions in parallel.

Also context is key, having a good claude.md file in the repo to explain patterns helps it to avoid pitfalls. If it's only context is the prompt you gave it and you tell it to implement a feature without a plan / spec outlined it will generate shit code.

It's situation specific. For tabulating data, yes. For everything else, probably not. But the thing is, you have to ask LLM if it can read the raw data to confirm if it is reading it right, before ordering it to execute more complex commands and tasks. You have to define the parameters one by one, one query everytime.

I think the intern comparison fits. The root of the problem is that AI can very good at the thing is is good at. That leads humans to believe that it is good at other things. This is often untrue.

Often the things it is good at are in the set of 'problems machines are good at'. Most professionals, people who are trained/experienced in their field face problem's that are NOT in that set. They are skilled, experienced problem solvers, who are solving difficult, real world problems. Not generic workers, or human resources.

The belief at the top is often that this machine which is 'so impressive', must therefore be good at everything. And this gets pushed down. Where people experience that same truth. The machine is incredibly good at the things it's good at, but it sucks doing what they do.

paraphrasing my grandpa - "To a suit with hammer, everything looks like a nail"

For every post I see of people complaining, I have to imagine there are 100 other people that get value out of LLMs quietly.

LLMs do a terrible job, however many things they're used for are so straightforward or unimportant that a terrible job is still "good enough".

Have you tried using skills/workflows? You can improve its context over time.

Fwiw, when I limit it to creating outlines based on given source docs or summarizing transcripts it does fine.

Definitely not worth what it cost to get there, but useful enough in those strict scenarios.

While I also don’t see how it’s productive, it can be useful for certain things, certain steps. But it really seems like you need to have the knowledge in question to help it do a good job.

People underestimate how much handholding it needs. You can tell it to do something and it might but you may not like the results. However with a bit of interaction or setting context, it might. The pretentious are calling it “prompt engineering” but it’s a combination of asking ai questions and modifying your terminology until it does what you want

People also don’t seem to understand ai really puts a premium on evaluation. You don’t see it being written but you own it, so you really need to look through the result in detail to understand whether it’s what you wanted. I see this in code a lot where the LLM produces something but a junior developer doesn’t have the skill to evaluate it before committing to source control

A previous job forced us to use them, I spent more time getting the damn thing to work than actually doing work

Is your work paying for dumb robots? Like Atlassian's shit? Or something built-in to your industry software (I guess some kinda CAD)? These are next to useless. Or is work only paying for basic model access? Also pretty useless for detailed work. The only models that give me consistent, detailed-ish work are the state of the art models. And even then you have to watch them, or have very strong verification/validation so they can bash their head against to eventually get the right result.

I'll say that I am not so much a booster, but more of a pragmatist. After the step change in quality this past fall/winter, I gave the SOTA models a try with hard earned cash. And it was worth it.

My ADHD makes it difficult to really finish personal projects once they get past the fun and interesting learning portion and neck deep into tedium of actually molding the code into the right shape or shaving the various yaks that came up. All motivation ceases. Unfortunately, my job is also my hobby. I don't wanna work after work, yo.

That game I always wanted to finish writing but got stuck at needing to grind out code? Done in an afternoon of carefully directing it. That programming language I spent significant amounts of time thinking and designing and getting the shitty PoC running but now needed to actually make it work? A week to the first version. Another to my first significant application written in that language which revealed flaws in my design for real use cases. Another to the next version with a conformance test suite which was then used along with the spec to do a complete reimplementation in another language. Another project was trying to "grow" a sorting algorithm expressed in a niche esoteric programming language using a genetic algorithm. Stuck at the point of needing to build the tools for analysis, needing a refactor to fix the poor persistence choices, just nothing but yaks to shave. Got it unstuck over a weekend and actually started to DO the damn experiment after spending so much time writing the esoteric lang interpreter and all of the experiment harness.

It is not perfect. It fucks up frequently. I have to really watch it and steer it. It loves mediocrity and shortcuts. All that said...

Like, holy shit. The amount of work I've finished or moved forward in two months is nothing short of miraculous given how many projects like these I have in various states of finished.

All I can say is that my experience aligned with your experience any time I needed to use a bolted-on AI to some product (Atlassian, Lucid, etc) but that does not reflect my experience when using SOTA models for real work.

I think it really depends on the task.

There are folks who manage to have their whole careers be basically put stuff from documentation and stack overflow to implement very basic stuff over and over again, and pray it works and doesn't need debugging. They hate coding, but it was heralded as a doctor/lawyer level pay but way easier to get into. LLMs can largely replace the work for those. These are humans I would never have trusted with anything significant, and only have them low stakes low risk stuff to keep them busy because management wants them utilized. Sure they spend a month to fall at delivering something basic, but management is happy enough.

Then there are folks who mostly live in code that is needed because it truly doesn't already exist. Those folks will find LLM relatively less useful. Now those folks do end up with braindead chores on occasion. Change from library x to y because whole they both do the same thing, x got discontinued. LLM can be useful at accelerating that because it's just so obvious but not quite as simple as search and replace. Or if you want to define some argv parsing you can let a codegen do that because it's easy but tedious.

To go back to the days of car analogies. Imagine some tech people got the world excited because they created tools to automate engineering in motor sports. People even come out saying how it helped them engineer their own vehicle and stories saying the most prolific motor racing is being taken over. You as an F1 engineer see it as mostly useless, but everyone keeps talking about it's going to replace engineers. Turns out everyone is actually taking about go karts and it is true that it works ok for that and that go karts are way more common than F1.

The problem is that to the world, programming is programming without distinction, and even the people in charge of the F1 type work don't know the difference because they were never technical either.

It really depends on the task and the tool. Current MOE models that have agentic hooks can actually be really useful for doing automated tasks. Generally, you don't want to be using AI to create things. What you want to do is hand it a very clear set of instructions along with source material. And then tell it to either iterate build on summarize or in some cases create from that.

I created a simple script with the help of AI to automate scanning files in from an automatic document reader. Convert them to OCRD PDFs. Scan through the document properly. Title the file based on the contents, then create a separate executive summary and then add an index to a master index file of a growing json.

Doing this allowed me to automate several steps that would have taken time and in the end I'm able to just search through my folders and my PDFs and very quickly find any information I need.

And this is only scratching the surface. I wouldn't have AI write me a resume or write me an email or a book. I might use it to generate an image that I then give to a real artist saying this is kind of what was in my head.

But boring stuff repetitive stuff. Things that really benefit from automation with a little bit of reasoning and thinking behind it. That's where we are right now with AI.