T156

u/T156@lemmy.world
27 posts · 2.4k comments

Recent posts

Recent comments

Does vibe coding feel incredible?

There's definitely a magic in watching it print code with almost no input. Like watching git history with Gource.

It's not good code, but it is cool to see.

They assume the AI will be able to get there with enough tokens/time with the help of the human at the other end. They have done this and decided it was good enough.

Or if it does break, just hit it with the AI stick until it works again.

The size of improvements over time is diminishing. We're not in "big bang" territory anymore and we're about two years into the "incremental refinement" period. We're about to enter the next AI Winter unless somebody comes up with a new architectural component as revolutionary as transformers have been for ML models.

The models are also getting extremely big as well, since the big improvement currently seems to largely be stuffing the model with more parameters, and making that work.

I can only imagine that the training cost has also been skyrocketing.

The core problem is that LLMs do not create. Full stop. All creativity is borne by the human inputs. Until that changes - until the model gains the capability to truly create new information, we've hit the limits in raw capability.

The models suppress outliers by design. Statistically speaking, the most novel thing is a garbled mess of random words, but random noise is useless, so it ends up being suppressed. You can see by fiddling the samplers, or increasing the temperature.

The Library of Babel is the most creative thing in the world, containing every possible combination of English words and letters. You can basically act like an LLM by trying to find a new coherent sentence in it, but also one that hasn't been said before. It's basically impossible.

But that is what an improvement is supposed to be. Compare that to finding a sentence that has been said, or something close to it.

It could also just be immaturity. They've not yet figured out how to draw the boundary between fantasy and reality, and separate their idealised character from the VA. Or what things may be better kept to themselves.

Not unless it can predict all the contributing factors.

DNA isn't the full story.

Maybe they got hit by a car, were a hobbyist boxer, or had surgery, for example.

libertarians to move to New Hampshire

Isn't that the place that ended up with a bear problem because of a libertarian town privatising all its services? So no-one took the rubbish, and then came the bears.

EDIT: It was!