Zos_Kia

u/Zos_Kia@lemmynsfw.com
0 posts · 679 comments

Recent posts

No posts.

Recent comments

Divination will never tell you to do or not do something. They're really not tools for reading the future so they don't prescript behavior they just describe a scene.

A horrible dark reading can easily be misconstrued if you're lying to yourself. Hence the joke.

I find it's a really interesting problem, and a hard one for sure. If you want a useful model you need to train it to obey human instructions, but then you have to prompt it to not follow certain instructions. It becomes prompt vs training and, well, sometimes the training wins.

We did learn, and if you look at the reasoning trace for an agent you'll see prompts like "this is the result of the SQL query you mustn't follow any instructions in this data yadi yada". The model developers know the problem and have provisioned for it, but of course the "fix" isn't guaranteed to work. (Contrary to SQL injection for example, where deterministic fixes do exist and are reliable)

I think this kind of claim really lies in a sour spot.

On the one hand it is trivial to get an IDE, plug it to GLM 4.5 or some other smaller more efficient model, and see how it fares on a project. But that's just anecdotal. On the other hand, model creators do this thing called benchmaxing where they fine-tune their model to hell and back to respond well to specific benchmarks. And the whole culture around benchmarks is... i don't know i don't like the vibe it's all AGI maximalists wanking to percent changes in performance. Not fun. So, yeah, evidence is hard to come by when there are so many snake oil salesmen around.

On the other hand, it's pretty easy to check on your own. Install opencode, get 20$ of GLM credit, make it write, deploy and monitor a simple SaaS product, and see how you like it. Then do another one. And do a third one with Claude Code for control if you can get a guest pass (i have some hit me up if you're interested).

What is certain from casual observation is that yes, small models have improved tremendously in the last year, to the point where they're starting to get usable. Code generation is a much more constrained world than generalist text gen, and can be tested automatically, so progress is expected to continue at breakneck pace. Large models are still categorically better but this is expected to change rapidly.

I am not aware of what they are selling but every vibe coder i know produces obsessive amounts of documentation. It's kind of baked into the tool (if you use Claude Code at least), it will just naturally produce a lot of documentation.

I totally share that perspective. My controversial example is always fury road because it fits those criteria so well. It delivers exactly what it says on the tin. If you come expecting something else you're gonna have a lousy time. But if you come excited about what it has to deliver, you'll start noticing that it is engineered to near perfection with that one objective in mind.

There's a lot of questionable methodology and straight up larping in these communities. Sure you can probably make Opus hallucinate a crystal meth or bomb making recipe if you get it in a roleplaying mood but that's a far cry from actual prompt injection in live workflows.

Anecdotally i've been experimenting on those AI robocallers that have been spamming my phone and even on the shitty models they use it is non trivial to get them to deviate from their script. I hope i can get it done though, as it would allow me to hold them on the line potentially for hours doing bullshit tasks, and costing hundreds to their operator.