Crumb
c/artificial_intel · by cypherpunks@lemmy.ml · 121d

LLMs Corrupt Your Documents When You Delegate: Our large-scale experiment with 19 LLMs reveals that […] even frontier models corrupt an average of 25% of document content by the end of long workflows

https://arxiv.org/abs/2604.15597

43 points · 6 comments · view on lemmy.world

6 Comments

HiddenLayer555@lemmy.ml · 8 pts · 121d (4 replies)

This is where commit based version control can help mitigate the damage or detect it early. The golden rule in AI assistance in programming (if you're going to use it) is to check the changes to make sure you understand everything before you commit it. If documents had a git-like version control system it would be easier to detect corruption early.

Or just use Markdown files in a git repo IDK

jtrek@startrek.website · 5 pts · 121d (1 reply)

That's not how humans work. We're lazy.

You ever use snapshot tests? They're garbage. They're garbage because people glance at a 300 line diff and go "that seems right", because a good chunk of the time it is. But some portion of the time, it's borked up.

Having it in git doesn't solve that problem.

SineSwiper@discuss.tchncs.de · 1 pts · 120d

That’s not how humans work. We’re lazy.

I'm not. Stop being lazy. Review your damn code.

LLMs don't work if you don't put the effort in. They save time when you do.

eldavi@lemmy.ml · 3 pts · 120d

Or just use Markdown files in a git repo IDK

my pleas to my new management to do this fall on deaf ears, but they're gung-ho about ai.

slacktoid@lemmy.ml · 1 pts · 121d

Or use markdown, LaTeX, typst. No reason to use word docs anymore imo

butsbutts@lemmy.ml · 2 pts · 120d

sounds like it sux!!