As someone who works in tech, currently testing AI integration into healthcare EHR- the current state of AI is simply not safe for anything outside transcription- and even that is error prone without strict re-reading (not scanning!) for error correction.
The errors can be subtle but life threatening. I highly recommended against integrating it - but the most lazy providers were already using AI illegally for their notes so this was seen as a middle road.
Medical care and provider training in the USA is not ok right now, and getting worse. AI and misinformation is accelerating the decline.
TL;DR: Three Hard Truths About AI Agents
After building 12+ production systems, here's what I've learned:
-Error rates compound exponentially in multi-step workflows. 95% reliability per step = 36% success over 20 steps. Production needs 99.9%+.
Context windows create quadratic token costs. -Long conversations become prohibitively expensive at scale.
-The real challenge isn't AI capabilities, it's designing tools and feedback systems that agents can actually use effectively.
The TL;DR of the TL;DR is compounding expensive, error-prone results.
It sounds like one should be building deliberate AI workflows with extra checks (automated or human in the loop) that make careful and cost efficient incremental progress toward a measurable goal.
Sounds like hard work... when we could just build 1,000,000 MCP servers instead. (raises pinkie to corner of mouth)
7 Comments
Xulai@mander.xyz · 10 pts · 1y
As someone who works in tech, currently testing AI integration into healthcare EHR- the current state of AI is simply not safe for anything outside transcription- and even that is error prone without strict re-reading (not scanning!) for error correction.
The errors can be subtle but life threatening. I highly recommended against integrating it - but the most lazy providers were already using AI illegally for their notes so this was seen as a middle road.
Medical care and provider training in the USA is not ok right now, and getting worse. AI and misinformation is accelerating the decline.
hansolo@lemmy.today · 4 pts · 1y
The TL;DR of the TL;DR is compounding expensive, error-prone results.
jbloggs777@discuss.tchncs.de · 3 pts · 1y
It sounds like one should be building deliberate AI workflows with extra checks (automated or human in the loop) that make careful and cost efficient incremental progress toward a measurable goal.
Sounds like hard work... when we could just build 1,000,000 MCP servers instead. (raises pinkie to corner of mouth)
DecaturNature@peachpie.theatl.social · 2 pts · 1y
Outwit1294@lemmy.today · 4 pts · 1y
Interesting. I have never seen the economic side of it being discussed outside of nvidia stock prices.
Dadifer@lemmy.world · 3 pts · 1y
I keep having the same question: would it benefit to have a separate agent whose job was to error-check the first agent?
scribbler@lemmy.world · 2 pts · 1y
The three stooges didn't seem any less likely to get into trouble despite their strength in numbers
PattyMcB@lemmy.world · 1 pts · 1y
Also... how many security vulnerabilities have those agents introduced?