Treat Agent Output Like Compiler Output

I'm no expert in this field, but surely a compilers output is deterministic.

There are plenty of examples of LLM generated code doing suicidally reckless things. Testing it by setting it loose on production sounds... risky.

What kind of automated tests do you guys use & how would they go if you did zero code review?

Sure, let's use untrusted AI to verify untrusted AI-generated code, what could go wrong?