AI agents that do real work (and where they break in production)

An AI agent is a model that can take actions, call your tools, query your data, run steps, not just answer in a chat box. On stage they look magical. In production they are ordinary software with a non-deterministic core, which is exactly what makes them hard.

We ship agents that run real tasks, and we get hired to fix ones that do not. The failure modes are consistent enough to list.

Where agents break

• No guardrails on actions. A demo agent that can issue a refund can also issue the wrong refund, to the wrong customer, ten times, before anyone notices. Actions need limits, confirmations, and reversibility.
• It works on the happy path only. The demo uses the clean input. Production sends the malformed PDF, the empty field, the customer who is in three states at once. The agent has to fail safely, not confidently.
• No evals. Nobody can say whether last week’s prompt tweak made it better or worse, because there is no test set of real cases to measure against. ‘It seems fine’ is not a release gate.
• No observability. When it does something wrong, there is no trace of what it saw, what it decided, and why. You cannot fix what you cannot see.

What a production agent needs

The same discipline as any critical system, plus a few AI-specific parts. Scoped, reversible actions with confirmation on the risky ones. An eval set of real inputs that runs on every change. Guardrails on what it is allowed to touch. Logging of every decision, so a human can audit it. And a clear boundary: what the agent does alone, and what it escalates to a person.

Start narrow

The agents that make it to production start small: one workflow, clear boundaries, a human in the loop where a mistake is expensive. Widen the scope as the evals earn the trust. An agent that reliably does one thing beats a demo that impressively does everything and ships nothing.