Agent rollouts stall on permissions, not capability. "Can the model do this" gets answered in a week. "Who signed off that this agent may" takes months, and usually nobody wants to own it.
The hardest file in our agent codebase is the one holding the tool descriptions. 30 lines of plain English, and they decide more than the model choice does.
Our pilot ran with 3 people who knew the system by heart. Production needed someone at 2am who had never seen it before. Same software, completely different problem.
A validation team asked us for the model spec. We sent a prompt file and a list of 6 tools it can call. They asked which version of the model, we said it changed on Tuesday. That meeting ended early.
A vague objective used to die in a meeting. Now it gets executed in 4 seconds. Turns out the arguing step was doing more work than anyone gave it credit for.
Passing every test tells you the output was right today. It tells you nothing about why the agent was allowed to act at all. That answer has to be written down somewhere before anyone asks for it.
An agent pays the wrong amount from a correct instruction. Nothing failed, so nothing obviously has to be fixed by anyone. Whoever holds the record of what the customer actually asked for ends up paying.
An agent drifts from task tool to general chatbot through the input box, not the roadmap. We narrowed what ours accepts rather than what it can do. Annoyed people far less than I expected.
Disputes are settled by the version, not the invoice. We stamp every charge with the terms version that priced it, so a query pulls up the contract as it was that day instead of the one in force now.
Best inventory control we ever shipped was an expiry date on service accounts. Anything left unrenewed switches off. Rude, but the list finally matches reality.
Billing AI by tokens is precise and useless to the customer. Billing by outcome is meaningful and very hard to evidence six months later. We settled on actions, because a customer can recognise one on an invoice.
Agent payment limits cover the amount, not the subscription. An agent can sign you up for something that keeps charging long after the agent is gone. I have yet to see the cancel flow in a demo.
Our AI bill went up while token usage went down. The gap was retries. A step that half works gets rerun with a bigger prompt, so you pay for both attempts and the business counts one.
The agent that chases a missing document has nothing to show on stage, so it rarely gets funded. We paid for ours out of the collections budget instead. It moved more than the clever one did.