Companies are about to get hit twice... first by the AI bill, then by the realization that their agents aren't retaining users the way they hoped. Both come down to the same thing: not having visibility into what's happening.

There are levels to agent optimization that most people skip past. First is just rewriting its own prompt. Next is prompt plus skills. After that, prompt, skills, and the tools it has access to. The more variables it controls, the more it can theoretically improve ... but also the more ways for it to optimize itself into a corner. Most agents claiming to be "self-improving" are barely operating at level one.

Self-improving agents sound magical until you ask the obvious question: improving toward what? If the metric is wrong, the agent optimizes itself into a worse product with total confidence. "Make users happy" sounds fine until the agent learns that happens to mean talking three times longer. Now you're burning tokens to make things worse. Agent optimization only works if you're optimizing for the right thing.

This little guy learns from his mistakes. Your agent can too. voker.ai/product#smart-skill…

If you just launched your agent and haven't dug into the usage data yet, you're not ready to let it self-optimize. Agent optimization works best at scale, with enough signal to know what's working and for whom. Without that, you're not optimizing.

Hot take: most agents are great at talking about solving your problem and bad at actually solving it. There's a difference between an agent that responds well and one that resolves. The bar should be resolutions, not output.

Build vs buy came up constantly at @Databricks Summit but in a different way... Teams aren't debating it upfront anymore. They're stuck with homegrown observability and monitoring tools they built a year ago that are already falling apart, and too expensive to keep fixing. Sunk cost is keeping a lot of bad systems alive way longer than they should be.

Pilots do not translate to Production. Many teams finally launched their first production agents this year. They've had a rude awakening; their offline evals and internal tests did NOT prepare them for the horrible mess of real world users.

Spent time thinking on convos from @Databricks Summit and one thing is clear: TokenMaxxing is officially dead. For the last year the move was just throw more tokens at the problem. Now every team is asking the same question: how do we cut cost without falling behind. The honeymoon phase of unlimited AI spend is over.

Agent fatigue is real and nobody talks about it enough. It's exhausting to constantly correct your agent, re-explain what you need, or nudge it back on track. At some point you just stop using it. A good agent shouldn't need that much guidance. And it should get better at understanding you over time, bare minimum.

Giving your agent a clear goal isn't enough. Most teams define what good looks like and assume the agent will find its way there, but the goal and the agent's ability to reason toward that goal are two completely different problems. You can have a perfect definition of success and still build an agent that can't achieve it.

A good agent is just a good product. Same rules apply. It solves a specific problem, knows its lane, and when a request falls outside of that lane it tells you and routes you somewhere better. An agent that tries to be everything is an agent that's good at nothing. The ones worth using have a clear purpose and stick to it.

A pattern is emerging, and it's not pretty. Companies are pouring money into AI with no way to connect the spend to actual results, and when the bill comes, the response isn't to fix the tracking; it's to cut headcount. Uber's COO can't tie their AI investment to a single consumer feature. Microsoft canceled licenses because token costs spiraled out of control. The layoffs happening right now aren't because AI replaced the work. Blind spending on AI is the problem. With the right cost tracking and optimization loop, agents can absolutely pull their weight. But you have to be able to see what's happening first.

Hosting a small invite-only dinner in SF next week. Product leaders from Uber, Databricks, and Workday talking through how they measure agent ROI in production (monitoring, evals, observability at scale). Handpicking guests who are running live agent products. If that's you: luma.com/0sdppvf0

Claude Fable 5 is out. Anthropic says it exceeds any model they've ever made generally available. Maybe. But "better on benchmarks" and "better for your product" are two very different things. Before you switch, know what you're measuring. The cost jump only makes sense if your agents actually improve with it.