We're excited to announce our partnership with ScaleDown, bringing task-specific small models and rigorous evals together for teams building production agents. Cost, speed, and quality usually feel like a pick-two problem. ScaleDown changes that math, by building small, purpose-built language models for the workhorse steps inside an agent: compression, summarization, extraction, and classification. These high-volume operations quietly drive most of your cost and latency, and they almost never need a frontier model. ScaleDown's numbers: comparable or better accuracy, 10 to 15x cheaper, and 2 to 20x faster. So why isn't every team already swapping? Because a model change is a lot easier to commit to when you can show it works. That's the gap Arthur closes. Our eval platform gives you an objective before-and-after on the metrics that matter, so a ScaleDown swap becomes evidence instead of a leap of faith. With Arthur and Scaledown, you can: → Compare across many models at once, not just one-versus-one → Evaluate real production workflows, not demo environments → Build evals anchored to real failure modes → Generate synthetic datasets, so you can measure from a single sample ScaleDown makes your agent leaner and faster while Arthur makes the improvement measurable and trustworthy. Together we close the loop between shipping an efficient model and proving it holds up in production. Read more (link in comments).

Ask a room of practitioners to define guardrails, evals, and policies, and the answers start to blur. The confusing part: the same underlying check can wear all three hats depending on how you use it. Take prompt injection detection. Run it in the request path and it's a guardrail, blocking the attack in real time. Run it offline across yesterday's traffic and it's an eval, telling you how often you're being probed. What changes is the job you've assigned the check, not the check itself. In our latest blog post, we break down the three modes and when to reach for each: → Policies set the standard - the rules and intent everyone signs up to → Guardrails hold the line - real-time enforcement that blocks bad inputs and outputs → Evals keep you honest - measurement that tells you whether behavior matches intent, and why it failed Read it here (link in comments). #AIAgents #LLMOps #Guardrails #AIReliability #ProductionAI

We used Claude as an SRE this month at Arthur and it turned out to be the best and worst SRE 🚨 Our Head of FDE, Ian McGraw, wrote up an interesting case study about a PostgreSQL I/O spike where Claude helped him chase the problem down but pointed at the wrong root cause along the way. If you're curious about why you should never give your agents access to your production environments, this is worth a read. Link to the blog in the comments. #AIAgents #Postgres #SRE

Not all OpenTelemetry traces are created equal. ⚡ If you're instrumenting an AI agent, the semantic convention you choose shapes how much you'll actually be able to debug later. When we built the Arthur engine, we evaluated both options and went with OpenInference over the OTEL-community GenAI conventions. Here's why it matters for production agents: → Richer LLM span detail — full prompts, completions, token counts, cost, and model parameters → First-class retrieval and re-ranking spans, which RAG-heavy agents live and die by → Clear span typing — LLM, TOOL, AGENT, CHAIN, RETRIEVER are all distinct, not lumped together → Explicit message, document, and tool-call types you can actually query against The OTEL GenAI conventions are improving, but compare two traces side by side and the expressiveness gap is obvious (Images: OTEL GenAI semantic on left and OpenInference semantic on right). This is the kind of decision that feels minor on day one and compounds for months. Part 1 of our Best Practices for Building Agents series goes deeper on what to trace, which frameworks ship with strong OTEL support out of the box, and how our FDE team approaches observability with enterprise customers. Read it here (link in comments) 👇

🔍 Manual compliance checks for agents don't scale. By the time you catch a policy violation, customers are already affected and your audit trail has gaps. That's the reality for most teams running AI agents in production. Compliance requirements are mounting, but the process is still spreadsheets, Slack reminders, and last-minute scrambles before audits. ⚡ Arthur's April platform release changes that equation entirely. Here's what's new: → Policy Management: create organizational policies with inline alert and attestation rules, assign them to models with a single API call, and watch compliance status update automatically → Automated Compliance Jobs: every model with assigned policies gets daily compliance verification. Violations trigger immediate webhook notifications to your incident response systems → Agent Governance at scale: track tool calls, API costs, and agent behavior across your entire fleet Read the full breakdown (link in comments) 👇

You trust Claude Code to read files, make edits, and fire off LLM calls autonomously. But do you actually know what it's doing? Each turn reads files, makes edits, runs searches, calls sub-agents, and fires off multiple LLM requests autonomously. But you don't see what's happening under the hood. Which tool failed and got retried? Did it read the right files before making that edit? What are you actually sharing with Anthropic? We built an open-source integration that changes that. It hooks into Claude Code's event system and sends structured OpenInference traces to Arthur Engine, giving you full visibility into every turn, every tool call, and every LLM request. Installs in 30 seconds. Works locally and in CI. Link in comments 👇

Building a functioning agent isn't enough if it can't pass a governance review ✅ As agent adoption grows, organizations are losing track of what's running, what data agents can access, what tools they invoke, and who owns them. Governance teams are responding by requiring agents to meet specific standards before they're allowed to operate in production. Here's what designing for governance actually looks like: → Centralized telemetry — governance tooling discovers agents by finding their traces. No traces = invisible to the org → Thorough instrumentation — traces need to capture tools, subagents, LLM providers, and data sources so compliance teams can see the full scope → Evals and guardrails as evidence — the observability, evals, and guardrails aren't just operational best practices, they're your proof that the agent meets enterprise standards → Clear ownership — every agent needs an accountable owner and a defined scope Our FDE team sees this consistently: designing for governance from the start is the key to getting agents into production faster. Read Part 6 (link in comments) #AIAgents #AIGovernance #EnterpriseAI #LLMs #AgentDevelopment