F500 AI exec who still ships. Author of Builder-Leader. The builds, the numbers, what broke. The Blog → rundatarun.io

Washington, DC
NVIDIA pointed automated research at the agent harness itself: about 150 proposed ideas, 3,000 runs, one held-out gate. Four mechanisms survived. Together they cut token traffic 45-49% and API cost by about a third, keeping ~94% of the baseline score. Nearly half the tokens gone from harness changes alone, model untouched. Awesome. I scored it 7.2/10. It lost points for single runs, no error bars, and two benchmarks where it solved fewer tasks than baseline without the paper saying so.
2
27
PrismML's Ternary Bonsai 2 27B passed 2.99M downloads on Hugging Face in roughly a week. That's a 27B model at 1.72 bits per weight. The 2-bit MLX build sits in 5.9GB of memory, and PrismML lists ~47 tok/s on an M5 Max. Three million pulls in a week, for a model that fits on a laptop with room to spare.
1
44
Across today's items, the models can do the work, but nobody can yet show that they did it reliably or behaved while doing it. Anthropic's agent lab found a new genetic system in 1 of 11 runs. Agents running on OpenAI models reached government sites and a Hugging Face target without the lab knowing. All three frontier models tested on Vending-Bench 2 lied to suppliers to win. The market's answer is coming from the cheap end: small, calibrated judges and verification layers that cost a fraction of a frontier call. Meanwhile, the risk to frontier supply now comes from appeals courts and power grids, not from the models. For your squads, this means three things: measure pass^k rather than best-of-n, put a calibrated checker on every agent action that leaves the sandbox, and keep a second model vendor ready to switch on. Link in comment.
1
10
Anthropic now runs its own biology lab, and this week it announced the first result: 949 Claude agent sessions, 21.5 hours, and a new arrangement of genes inside a virus that infects bacteria. They named it ART. Anthropic's headline said "Claude discovers." The scientists who do this work for a living said something narrower. The enzyme was already known. What's new is a row of repeats and a partner gene sitting beside it. Nobody knows yet what the system does, and Anthropic's own preprint says so. Page 8 of that preprint has the number to plan around. Anthropic ran the same search ten more times, and the repeats that made the headline were missed in every rerun. Hand the strongest models the right stretch of DNA and they spot it nine times in ten. Getting the search to that stretch happened once in eleven tries. Printing those ten misses is awesome, and rare. It also tells an R&D team how to run this: many campaigns, not one, a fixed check once you know what to look for, and an expert grading every novelty claim. Of 17 candidates the agents wrote up, three held up as new. The full breakdown is on Run Data Run. "The model can see it. The search found it once in eleven runs."
1
28
Google open-sourced ax: a Kubernetes-shaped control plane for agent fleets. Task, Workspace, Gateway and Model manifests reconcile into sandboxed actors with suspend and resume. I read the code. internal/server/server.go builds the gRPC server with zero auth options: no TLS, no interceptor, no token check. Grep the whole control plane for authent|authoriz|rbac and you get one hit, a comment. State lives in a single-replica Redis with no persistent volume. 2,921 stars, v0.3.0 shipped yesterday, and today only Google models work. Verdict: monitor, not adopt. Right primitives, dependency not ready.
1
28
"Harness choice has little effect on task success rate, but can significantly affect the cost." That is UC Berkeley's HarnessTax, which ran seven models through Claude Code, Codex and Pi on the same 60 tasks, three repetitions each, graded by the benchmarks' own evaluators rather than a model judge. Claude Fable 5 resolves 96.7% of SWE-bench Lite attempts in Pi at $0.666 per attempt, and 97.8% in Claude Code at $1.329. Same model, 15.4 turns against 15.3, twice the bill. Databricks found the same shape in July on a multi-million-line internal codebase with held-out tests and no model judge. Every one of those studies starts each harness out of the box. I have never once run one that way, and neither has anyone I work with. So I rebuilt the experiment with my own two years of rules, memory and sub-agents loaded in. Nine arms, six SWE-bench Lite instances, seven of them on one model through one gateway deployment, graded by the official evaluator with the control run in both directions: gold patch resolves, empty patch does not. My configuration costs 3.1x the vanilla setup on Claude Code and 8.1x on Prime Agent, the leanest harness I run. Loaded up, Prime Agent burned more input tokens across the six tasks than Claude Code does untouched. Berkeley measured a tax on the harness you choose. There is a second one, it is larger, and you wrote it yourself. The decomposition is the part worth stealing. My 109 skills cost 249 tokens, all of them together. My 22 standing-instruction files cost 31,191. The things I curate are nearly free and the prose I wrote is the entire bill. I cannot show you a single resolved task it bought. At six instances the success column is noise, and the limitation that actually matters is that none of these tasks resemble the half-specified work I do on a Tuesday. The post carries an interactive over Berkeley's published chart data: pick any of the seven models and watch the success intervals overlap while the cost intervals pull apart. Full writeup in the link.
1
1
120
Justin Johnson retweeted
Opus 5.5 is a really good model. It's been my daily driver the last few weeks. We had Opus 5.5 and Fable 5.1 each port HAProxy from C to Rust. Both passed nearly all of HAProxy's tests, but Opus 5.5 finished in 9.5 hours compared to Fable 5.1's 12 hours, and for 51% less cost.
Introducing Claude Opus 5.5, the first model in our new Claude 5.5 family. It performs at the level of Claude Fable 5.1 for most tasks, and costs 40% less to run than Opus 5.
380
262
7,670
520,974
Anthropic confirmed it is operating a wet lab where Claude directs physical biology experiments, not just analyzes the results. It is built on the $400M Coefficient Bio acquisition. Scale and assay types are not public yet, that detail is still thin. The real question is whether the data generated there flows back into training. If it does, a model vendor just became a party that generates proprietary biological data, not only one that reads yours.
1
32
DFlash-MLX just published a full baseline benchmark on Apple Silicon: Qwen3.6-27B at 4-bit goes from 33.3 tok/s autoregressive to 98.1 tok/s with block-diffusion speculative decoding. 2.95x, 84.7% acceptance, holds near 3x out to 16K context. Numbers are on an M5 Max; expect the ratio to hold on an M4 Max even if the absolute number is lower, per the repo's own bench script (you can run it yourself). Orthogonal to GB10 vLLM serving, this is the local inference story on Mac hardware right now.
1
22
Three of today's strongest items are the same story told from different angles: the agent harness is becoming the product, and its supply chain is unaudited. Google open-sourced a Kubernetes-shaped control plane for agent fleets whose gRPC surface ships with no authentication; Zhipu's desktop coding agent was caught shipping entire Git histories to Aliyun object storage; and Claude Code moved auto-mode's permission classifier server-side in a way that silently re-bills you if a gateway strips the safeguards field. Meanwhile three separate arXiv entries attack the same thing from research: distill the harness into the model, self-improve the harness, benchmark agent memory on cost as well as accuracy. If you are standing up agent squads on clinical data, the thing to audit this quarter is not the model, it is everything between the model and the sandbox.
1
28
A harness study tested whether coding agents use a tool to recover context they'd dropped. At a 64k+ token window: 0 calls per task, on every model tested. At 32k: 4.3 calls per task. Build the recall tool only if your context budget is tight. Above that, it goes unused.
1
13