Really happy to see FrontierSmith accepted to NeurIPS as a Spotlight! It’s been a great year working with @wenhaocha1 , and I’m very proud of what the FrontierCS team has built together. We also have more and more new work coming soon. Excited to share them!

The @bespokelabsai blog post on Autoresearch exam is worth reading, shows minimal difference in performance with standard harnesses Much more to come, reliable frontier is within reach for all benchmarks.bespokelabs.ai/au…

Introducing Bespoke Nimble: an open data, open model, open recipe for an open Jev. Code and info: github.com/bespokelabsai/nim… Model: huggingface.co/bespokelabs/B… Data: * A new data curation recipe called contrastive data curation. * Slightly change facts to generate negative data. This pushes the model to discriminate better and become a better decision maker. The calibration is implicit. * Didn't do ablations but I think this is a critical piece! * This also means training data doesn't need probabilities. * Data covered 10 categories, and is fully synthetic. * This data is split into train and eval. Training * LoRA finetune of Qwen3.5-9B. * Distillation-free: we use Jev to only evaluate. * No RL yet! Serving * Parallel constrained decoding as suggested by @NielsRogge and @harshagundal. Results: * The post-trained Qwen (Nimble) became substantially better on our curated eval: 66% for Qwen to 90% for Nimble. Jev is at 93%. * 100ms on H100 and free to use on your macbook! Feel the AGI for free. * 2 days of building in public. :) Big caveat is that there is no standard benchmark to measure performance, and it's possible Nimble is much worse on other benchmarks compared to Jev. But it should be better than Qwen! We thank @typesafeai for making Jev and the inspiring discussions in the community. Hope this release lifts all the boats and encourages more research and activity in this space.

Introducing Benchmark Reviews: our new initiative to audit AI benchmarks. We are launching with 15 benchmarks: 4 Verified, 9 Flawed, and 2 with not enough information for a review.

We just published the largest review of cold exposure for mental health

🧵 With unlimited compute, how fast can agents surpass humans? We introduce Elo-per-token analysis to profile agent performance curves across multiple open-ended tasks. • Agents initially scale faster than repeated sampling, but over long horizons converge toward their theoretical log-linear scaling curve. Humans, in contrast, improve superlinearly. • These curves also tell us how to spend test-time compute: the scaling inflection point gives a simple rule for splitting a fixed budget across agent sessions. Split a long run in a principled way, and you can get significant gains over a single run. • Fitting human-time and agent-token curves also gives us a fun way to translate AI compute into human time. Taking OpenAI’s ~130B-token Navier–Stokes run as input and extrapolating across the two curves gives an equivalent of ~41 years of work by a mathematician at 8 hours/day 😮.

A 30-second cold blast at the end of your shower cuts sick days by 29 percent. A 2016 trial of 3,018 adults in the Netherlands is the largest cold-shower study ever done. Geert Buijze at the Academic Medical Centre in Amsterdam split participants into four groups: 30, 60, or 90 seconds of cold water at the end of a shower, or no cold water. They did this for 30 days straight. All three cold groups had 29 percent fewer sick days. The length did not matter. Thirty seconds worked as well as ninety. The mechanism is thought to involve the immune system. Brief cold exposure triggers a spike in white blood cell count and an increase in metabolic rate, both of which are part of the body's response to what it perceives as a physical threat. Over time, repeated cold exposure appears to train the immune system to respond faster and more efficiently to infections. The study also found that 91 percent of participants in the cold-water group said they would continue the practice after the trial ended, even though most of them described the experience as unpleasant. They reported feeling more energetic, and the energy boost was the reason they kept going. Thirty seconds. At the end of the shower you were already taking. The water does not have to be ice cold. It has to be cold enough to make you gasp, and you have to stay in it for a count of thirty. pubmed.ncbi.nlm.nih.gov/2763…

Releasing AutoResearchExam harness along with a setup and scoring guide for running the benchmark with your own custom harness. Eager to see your experimental insights! Harness: github.com/bespokelabsai/Aut… Custom harness guide: github.com/bespokelabsai/Aut…

Agents let us build systems for different workloads and requirements. But… can we trust what they build? We release 🌟SkySynth🌟: an engine for synthesizing high-performance, just-in-time (JIT) systems we can trust, by co-evolving formal proofs and tests alongside the code. Results: 💿 KV stores up to 2.3× faster than Redis and FASTER + formally verified stores with 2.9× Claude Code's pass rate 🚏 Model routers up to 48% cheaper than a general router ⚡ Specialized inference engine with 2.2× the throughput of vLLM/SGLang 🧵👇

The agent ecosystem is evolving rapidly and we are trying to capture a picture of where we are and where things are headed. We're now collecting responses for the 2026 survey of agents. berkeley.qualtrics.com/jfe/f… Help us create a more complete picture of how agent systems are evolving, what challenges remain, and where future research may be most impactful.