Are robots ready to accelerate science labs? We created the WetLabs Benchmark to test 9 critical tasks, from easy to hard, and let 3 leading models take 20 attempts per task Astra leads but it comes close...

If you founded a company between 2018 and 2024 and you haven't yet pivoted to sandboxes, why? What do you know that nobody else does?

Much of why agentic map/filter/reduce workflows aren't more popular right now is because of cost. But it feels like the only natural future for some spaces, such as hard search (e.g. find all customer traces matching condition xyz), as cost of useful intelligence drops or when results are valuable enough

Distillation is a common way to prepare a model for RL where learning occurs from a stronger model, and then improves through trial and error. But how much does that better starting point help after RL? We conduct preliminary research on this question across model sizes and reasoning tasks. 🧵

JUST IN: Stanford University is accused of using AI to change students’ race & gender while making others appear thinner in advertising, with one student saying he felt “silenced & erased.”

a lot of people would stop talking about emergent phenomena if they actually looked at the data

Using open benchmarks has gotten very frustrating because it's very hard to disentangle what the model already knows / is actually bottlenecked by. E.g. when testing RLMs on "new" long benchmarks, some models will just magically regex for the right offloaded information and grab the solution... This is roughly how I've interpreted the HarnessTax paper as well, where performance seems to basically be a function of only the model capability and not the harness, but in practice we observe very different behavior / preferences with different harnesses. Model capability dominates on open benchmarks specifically. Not really sure what this implies because newer benchmarks constantly get swallowed, but I hope someone comes up with a better way to benchmark models in the open because it's largely uninteresting atm. And no, the solution isn't a benchmark that tests how far models get in a 2-day task. There's definitely a better way to host and/or manage benchmarks to actually gauge capabilities, at least for longer than a year.

cope. seethe. rage Welcome to the finale I partnered with the Wall Street Journal to release this article Probably the single worst hit piece ever written about @Kalshi Guess people will think twice before insulting me in public again wsj.com/finance/currencies/5…

incredible release - xiaomi mimo v2.6. let us do a deep dive thread for RL as that is the hero of this release. 1. first of all, the primary idea in the paper is they explicitly scale three things - batch size and throughput, environment diversity and grader compute. they discuss their large batch size choice. they primarily bat for it from the angle of throughout and ease of scaling GPUs here but I would have loved to see some discussion on large batch and how it helps with training various types of envs together, if it helps or hinders when tasks can get orthogonal.

MiMo-V2.6: The Hard Road to Scaling Up RL MiMo-V2.6 is very likely one of the largest single RL runs, by compute, that any open-source model team has undertaken to date. In an era when compute is brutally scarce, we still chose to dedicate a team of several dozen people to one goal over an extended period: scaling up RL. That takes more than research conviction. It takes a vision for AGI, respect for the unknown, and the nerve to walk straight into the hardest problems. The result is a model whose potential was built through mid-training and unlocked through heavy RL. Today, it is the number one open-source model. I strongly recommend reading the technical report. I believe it will become one of those papers that Agent RL practitioners keep reopening and discovering something new in each time. In my view, the research innovations and engineering challenges behind it surpass those of DeepSeek R1, which I was partly involved in. Some will ask: why MixRL instead of MOPD? First, they are not competing choices. We ran MixRL on verifiable tasks of moderate difficulty, including code and related agentic tasks, and found that the resulting models generalize remarkably well. Second, tasks that are difficult to verify, extremely long-horizon, or simply too challenging to include in a joint RL run are trained separately. Including them would substantially reduce rollout efficiency or introduce significant rollout staleness. We then merge the resulting capabilities through MOPD. Games, 3D tasks, and tasks with subjective evaluation signals all fall into this category. There is also a third, slightly cheeky answer. Our team is flat enough and free enough of organizational silos that MixRL simply is not difficult for us. More importantly, everyone enjoys working this way. People from different domains come together every day, driven by the pursuit of AGI and intelligence that can continuously improve itself, to confront and resolve the RL bottlenecks in each field. I will always remember the RL daily update meetings from this period. They were intense and dense, with intelligence emerging in real time. To help the open-source community focus on solving real Agentic RL problems, we have released a Qwen model distilled from MiMo RL trajectories as a stronger starting point for RL, along with 7K diverse environments and a complete RL training framework. We hope these resources will help move Agentic RL research forward. MiMo-V2.6 is only the beginning. In an era when intelligence is easy to replicate, we still choose the hard road toward self-improvement and AGI. Much of what lies ahead remains unknown. But we are willing to keep investing the time, compute, and passion required to take on one hard problem after another and work each of them all the way through, until intelligence crosses into a new regime.

Xiaomi is contributing a LOT to open research with this release. All of their training run details are basically out there in the open -- reward plots, hyperparameters, data mixtures, even costs. They also release a subset of their RL envs and a smaller model for people to play around with on more of a hobbyist budget.