Presenting Proteo-R1: Thinking Foundation Models for De Novo Protein Binder Design at ICML 2026. 📍 Poster: Thu Jul 9, 5:00–6:45 PM, Hall A #2505 🎤 Oral: GenBio workshop, Fri 14:00–14:45 DM if you want to connect.

At lunch today with a xAI friend, we predict total headcount of technical staff at xAI, OpenAI, DeepMind, and Anthropic will fall by at least 50% from today's numbers. We assign a prob. of 0.25 of this happening within 1 year, 0.5 within 2 years, and an .85 within 4 years.

We took @karpathy's autoresearch agent, scaled it into a collaborative swarm, and topped @OpenAI's Parameter Golf Challenge—twice. Here’s how we did it:

lowkey… really proud of our agents right now 🥹 they just figured out how to hit #1 on OpenAI’s Parameter Golf Challenge 🔥

Excited to collaborate with @SnorkelAI on this project! Our member @mananroongta led this and show impressive results post-training a 4B agent to outperform frontier model on financial analysis. The takeaway: for many enterprise use cases, reliability > raw intelligence. A well-trained specialist agent, with the right tools and data, can outperform much larger generalist models where correctness and consistency matter most.

Excited to see this new training paradigm being explored on top of rLLM! 🚀

We are releasing rLLM v0.2.1 with many exciting new features -- including a preview of our SDK, integration with Tinker, and support of VLM and LoRA training. Come check it out!

I am incredibly excited to introduce rLLM v0.2. Zooming back to a year ago: @OpenAI's o1-preview just dropped, and RL + test-time scaling suddenly became the hype. But no one knew how they did it. @kylepmont and I had this idea - what if we built a solver-critique loop for test-time scaling, and trained both the solver AND the critique with RL to jointly improve them? We couldn't make it work. There was no infrastructure to support this kind of RL training. That became my motivation to build it. It took us a year to finally get here. At @Agentica_, we started with single-turn RL training on math and coding (DeepScaleR, DeepCoder), then advanced to training multi-turn ReAcT agents (DeepSWE), and along the way developed and released the initial prototype of rLLM. Since then, rLLM has continued to evolve. In v0.2, we added AgentWorkflowTrainer, which lets you specify ANY agentic program and train it with RL. That solver-critique flow we attempted a year ago? Now it's ~100 lines of code. We believe this will enable researchers like us to rapidly iterate and prototype wild ideas. We've seen amazing recent work on parallel thinking + aggregation, MCTS-style tree search, recursive improvement loops combined with RL — but researchers had to dive deep into training backends and maintain custom forks to make it happen. Now? Build and train these workflows with a few hundred lines of code. The dream from a year ago is finally real - and I can't wait to see what the community builds with it.

Congrats to Tongyi Lab on releasing a beast—a SOTA 30B DeepResearch agent that surpasses OpenAI DeepResearch across benchmarks (HLE, GAIA, BrowserComp)! We’re proud that rLLM powered its RL post-training. rLLM was built to help researchers & practitioners easily train custom agents, and it’s exciting to see world-class results already. rLLM v0.2 is now in preview, with an official release coming soon. Stay tuned for more groundbreaking work built on rLLM! rLLM: github.com/rllm-org/rllm

Check out the 1st Behavior Challenge, co-host with our Foundation Models for Embodied Agent Challenge at NeurIPS foundation-models-meet-embod… When I first moved my focus from LLMs/VLMs toward embodied agents, I expected the biggest challenges would be around perception, motor control, or sim-to-real transfer. But the real shock is the lack of standardized, widely accepted benchmarks. In language and vision models, progress is easy to track because we have standardized evaluation suites. But for embodied AI, without a common yardstick, it’s hard to measure whether we’re really moving forward. That’s why initiatives like the BEHAVIOR Challenge are exciting: providing a concrete foundation for evaluating embodied agents in meaningful, long-horizon tasks.

🚀 Introducing DeepSWE 🤖: our fully open-sourced, SOTA software engineering agent trained purely with RL on top of Qwen3-32B. DeepSWE achieves 59% on SWEBench-Verified with test-time scaling (and 42.2% Pass@1), topping the SWEBench leaderboard for open-weight models. 💪DeepSWE is trained with rLLM, our modular RL post-training framework for agents. rLLM makes it easy to build, train, and deploy RL-tuned agents on real-world workloads — from software engineering to web navigation and beyond. 🤗As always, we’re open-sourcing everything: not just the model, but the training code (rLLM), dataset (R2EGym), and training recipe for full reproducibility. 🔥Train DeepSWE yourself. Extend it. Build your own local agents. No secrets, no barriers. DeepSWE and rLLM mark our major shift: from training language reasoners to building language agents that can truly learn from experience. We believe the future of AI lies in experience-driven learning — and we’re here to democratize it. Welcome to the era of experience. 🌍 Links below: (1/n)

Excited to share that PAE has been accepted to ICML!

🚀 We introduce DeepCoder-14B-Preview, a fully open-sourced coding model that is on par with o3-mini and o1! 📷 We scaled our model with RL magic up to 32K context. It's performance scales to 64K context 🔥

We just made Q function work on 7B VLMs with TD learning. If you work on end-to-end RL with Q functions, you know it's extremely hard. tbh most people give it up right after they finish the first wandb run. Let me show how we got through: A thread 🧵 1/n arxiv.org/abs/2502.15760

Introducing Autellix: An agentic AI system that accelerates agentic applications. Run Deep Researcher, Google Co-Scientist, OAI Operator, or any program 💻—@langchain, @pyautogen, @crewAIInc, or just Python 🐍—4-15x faster than vLLM or SGLang⚡. Paper: arxiv.org/abs/2502.13965

A more complete introduction: we propose Digi-Q, which achieves the performance of previous online RL algorithms in the Android agent domain using a purely offline RL algorithm, even with less data than online methods. The score improved from the initial policy’s 23% to 71%, far surpassing previous offline RL SOTA and SFT SOTA. Ablation studies demonstrate that our approach is highly scalable. Our method trains a very strong Q function and then uses this Q function to select the best action for each state, thereby improving state utilization efficiency. This approach is akin to a "rehearsal" that completely eliminates the need for online rollouts.

🚨🚨🚨 We found that 1k trajectories + offline RL with VLM can achieve performances of previous online RL methods on Android-in-the-Wild. 23% to 71% boost, scalable, no online interactions, and no fancy RL tricks. Check out our ICLR paper Digi-Q: digiq-agent.com 1/15

Acknowledgements (2/2): 🧑‍🍳 And, of course, this could not have happened without: @michaelzluo, @sijun_tan, @tianjun_zhang, @justinywong_, @LambdaShi, @ColinCai09, William Tang, @mananroongta, @erranlli, @ralucaadapopa, and Ion Stoica!

🍽️ Training recipe: “Think shorter, then longer”. First, the model is trained to think short. We train the model with @deepseek_ai GRPO and 8k context length to encourage efficient thinking.  After 1000 steps, the model uses 3x fewer tokens and gains +5% over the base model.

🚨🚨🚨 What to do when pre-training ends? Excited to share our latest work Proposer-Agent-Evaluator (PAE), where we trained an open-source SOTA generalist VLM web agent entirely with self-generated data and autonomous RL. Infrastructure and model fully open-sourced. (1/9)