We finished the Training Agents series. Six live sessions over six months, from evaluating agents to training them inside real environments. All of it is on the Hugging Face YouTube channel and all of the code is open.
Here's what we did and who made it happen:
1. Agentic Evaluations WorkshopWhere agent evals actually stand, and why benchmark scores don't match what people see in use. With Avijit Ghosh and Nathan Habib (Hugging Face), Arvind Narayanan (Princeton), Pierre Andrews (Meta), J.J. Allaire (UK AI Security Institute) and Mahesh Sathiamoorthy (Bespoke Labs).
2. RL for Agents Workshop Environments, rollouts, reward design and the inference bottlenecks that appear when you move from RL for LLMs to RL for agents. With Lewis Tunstall (Hugging Face), Will Brown (Prime Intellect), Ofir Press (Princeton) and Alex Zhang (MIT CSAIL).
3. Training Agents 1: SFT on agent traces Public coding-agent traces turned into prompt/completion data, a TRL + LoRA fine-tune on Hugging Face Jobs, metrics in Trackio, and an honest look at what the first eval numbers can and cannot tell you. Joined by Sergio Paniego and Quentin GallouΓ©dec.
4. Training Agents 2: Distillation Off-policy, on-policy and self-distillation for moving capability from a teacher into a smaller coding agent.
5. Training Agents 3: Reinforcement learning GRPO after SFT: group sampling, verifiable reward functions, reading the reward/KL/length curves, and three experiments, one of them with a deliberately gameable reward so we could watch the hacking happen.
6. Training Agents 4: From reward functions to environments The reward stops being a function and becomes a place the agent acts in. We walked the reset()/step() contract from Gym to LLM agents, built an OpenEnv environment and pushed it to the Hub, plugged it into TRL's GRPOTrainer, then trained a real coding agent (OpenCode) through Harbor with AsyncGRPOTrainer on Hugging Face sandboxes.
The series has passed 300k views. Thank you to every speaker, to the TRL team, and to everyone who showed up live with questions.
Playlist: youtube.com/playlist?list=PLβ¦
47
233
1,648
99,682
Fun data point for the "imitation first, then RL" recipe: a solo dev got a Clash Royale agent to Hall of Fame this way. 252K replays β IL β PPO self-play, fully open source. Wrote it up here π
Someone trained a Clash Royale bot to Hall of Fame and open-sourced everything.
252,238 real replays β imitation learning β PPO self-play. 122 cards. Runs fully offline.
The deck that hit HoF: 2.6 Hog Cycle.
Here's the whole loop in 5 lines π
Sep 14, 2026 Β· 3:50 AM UTC
1
26