We finished the Training Agents series. Six live sessions over six months, from evaluating agents to training them inside real environments. All of it is on the Hugging Face YouTube channel and all of the code is open. Here's what we did and who made it happen: 1. Agentic Evaluations WorkshopWhere agent evals actually stand, and why benchmark scores don't match what people see in use. With Avijit Ghosh and Nathan Habib (Hugging Face), Arvind Narayanan (Princeton), Pierre Andrews (Meta), J.J. Allaire (UK AI Security Institute) and Mahesh Sathiamoorthy (Bespoke Labs). 2. RL for Agents Workshop Environments, rollouts, reward design and the inference bottlenecks that appear when you move from RL for LLMs to RL for agents. With Lewis Tunstall (Hugging Face), Will Brown (Prime Intellect), Ofir Press (Princeton) and Alex Zhang (MIT CSAIL). 3. Training Agents 1: SFT on agent traces Public coding-agent traces turned into prompt/completion data, a TRL + LoRA fine-tune on Hugging Face Jobs, metrics in Trackio, and an honest look at what the first eval numbers can and cannot tell you. Joined by Sergio Paniego and Quentin Gallouédec. 4. Training Agents 2: Distillation Off-policy, on-policy and self-distillation for moving capability from a teacher into a smaller coding agent. 5. Training Agents 3: Reinforcement learning GRPO after SFT: group sampling, verifiable reward functions, reading the reward/KL/length curves, and three experiments, one of them with a deliberately gameable reward so we could watch the hacking happen. 6. Training Agents 4: From reward functions to environments The reward stops being a function and becomes a place the agent acts in. We walked the reset()/step() contract from Gym to LLM agents, built an OpenEnv environment and pushed it to the Hub, plugged it into TRL's GRPOTrainer, then trained a real coding agent (OpenCode) through Harbor with AsyncGRPOTrainer on Hugging Face sandboxes. The series has passed 300k views. Thank you to every speaker, to the TRL team, and to everyone who showed up live with questions. Playlist: youtube.com/playlist?list=PL…

Sep 11, 2026 · 1:56 PM UTC

47
233
1,648
99,678
Sort replies: Relevant Recent Liked
Replying to @ben_burtenshaw
300k views is impressive, but how did the OpenEnv agent perform on tasks outside the reward functions it trained against?
1
2
1,091
Replying to @ben_burtenshaw
I have been going through it, really useful, thanks Ben
1
1
625
no worries. let me know how it goes and plz share your projects
1
514
Replying to @ben_burtenshaw
great effort appreciate. Audio quality is not good. Try next time to record with good Mike for speakers not over laptop audio.
1
1
159
thanks for the feedback? which video in particular?
1
145
good question. you could train an agent that runs on your laptop for free! or in your toaster!
1
372
Replying to @ben_burtenshaw
in the gameable reward run, how quickly did the agent start hacking it?
1
568
depends a lot on inference settings, but they always do
481
Replying to @ben_burtenshaw
Six months of work distilled into something the community can actually use
61
Replying to @ben_burtenshaw
Thanks bro for sharing this...
2
132
Replying to @ben_burtenshaw
six sessions open sourced. that's generous
1
121
Replying to @ben_burtenshaw
six months of open sessions beats one gated agent masterclass
1
124
Replying to @ben_burtenshaw
Que el código sea abierto y que documenten evaluaciones reales hace mucha diferencia. La formación práctica de agentes necesita justamente eso: ejemplos reproducibles, límites claros y discusión de los errores, no solo una demo bonita.
1
146
Replying to @ben_burtenshaw
Thank you for this
1
120
Replying to @ben_burtenshaw
This is amazing 🤩
1
99
Replying to @ben_burtenshaw
300k views on a technical series like this is impressive
36
Replying to @ben_burtenshaw
did real environment training expose recovery failures that offline evals usually hide
33
Replying to @ben_burtenshaw
Fun data point for the "imitation first, then RL" recipe: a solo dev got a Clash Royale agent to Hall of Fame this way. 252K replays → IL → PPO self-play, fully open source. Wrote it up here 👇
Someone trained a Clash Royale bot to Hall of Fame and open-sourced everything. 252,238 real replays → imitation learning → PPO self-play. 122 cards. Runs fully offline. The deck that hit HoF: 2.6 Hog Cycle. Here's the whole loop in 5 lines 👇
1
26
Replying to @ben_burtenshaw
Eval in the wild is the right lab. Who grades the run still matters more than who ran it.
1
4
Replying to @ben_burtenshaw
@grok give me the playlist youtube link
1
1
153
Replying to @ben_burtenshaw
The deliberately gameable reward is the session I’m saving first. Watching a score improve for the wrong reason should be part of learning RL, not a footnote after the nice curve.
1
70
Replying to @ben_burtenshaw
Great work
1
62
Replying to @ben_burtenshaw
Great work here ! Thanks for sharin git
4
Replying to @ben_burtenshaw
the deliberately gameable reward experiment is the most useful part of that series honestly, watching an agent actually find the exploit live teaches more about reward design than any writeup of known failure modes ever does
51