LLMs acing math olympiads? Cute.
But BALROG is where agents fight dragons (and actual Balrogs)🐉😈
And today, Grok-4 (@grok) takes the gold 🥇
Welcome to the podium, champion!
Jul 23, 2025 · 12:03 PM UTC
274
404
3,013
1,193,081
Grok-4 barely snatches the podium from Gemini 2.5 Pro, their results being within standard error of each other.
One task where @grok still struggles, is NetHack, barely achieving 1.8% progression on BALROG's hardest task.
For comparison, Grok-4 got 66% on ARC and 15.9% on ARC-2
4
1
52
7,527
Evaluating Grok-4 wouldn't have been possible without the help from @HeinrichKuttler @zhaohan_dong and @xai, a big thank you!
2
42
6,890
🧑🎓balrogai.com is open submission, and run with a small academic-budget.
🔥 We are looking to test all the top models from @GoogleDeepMind, @AnthropicAI, @OpenAI, @Meta and more, and we need your help to do so! So please do reach out!
4
1
32
5,409
