Official handle for the NetHack Learning Environment (arxiv.org/abs/2006.13760)
Mazes of Menace
- Tweets 109
- Following 28
- Followers 1,211
- Likes 142
The NetHack Learning Environment retweeted
Impressive. Luckily we can turn @NetHack_LE / balrogai.com into an ASI benchmark in an instant. The human world record is 61 consecutive NetHack ascensions (teddit.net/r/nethack/comment…). Also curious to see if Astra can ascend in all roles (nethackwiki.com/wiki/Z-score).
Hot damn!
Congrats on the AGI, @OpenAI! 😉
kenforthewin.github.io/blog/…
9
9
90
10,486
Latest release of
@NetHack_LE
(v1.3.0) is now available and includes rendering the dungeon with tiles for all you convoluted people out there, seeding options for getting past the pesky full moon issue and more. Thanks to all the contributors! ⚔️
1
1
18
1,807
1 43.6 Grok-4-Wiz-AI-Cha died in The Dungeons of Doom on level 1. Killed by a housecat.
LLMs acing math olympiads? Cute.
But BALROG is where agents fight dragons (and actual Balrogs)🐉😈
And today, Grok-4 (@grok) takes the gold 🥇
Welcome to the podium, champion!
1
5
28
4,770
cc @_rockt
Whatever the thing is you think AI can’t do, benchmark it and then the world will hill climb towards it.
3
443
Most video games kill your potential.
In NetHack, it gets killed by a random monster first.
1
1
4
842
The new maintainers of NLE have released version 1.0.0:
pypi.org/project/nle/
4
291
nymph, leprechaun, floating eye, monster causing concern, dungeon floor, boulder, dungeon floor, boulder
2
5
17
6,582
niche?
Congrats @samcharrington for making it on the Top 8 AI Podcasts to Follow in 2024 with @twimlai according to @perplexity_ai. Was surprised to see @NetHack_LE mentioned there since it is somewhat niche :)
3
256
"It seems to me good enough for a birthday-party," said Frodo.
Happy "AI still can't learn to play NetHack" day for those of you who celebrate.
On this day in 2020, we released @NetHack_LE. Despite tremendous progress in AI over the last four years, this challenge is still very far from being solved. From our NeurIPS paper (arxiv.org/abs/2006.13760): "Aside from procedurally generated content, NetHack is an attractive research platform as it contains hundreds of enemy and object types, it has complex and stochastic environment dynamics, and there is a clearly defined goal (descend the dungeon, retrieve an amulet, and ascend). Furthermore, NetHack is difficult to master for human players, who often rely on external knowledge to learn about strategies and NetHack’s complex dynamics and secrets."
Even current state-of-the-art methods (arxiv.org/abs/2402.02868) don't make it past the first few dungeon levels. We probably still need many innovations on memory and planing, intrinsic motivation, conditioning on domain-specific and procedural knowledge in natural language (nethackwiki.com/), and imitating expert behavior (alt.org/nethack/), before seeing the first learning system to ascend in NetHack. Foundation models will surely play a major role in this and it is great to see that more and more people are looking into this (e.g. arxiv.org/abs/2310.00166, arxiv.org/abs/2312.07540, arxiv.org/abs/2403.00690). While many other benchmarks are saturated by LLMs, I believe NetHack will still be very challenging for (LLM) agents going forward.
10
796
"We argue that NetHack is sufficiently complex [...]"
So here's a story of, by far, the weirdest bug I've encountered in my CS career.
Along with @maciejwolczyk we've been training a neural network that learns how to play NetHack, an old roguelike game, that looks like in the screenshot. Recenlty, something unexpected happened.
2
16
74
11,120
You are lucky! Full moon tonight. 🌕
So here's a story of, by far, the weirdest bug I've encountered in my CS career.
Along with @maciejwolczyk we've been training a neural network that learns how to play NetHack, an old roguelike game, that looks like in the screenshot. Recenlty, something unexpected happened.
2
30
4,117
@GoogleDeepMind any updates?
3
141