Presenting my grand unified theory of ML researcher impact: Your impact is directly proportional to how much pain you cause to infra. Fundamentally, you can only inflict pain upon infra if your approach actually works. And the better your approach works the more pain infra is forced to endure. So, to give some examples: - MoE's add a ton of data-dependent computation => pain (shazeer++) - GDN/KDA are the most complex architecture I've been forced to care about and a very annoying matrix inversion => pain (sonta++) - Muon is much more annoying than Adam and causes annoying restrictions on parallelism => pain (keller/jeremy++) - RL scaling forced many researchers to care about LLM inference and RL infra as a category => pain (tworek++) Even papers like Attention Is All You Need have lead to significant pain! Before transformers were invented everyone was running small jobs and I never needed to think about kv-caches or 6D parallelism.

i did start with a rudimentary implementation of pokemon stemming from a native rewrite of pokemon firered. the starting point i used gets around 4,000,000 steps per second as an rl env. here is the entire prompt (caution: long!!!):

Reflection is partnering with Shinsegae Group to build a 250-megawatt sovereign AI factory for the Republic of Korea. Open intelligence. Built on trust between allies. Owned by the nations that need it most. The future of sovereign AI. Read more in the @WSJ.

Hi friends, after three incredible years at OpenAI I am excited to share that I am starting a new chapter at @reflection_ai, where I will be leading the Science of Scaling team. Our mission is to deepen the scientific understanding of large scale learning and to turn compute into intelligence as efficiently and predictably as possible.

🎉 Next week, I am excited to join @reflection_ai as a Member of Technical Staff to help build the open intelligence ecosystem of the Western world. It's the most exciting opportunity to help software builders in our time, and will shape many years of AI Engineering in the medium-term before AGI. Not just about Western vs Eastern open models, but more about how AI-driven software will look like in 2030. I spent some time articulating my thoughts about where we're going as a community and why... which became a whole blog post. Take a look, hope it interests you! (And if it really does, we are hiring in NYC, SF, and London 😉) alexpolozov.com/blog/reflect…