A fun small win for automated research: We found and helped fix some edge cases that could affect inference performance in vLLM and SGLang. In our last blog post, we described a reward hacking judge we developed for performance optimization tasks. While applying the judge to some new autoresearch work, it discovered that some FlashInfer (a library underpinning vLLM and SGLang) kernels used a hard coded value of -50,000 as a masked-attention sentinel -- even though valid QK values can be smaller. Corner cases like this that are numerically wrong but silent can cause a huge amount of headache to find and fix, e.g., the historical debate around flash attention (arxiv.org/abs/2405.02803). Great use case for AI. github.com/flashinfer-ai/fla…

It was such a privilege to join cofounders, @_rockt and Alexey Dosovitskiy, and a phenomenal team to celebrate @Recursive_SI’s launch and $650m raise last night. I have been struck by Tim’s exceptional ambition, experience and humility in the last few months. I am delighted he and team will be pushing the frontier of AI right here in the UK 🇬🇧