We built Lighthouse Attention to answer a simple question, can you accelerate your pre-training with sparse attention while still getting a model that works perfectly with dense attention at inference? Turns out yes! Most of the well known papers apply sparsity asymetrically, only over KVs, but we do Pool Q, K, V symmetrically across a pyramid, score with l2 norms, dilated or gla scorer, gather a small dense sub-sequence, and run stock attention on it. No custom kernels, no learned scorers, no auxiliary losses. The selection logic never touches the attention kernel. The part I'm most proud of is the recoverability. After sparse pretraining you do a brief dense attention resume and the model comes back to full dense-attention quality. Every single run we tested matched or beat dense-from-scratch at the same token budget. The sparse stage doesn't hollow anything out. At 512K context on a single B200 this runs ~17x faster fwd+bwd than SDPA. We also got it working under context parallelism with 1M-token training across 32 Blackwell GPUs, and the gathered sub-sequence is dense so it just plugs into ring attention without any sparse-aware collectives. The whole thing is two new files and ~600 lines of edits on top of upstream torchtitan. Huge thanks to @bloc97_ and @theemozilla for building this together!

Most inference talks depends on how cheaply you can generate a million tokens but deepseek said a provider could have extremely cheap token prices and still build a strong business if it serves enough tokens per GPU hr. And that too by saying that there better internal versions of kernel libraries such as DeepGEMM / FlashMLA are almost +-5% of open released kernels. So their overall leverage is an aggregate of KV cache reuse, SWA bounded replay, wide expert parallel, spec decode, async schedule and load balancing.

It's better than an RTX 6000 for a third of the price.

🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 github.com/deepseek-ai/DeepI… 1/4

I interviewed @neilmovva for @substratemag on latency-insensitive inference, alternative chips, and why massive datacenters face diseconomies of scale. "We've always predicted that async inference is going to eat the world. That could easily be 95% or more of inference demand." Neil's been at the frontier of inference for years, from NVIDIA to co-founding @sailresearchco. I learned a lot from our conversation - you can listen below.

You can literally reverse engineer AIR with REA mcp + hermes skill and make most out of your hardware, the possibilities are endless. I have my version of AIR m4 silicon snapshot which I can refer to inject performance hints now directly in the metal emitted AIR.

Hermes Agent v0.22.0: The Threshold Release Changelog below

Introducing XPU Grasshopper, an AI chip co-designed with AI for the era of recursive self-improvement It went from its first tokens to 816x faster in 13 weeks Pre-order the FPGA devkit: zscc.ai/grasshopper

Introducing Rápido TTS — a lightweight Brazilian Portuguese TTS model 🇧🇷 Just 21M parameters and a 42 MB checkpoint, built specifically for pt-BR. Designed for real-time speech generation on both CPU and GPU. 1/2

There is a reason why hermes is at the top, its the agent that grows with you!

short 15s explainer videos made with H3 max for 300+ results from openai's math catalogue: gokayfem.github.io/openai-ma…

Introducing Hermes Index: an average of benchmarks measuring model performance in Hermes Agent. Our goal with Hermes Index is to better inform users choosing between models and to show labs how they perform in Hermes. portal.nousresearch.com/benc…

Blog after a long time, some notes on understanding and building a faster GQA decode, its a follow-up on the colfax blog on S/P ping pong for FA4 decode. ighoshsubho.bearblog.dev/bui…

ok now we are here, but the formal verification becomes important although agents get the control at ptx trust boundary the numbers are a bit shady since it mentions flash attention but not the exact thing, though the pipeline for agent ptx loop looks good

hand wrote a blackwell gemm in cutedsl and went from 164 to 1401 TFLOP/s on a B200, about 97% of cuBLAS

One very cool stuff about NVIDIA's Groq 3 LPU is its AOT instruction scheduling brings determinism in power draw as by-product. In normal gpu, roughly speaking, the hardware is constantly fetching new instructions if ready so it can increase the parallelism, but Groq’s LPU compiler builds a cycle by cycle schedule before inference starts and this let the compiler know exactly when each piece of data moves and when each operation fires. And if you know every operation in advance, you also know the power draw in advance! And what follows is, the compiler builds a curve of how much current the chip will pull on every cycle, before anything runs. NVIDIA uses that curve in two ways. 1. They use it like Preemptive Power system, The compiler knows which cycle a spike will land on, so it tells the power delivery system to start raising the voltage early. By the time any mma starts, the supply is already on its way up, and the capacitors have much less to cover. 2. And the second idea is really cool, they call it Clock Period Synthesis. Here the compiler can make individual clock cycles longer or shorter. It stretches the few cycles with the sharpest current spikes, so the same jump in current happens over more time. The spike is gentler, so the voltage dips less. This only works because LPX keeps clocks in sync within each chip and across all the chips. so instead of - low power → BAM, high power, we get something like - low power → medium → high power blog - developer.nvidia.com/blog/ho…

I still think of times like that, more than missions in GTAs I was more into exploring open world.

my X circle is goated af! thanks everyone >v<

About Qualcomm's Oryon CPU chipsandcheese.com/p/qualcom…

just @drisspg things