EAGLE-4 is here. From EAGLE-1 to EAGLE-3, we’ve kept making inference faster. EAGLE-4 is our fastest yet. 961.6 tokens/s on @Alibaba_Qwen’s Qwen3.8-27B with TokenSpeed (BF16, 4× @NVIDIA B300, TP4, one request, T=0). Github: github.com/eagle-hq/EAGLE-4

EAGLE-4 is here. From EAGLE-1 to EAGLE-3, we’ve kept making inference faster. EAGLE-4 is our fastest yet. 961.6 tokens/s on @Alibaba_Qwen’s Qwen3.8-27B with TokenSpeed (BF16, 4× @NVIDIA B300, TP4, one request, T=0). Github: github.com/eagle-hq/EAGLE-4

This is a great step forward for open-source models. The cost advantage makes it possible to scale inference and verification in ways that would be expensive with closed models. The tradeoff is latency. Verification can require dozens of inference passes per task. While most can run in parallel, some must run sequentially, making end-to-end latency multiple times longer than a single generation. With EAGLE, we can accelerate those inference passes and bring the end-to-end time faster than a single standard generation, while keeping the benefits of self-verification.

AI hardware has advanced rapidly over the past few years, delivering unprecedented compute for training and inference. As AI systems continue to scale, achieving higher performance increasingly depends on both more capable hardware and more efficient software. Speculative decoding is a prime example. Rather than requiring new hardware, it unlocks more performance from existing accelerators by fundamentally changing how inference is executed. A lightweight draft model predicts multiple future tokens, which are then verified in parallel by the target model. By reducing sequential decoding steps and improving hardware utilization, speculative decoding enables substantially higher throughput from the same underlying compute. This reflects a broader shift in AI infrastructure. Software is redefining AI performance by unlocking more from increasingly powerful hardware. As accelerators continue to evolve, advances in inference algorithms will be just as critical as advances in the hardware itself, together defining the next generation of AI systems.