NullSense@SenseNull

Computing centroids

 localhost
github.com/nullsense
Joined July 2020
5 Photos and videos
Trained an EAGLE-3 speculative decoding model, get some speed on your Nanbeige4.2-3B model! ⚡️⚡️⚡️ Decodes up to +48% faster with it, scales to ~3,100tok/s Completely lossess One vLLM config. GGUF coming soon.
2
1
3
90
NullSense retweeted
Ahmad
@TheAhmadOsman
Jun 12
Article

Anthropic's War on Opensource AI

Anthropic wants the public to see one thing: the careful lab, the safety lab, the grown-up in the room trying to keep frontier AI from running off a cliff. However, the pattern around Anthropic does

129
418
2,130
1,948,263
Trained an EAGLE-3 speculative decoding model, get some speed on your Nanbeige4.2-3B model! ⚡️⚡️⚡️ Decodes up to +48% faster with it, scales to ~3,100tok/s Completely lossess One vLLM config. GGUF coming soon.
2
1
3
90
more replies
It also scales. single-stream it's fast, but aggregate throughput climbs to ~3,100 tok/s before the GPU goes compute-bound around 64 streams.
EAGLE-3 stays net-positive with ~0.39 acceptance. Fast at every load.

ALT EAGLE-3 stays net-positive with ~0.39 acceptance. Fast at every load.

1
1
9
This is the last piece of the set. Three quants (FP8, NVFP4 LoopShield, NVFP4A16) plus the EAGLE-3 head. What are you running Nanbeige for? and what tok/s are you getting with the quant + draft head? genuinely curious. Get em here:
Four quants to run Nanbeige-4.2-3B

ALT Four quants to run Nanbeige-4.2-3B
1
9
Load more