UPDATE: Kev-0.6B, 4B, and 8B are now available. Kev is a family of small open source Jev-like decision models you can train and run yourself. This new family is based on Qwen3 using the same LoRA + small pointer head technique as before, but scaled up. Out of domain, on data Kev never trained on: Kev-8B 79.6%, Jev 85.7%. • Drop-in TypeSafe System One API; their SDK works with one `base_url` change • Kev-4B serves on a 32 GB Mac in bf16: ~300 ms for five questions, ~40 ms on an H100 • Repeated documents hit a KV cache: 2-2.5x faster • Apache 2.0 License. Kev-4B trains in 40 minutes on one H100. Kev-8B in 83 minutes. Code, weights, evals: github.com/jaredpalmer/kev

I've been running DeepSeek V4 Flash 0731 through DS4 engine on the Framework. With a Q2-class quant, I was getting roughly ~200 tok/s prefill and ~15 tok/s decode. I did not feel that this configuration was fast enough to work well in an ongoing Hermes loop. Qwen3.8-Flash-Next is a much smaller model while still roughly on the same capability level of DeepSeek V4 Flash 0731 (even better on some tasks), and I'm running it at a Q4-class quant. Wilkin has been aggressively optimizing llama.cpp/ROCm specifically for Qwen3.8-Flash-Next on Strix Halo. On my Framework I'm seeing roughly ~1,300 tok/s prefill and ~30 tok/s sustained decode at around 131K context.