A lot of people in this community were genuinely shocked when they learned this: I do not own a DGX Spark. Much of the Spark-specific work I’ve released, including day-zero EXL3 and GGUF quants, vllm-exl3 development, custom kernels, SixCat evaluations, and reproducible serving recipes, has only been possible because incredible people in this community have loaned me access to their own hardware and clusters. I’m deeply grateful for that support. But borrowed access is temporary and unpredictable. I often have a new model, fix, or benchmark ready, then have to wait until someone else’s machine is available before I can validate and publish it properly. So I’m raising **$5,000 toward a DGX Spark of my own.** 🙏 𝗪𝗛𝗔𝗧 𝗔 𝗗𝗘𝗗𝗜𝗖𝗔𝗧𝗘𝗗 𝗦𝗣𝗔𝗥𝗞 𝗪𝗢𝗨𝗟𝗗 𝗘𝗡𝗔𝗕𝗟𝗘 → More true day-zero quantized model releases → Immediate hardware validation instead of projected results → Continuous vllm-exl3 and ExLlamaV3 kernel development → SixCat quality, speed, concurrency, and long-context benchmarks → Reproducible one-Spark recipes the community can run themselves → Faster debugging when new models or inference engines break It would turn a lot of: “I built it, but I need someone with a Spark to test it” into: “I built it, tested it, documented it, and released the recipe.” 𝗠𝗬 𝗣𝗨𝗕𝗟𝗜𝗖 𝗪𝗢𝗥𝗞 𝗦𝗧𝗔𝗬𝗦 𝗣𝗨𝗕𝗟𝗜𝗖 This is not an access fee, preorder, or paywall. The models, recipes, benchmarks, fixes, and tools will remain public. This is completely optional support for the hardware behind that work. I’m planning to place the order within 48 hours and will personally cover whatever the fundraiser does not. For complete transparency, I’ll publish: → The purchase receipt with private details removed → Photos of the actual DGX Spark → Total community contributions → Platform and payment fees → Final purchase cost → The amount I contributed myself To everyone who has loaned me hardware, funded compute, tested a build, submitted a PR, shared my work, or simply encouraged me: thank you. I genuinely could not have pushed this work as far as I have without this community. A contribution would mean a lot. A repost helps just as much. ❤️ ko-fi.com/victorangelcruz/go…

Even with the DGX Spark sold out everywhere I can’t believe we are already at $3,674 total! 73.48% of the $5,000 goal! With only $1,326 left (26.52%)!! I’m extremely grateful to you all! I’m also keeping my eyes peeled on MicroCenter stock (Columbia, OH got 6 more earlier today) and on eBay!! Tomorrow is my final day of the fund! If anyone is willing to contribute or share it’ll be greatly appreciated! Thank you!! - Ko-fi: $2,174 — 43.48% of goal, 59.17% of everything raised ko-fi.com/victorangelcruz/go… - BTC: $1,500 — 30.00% of goal, 40.83% of everything raised 3GT8KQuh9xagVbFsGjHSvaNAUW5bXsty3i

A lot of people in this community were genuinely shocked when they learned this: I do not own a DGX Spark. Much of the Spark-specific work I’ve released, including day-zero EXL3 and GGUF quants, vllm-exl3 development, custom kernels, SixCat evaluations, and reproducible serving recipes, has only been possible because incredible people in this community have loaned me access to their own hardware and clusters. I’m deeply grateful for that support. But borrowed access is temporary and unpredictable. I often have a new model, fix, or benchmark ready, then have to wait until someone else’s machine is available before I can validate and publish it properly. So I’m raising **$5,000 toward a DGX Spark of my own.** 🙏 𝗪𝗛𝗔𝗧 𝗔 𝗗𝗘𝗗𝗜𝗖𝗔𝗧𝗘𝗗 𝗦𝗣𝗔𝗥𝗞 𝗪𝗢𝗨𝗟𝗗 𝗘𝗡𝗔𝗕𝗟𝗘 → More true day-zero quantized model releases → Immediate hardware validation instead of projected results → Continuous vllm-exl3 and ExLlamaV3 kernel development → SixCat quality, speed, concurrency, and long-context benchmarks → Reproducible one-Spark recipes the community can run themselves → Faster debugging when new models or inference engines break It would turn a lot of: “I built it, but I need someone with a Spark to test it” into: “I built it, tested it, documented it, and released the recipe.” 𝗠𝗬 𝗣𝗨𝗕𝗟𝗜𝗖 𝗪𝗢𝗥𝗞 𝗦𝗧𝗔𝗬𝗦 𝗣𝗨𝗕𝗟𝗜𝗖 This is not an access fee, preorder, or paywall. The models, recipes, benchmarks, fixes, and tools will remain public. This is completely optional support for the hardware behind that work. I’m planning to place the order within 48 hours and will personally cover whatever the fundraiser does not. For complete transparency, I’ll publish: → The purchase receipt with private details removed → Photos of the actual DGX Spark → Total community contributions → Platform and payment fees → Final purchase cost → The amount I contributed myself To everyone who has loaned me hardware, funded compute, tested a build, submitted a PR, shared my work, or simply encouraged me: thank you. I genuinely could not have pushed this work as far as I have without this community. A contribution would mean a lot. A repost helps just as much. ❤️ ko-fi.com/victorangelcruz/go…

A lot of people in this community were genuinely shocked when they learned this: I do not own a DGX Spark. Much of the Spark-specific work I’ve released, including day-zero EXL3 and GGUF quants, vllm-exl3 development, custom kernels, SixCat evaluations, and reproducible serving recipes, has only been possible because incredible people in this community have loaned me access to their own hardware and clusters. I’m deeply grateful for that support. But borrowed access is temporary and unpredictable. I often have a new model, fix, or benchmark ready, then have to wait until someone else’s machine is available before I can validate and publish it properly. So I’m raising **$5,000 toward a DGX Spark of my own.** 🙏 𝗪𝗛𝗔𝗧 𝗔 𝗗𝗘𝗗𝗜𝗖𝗔𝗧𝗘𝗗 𝗦𝗣𝗔𝗥𝗞 𝗪𝗢𝗨𝗟𝗗 𝗘𝗡𝗔𝗕𝗟𝗘 → More true day-zero quantized model releases → Immediate hardware validation instead of projected results → Continuous vllm-exl3 and ExLlamaV3 kernel development → SixCat quality, speed, concurrency, and long-context benchmarks → Reproducible one-Spark recipes the community can run themselves → Faster debugging when new models or inference engines break It would turn a lot of: “I built it, but I need someone with a Spark to test it” into: “I built it, tested it, documented it, and released the recipe.” 𝗠𝗬 𝗣𝗨𝗕𝗟𝗜𝗖 𝗪𝗢𝗥𝗞 𝗦𝗧𝗔𝗬𝗦 𝗣𝗨𝗕𝗟𝗜𝗖 This is not an access fee, preorder, or paywall. The models, recipes, benchmarks, fixes, and tools will remain public. This is completely optional support for the hardware behind that work. I’m planning to place the order within 48 hours and will personally cover whatever the fundraiser does not. For complete transparency, I’ll publish: → The purchase receipt with private details removed → Photos of the actual DGX Spark → Total community contributions → Platform and payment fees → Final purchase cost → The amount I contributed myself To everyone who has loaned me hardware, funded compute, tested a build, submitted a PR, shared my work, or simply encouraged me: thank you. I genuinely could not have pushed this work as far as I have without this community. A contribution would mean a lot. A repost helps just as much. ❤️ ko-fi.com/victorangelcruz/go…

We got @DeepSeek-V4.1-Flash running in EXL3 on 4× Sparks! I was happy to colab with Cruz (@ViC305) and get his vllm-exl3 and his DeepSeek-V4.1 recipe working on TP=4! Single-stream decode: 30.2 tok/s prose, 33.4 tok/s on code, with a best warm 33.37 tok/s Aggregate: 27.8 / 43.9 / 57.7 / 61.5 tok/s at c=1/2/4/8. As you can see, there is opportunity for growth here, but it is going to take a week or so of addressing some drafter issues that might not get me much, so I am switching gears to another project for now, but might come back to this later. KV budget: 3.97m tokens Context: full 1m Every figure is a 30-second sustained cell from llm-inference-bench, not a burst. Get it here! github.com/vcruz305/DeepSeek…

K2-Horizon-36B-A4B scores 25 on the Artificial Analysis Intelligence Index, matching models with over 20× the total parameters while using 4B active parameters per token. These capabilities come from our new architecture MoVA (Mixture-of-Value Attention), which incorporates MoE-based sparsity into the compute of value vectors in multi-head attention. It opens a second axis for scaling sparsity in an LLM, beyond MoE in the FFN module. Importantly, MoVA enjoys the following advantages: • Simple and compatible with efficient attention algorithms, such as flash attention, GQA, and sparse attention • No additional KV cache cost comparing to standard GQA K2-Horizon-36B-A4B available at: huggingface.co/IFM/K2-Horizo…

New model: claude-opus-5-5 Maker: Anthropic Detected in Anthropic API models.

So You Just Bought the Hardware. Now What? I keep getting asked some version of: “I have X hardware but what model should I actually run on it?” There are honestly just the setups id actually start with right now. Weighing the reliability pretty heavily not just benchmark scores. If ive actually abused it on my own hardware for hours without it failing, that counts for a lot.. 1× DGX Spark → Qwen3.8 Flash Next Probably my favourite all round starting point on one Spark right now. Huge model for one box pretty gooood agent/tool potential..also there are multiple solid recipes for it. I put the Unsloth UD-Q2_K_XL build through a full 6 hour soak too: 9,433 requests, 0 failures no crash. So this one isnt just me liking the benchmark numbers. 2× DGX Sparks → MiMo 2.6 / GLM-5.3 Flash This is where things get really interesting...so after testing MiMo properly Id actually put it right at the front now. Tonys TP2 recipe reproduced almost bang on for me it hit 159.98 tok/s at C6, then survived a full 6 hour mixed workload with 65,226 requests and 0 restarts... 0 OOM and 0 NCCL...yeah. let that soak in, haha, sorry. GLM-5.3 is still right beside it. The EXL3 work on two Sparks has gotten really good and the recipes are much more mature than they were i guess not too long ago. so If I wanted the more established path Id probably lean GLM. If I wanted to play with what impressed me most this week hands down MiMo. 3× DGX Sparks → GLM-5.3 Flash Same answer. Dont change model jst because you bought another Spark 😂 TP3 gives it more room and this is one of those setups where the third box actually makes sense...even though i find a hard time commiting to anything past TP2, its honeslty a sweet spot. If you can get it to run well and you save a spark...why wouldnt you. 4× DGX Sparks → DeepSeek V4.1 Flash This is where I’d move to the bigger DS setup. TP4 much more room to breathe and current recipes have actually demonstrated the full 1M context instead of just putting 1000000 in a config file and pretending shes as stable as my partner. Personally dedicating 4 Sparks to one model instead of running two really good TP2 lanes is really not my thing at the moment. RTX 5090 32GB → Qwen3.8-27B This is king. Fight me. The amount of performance we've maaged to drag out of this model on a 5090 with autoround/exl3/dflash stuff is insane. RTX 3090 24GB → Qwen3.8-27B This card and its ability to run THIS model is just unbelievable. Limits were pushed, boundaries were broken. And before someone starts a war in the replies: these are not the objectively best models in existence theyre the ones I think make the most sense for the hardware. thats a completely different question. These guys have done a ridiculous amount of work on the Spark side, and @ViC305 @turboderp_ and @Tech2Wild have some really interesting EXL3/other builds Id use depending on whether I cared more about quality..context, speed or just seeing how far I could abuse the hardware. Im just linking the builders below instead of dumping 19 individual recipes on you. Go shopping based on whether you care more about quality, context, speed, or seeing how badly you can abuse the hardware. Vic : github.com/vcruz305 Turbo : huggingface.co/turboderp Tony : github.com/tonyd2wild Unsloth : github.com/unslothai

I have to retract what I posted as the fastest speed of Qwen 3.8 Flash on a single DGX Spark. Cruz shipped a custom ExLlamaV3 fork and a native engine path. I ran the same model on the same Spark: 102.6 max, 80 tok/s on code, 70 tok/s on prose. My previous benchmark and community recipes was 48.6 tok/s. I ran my full grid on it so you don't have to. This might now be the best way to run this model, without breaking the bank on one DGX Spark. THE QUANT EXL3 at 3.05 bits per weight. Trellis coding packs information more efficiently than plain rounding, so 3 EXL3 bits should land near 4.5 conventional bits. On paper that sits above FP4. I wasn't able to verify this, subjectively, running this on my harness feels at least on par FP4, and definetely not worse. A single source says NVFP4 is better on longer context. THE RECIPE One line for the DD crowd: vcruz305/exllamav3 at 523ecd3, draft depth five, dynamic stop 0.6, 8-bit KV, int8 mixers with exact fp16 reduction order, big cores pinned. Every flag in that ladder earned its tok/s. The mixers shipped fp16 inside a 3-bit model, and storing them int8 is worth 7 to 13 percent decode on its own. The 8-bit KV adds 3 tok/s at 4k and 7 at 240k, where twelve full-attention layers stream the whole cache every step. THE NUMBERS All measured on my Spark this week. Thinking off, T=0, empty KV, decode after first token, better of two reps. A speed record from a bench that cannot multiply means nothing: 17 × 19 → 323 runs first, every grid. Code (50 identical Python clamps): 102.6 tok/s. Counting 1 to 200: 107.1, with 99 percent of drafts accepted. Nothing is more predictable than numbers in order. Prose (explain a hash map, 600 tokens): 71.7, two reps at 70.5 and 71.7. JSON catalog: 66.2. Cruz's published number on the same fork: 79 tok/s code. Mine is 102.6 because repetitive code clamps accept nearly every draft. His 53 prose is a 350-word story. My 71.7 is a 600-token explanation. Same SHA, different jobs. Both stand. Sanity check: a Cruz-style greedy 400-token code job runs 79.5 on my card, 74 percent draft acceptance. His 79 reproduced. CONTEXT Needle at 5/50/95 depth, three codes: 15 for 15, one buried 243,635 tokens deep. Prefill above 1,000 tok/s at that depth. Decode at depth (256-token jobs, KV resident): 8k 51.5, 32k 55.6, 64k 67.4, 110k 66.0, 240k 55.8. Decode rises into the mid 60s at 64k before settling. No cliff. With the cache already full: count 112.9 at 32k, 82.0 at 128k. Prose 52.2 at 32k, 49.0 at 128k. Code 111.1 at 32k, 69.0 at 128k. Tools called at every depth, no XML leaking into content. AGENT LANE Tools on at 35k resident: proper tool calls returned, 58 and 57 tokens, prefill about 980 tok/s. Trust check: the first pass printed agent decode up to 386,000 tok/s. Pretty kickass. Tools off, forced 2,048-token HTML output: 86.8 and 88.3 tok/s engine rate, 86-87 percent draft acceptance. Wall clock lands near 35 tok/s because a 35k prefill puts 36 seconds behind every first token. Decode was never the agent bottleneck on this box. Prefill is. The research client ran an 80k tool loop on this serve and lived. ENERGY The prose cell: 0.85 joules per token on one GPU rail. At that rate: 6.2 million tokens a day for about 27 cents of electricity. The same tokens on Grok 4.6 output pricing are $37 a day. The box still costs a car. THE HONEST LIMITS The native engine runs one job at a time. If you need concurrent streams, Cruz ships a vLLM lane alongside it. Different engine, lower single-stream speed, real concurrency. Pick per workload. MY READ This recipe is the killer app for a single Spark. One quiet box, frontier weights, 100 tok/s code, 71 prose, a 262k window that holds, tools clean at depth. If you own one Spark, this is the loadout. Sent Cruz a PR with light fixes from this grid. Merged same day. 100 tok/s on code from one small box was not supposed to happen this year. Benchmark, sources and recipe in reply 👇