Um, Codex:

The pricing for DGX Sparks here in Thailand: $5,797, $5,689 and $7,676. No idea why the Asus box is so much more expensive. The MSA has a 3 year warranty (vs 1 year for the others) and has a Gen5 drive (vs Gen4 in the others).

Think I will name them Deep, Seek and Qwen to keep it easy. Time to source a rack and a better cooling system.

Adding to the hardware stack.

Test configuration: - Apple M5 / 32 GB unified memory - Ollama 0.32.13 native MLX - Qwen 3.8 27B NVFP4, 32K context - MTP speculative decoding verified active - 3 workloads × 3 concurrency levels × 2 thinking modes × 2 repeats - 84 streams / 84,000 completion tokens / zero errors The Ollama MLX concurrency limitation is tracked in issue #17280: github.com/ollama/ollama/iss… Post-fix estimate—not measured: 22–28 aggregate t/s at p2 and 28–40 t/s at p4 if Ollama adds efficient fused continuous batching.

At a fixed 1,000-token cap, Thinking High reached visible output in 38 of 42 streams. Four reasoning streams spent the allowance before completing a visible answer. Thinking Off produced visible output in all 42 streams. This measures serving behavior under a fixed budget—not answer quality.

Thinking High versus Off barely changed overall serving speed: - Off: 15.26 aggregate t/s - High: 15.01 aggregate t/s Off was 1.7% faster overall, but High was 4.9% faster on the reasoning workload. The effect was workload-dependent.

Because requests queued instead of continuously batching, latency rose sharply: - Mean TTFT: 1.3s → 34.5s → 98.9s - Condition wall time: 66.9s → 134.7s → 261.7s The p2/p4 numbers characterize FIFO queueing—not true simultaneous decode scaling.

The surprising result was concurrency. Aggregate throughput stayed nearly flat as I requested more simultaneous streams: - p1: 15.0 t/s - p2: 14.9 t/s - p4: 15.5 t/s `OLLAMA_NUM_PARALLEL=4` was enabled and verified, but the current MLX runner serialized the requests.

Still on my quest to find the minimum spec Apple silicon that Qwen 3.8 27B is useable on. Same 36 controlled benchmarks but this time using Ollama's official Qwen 3.8 27B NVFP4 MLX package on the M5 MacBook Pro 32GB. In matched one-stream, Thinking Off tests, NVFP4 MLX averaged **2.29× the throughput** of the previous LM Studio GGUF Q4_K_M deployment: 15.15 vs 6.62 tokens/s.

Important caveat: this is a serving-efficiency experiment, not a quality benchmark. Thinking High consumed the full 1,000-token allowance as hidden reasoning and never reached a visible answer. A fair quality comparison needs a larger reasoning budget plus a separate visible-answer allowance. Configuration: Apple M5 / 32 GB unified memory, LM Studio, Qwen 3.8 27B Q4_K_M, 32K context, four slots, MTP speculative decoding on, temperature 0, three workloads, two repeats per condition.

Concurrency improved total throughput but reduced each agent's speed. From 1 to 4 parallel agents, aggregate throughput rose about 46%, while average per-stream decode speed fell about 64%. This is useful for batch work, but less attractive when interactive responsiveness matters.

Thinking Off was faster at every tested concurrency: - 1 agent: +16.1% sustained throughput - 2 agents: +12.0% - 4 agents: +15.0% Across the balanced test matrix, Thinking Off was 14.3% faster overall, with 32.9% lower TTFT and 12.6% lower end-to-end time.

I ran 36 controlled local benchmarks of Qwen 3.8 27B Q4_K_M on a 32 GB Apple M5 MacBook Pro, comparing Thinking High vs Thinking Off at 1, 2 and 4 parallel agents. The biggest result: with a 1,000-token cap, Thinking High produced 0 visible answers across 42 streams. Every generated token was reported as reasoning.

5 Hours to go...

Added my machines and unlocked full access!

Done.

If you are running @MiaAI_lab recipe for DSV4F on 2 x Sparks, consider updating to her latest push. My results after updating:

BOOM Results:

sparkDash is now live. Thank you @MiaAI_lab for a great tool.

Results for DeepSeek v4 Flash-0731 on 2 x DGX Sparks comparing thinking set to off, high and max. Across the 27 runs, reasoning off delivered the highest mean sustained throughput at every concurrency level. The difference became negligible with four agents: 118.2 tok/s off versus 116.7 max and 116.1 high. Reasoning high led per-stream throughput with two agents, while reasoning off led at one and four agents.

DeepSeek V4 Flash-0731 on 2× DGX Spark using @Tech2Wild Reasoning benchmark with 1, 2, and 4 parallel runs (spec decode on and thinking set to max):

DeepSeek V4 Flash-0731 on 2× DGX Spark using @Tech2Wild Json Generator benchmark with 1, 2, and 4 parallel runs (spec decode on and thinking set to max):

DeepSeek V4 Flash-0731 on 2× DGX Spark using @Tech2Wild HTML Game benchmark with 1, 2, and 4 parallel runs (spec decode on and thinking set to max):

Reserved.

I upgraded DSV4F on my sparks from @mr_r0b0t recipe v0.25 to v0.26. The upgrade is worthwhile operationally: slightly lower single-user decode speed, but better responsiveness, concurrency, native SM121 execution, and substantially more context capacity.

I asked Grok Heavy to update the AI Interaction Chart, adding categories for people running local inference and near-frontier local inference and here is what it produced:

I asked Grok Heavy to update that chart:

Here is what I got on SuperGrok Heavy:

With thinking off, average sustained throughput rises from 83.3 to 88.5 tok/s—within 1.1% of the older model’s 89.5 tok/s baseline. High thinking carries a measurable throughput cost.

This comparison highlights the main result: 0731 improves HTML generation throughput but generally underperforms the older model on JSON and reasoning. Overall sustained throughput decreased about 7%. DeepSeek V4 Flash 0731 doesn’t appear to be a universal performance upgrade on dual DGX Sparks. It improves sustained HTML/code generation but regresses throughput on JSON and reasoning-heavy workloads. Whether it is the better model depends on whether its post-training improvements produce measurably better answers.

Results for DeepSeek V4 Flash-0731 on 2× DGX Spark using @Tech2Wild HTML Game benchmark with 1, 2 and 4 parallel runs (spec decode on and thinking set to high):

Results for DeepSeek V4 Flash-0731 on 2× DGX Spark using @Tech2Wild Json benchmark with 1, 2 and 4 parallel runs (spec decode on and thinking set to high):

Results for DeepSeek V4 Flash-0731 on 2× DGX Spark using @Tech2Wild reasoning benchmark with 1, 2 and 4 parallel runs (spec decode on and thinking set to high):

Results comparing DSV4F with speculative decoding OFF vs ON:

Reasoning results:

Json Generator results:

HTML Game results:

Here is DeepSeek V4Flash with speculative decode ON. Test configuration: • 2× NVIDIA DGX Spark, each with one GB10 and 128GB unified memory • TP=2 across both nodes using vLLM’s native multiprocess backend and NCCL over ConnectX-7 RoCE/IB—not Ray • Same official deepseek-ai/DeepSeek-V4-Flash-DSpark checkpoint as the baseline • 284B total parameters, 13B activated per token • Mixed FP4/FP8 weights: MoE experts in FP4, most other parameters in FP8 • vLLM 0.25.2 dev, PyTorch 2.11.0, CUDA 13.0, NCCL 2.28.9 and FlashInfer 0.6.15 • FlashInfer B12X MXFP4 MoE backend and sparse-MLA attention • Native DSpark speculative decoding enabled with K=5 • nvfp4_ds_mla KV cache • 200,000-token configured context • Maximum 16 sequences and 16,384 batched tokens • 84% GPU-memory utilization and prefix caching enabled • Breakable CUDA-graph execution; no --enforce-eager • Benchmark monitor: parallel 6, 4,096 maximum output tokens, temperature 0.2 and thinking off • OpenAI-compatible local API Important: this uses the same model checkpoint as the earlier non-speculative baseline. The difference is the newer DSpark-enabled inference stack, optimized KV format, B12X backend and K=5 speculative decoding. Speculative-token acceptance is workload-dependent. #DGXSpark #LocalAI #NVIDIA As with before, I used the tool by @Tech2Wild.

Here are the results for the reasoning run:

Here are the results for his Json generator, again in 1, 2 and 4 parallel runs:

I used @Tech2Wild benchmarking app, thanks to him for making some easy for us norms to use! Here are the results for his HTML Game run in 1, 2 and 4 parallel runs:

First Benchmarks: Test configuration: • 2× NVIDIA DGX Spark, each with one GB10 and 128GB unified memory • Tensor-parallel deployment across both nodes: TP=2 using Ray and NCCL • Dual 200Gb/s RoCE-capable interconnects • Official deepseek-ai/DeepSeek-V4-Flash-DSpark checkpoint • 284B total parameters, 13B activated per token • Mixed FP4/FP8 weights: MoE experts in FP4, most other parameters in FP8 • vLLM 0.23.1rc1.dev1511, CUDA 13.0.2, NCCL 2.28.3 • FP8 KV cache, 262,144-token configured context • Maximum four concurrent sequences and 8,192 batched tokens • Prefix caching enabled and eager execution enforced • Thinking enabled with high reasoning effort • OpenAI-compatible local API Important: these results are a non-speculative baseline. Although this is the DSpark checkpoint, DSpark speculative decoding was not enabled for this run.

Just unboxed two NVIDIA DGX Sparks 🔥 This is my first real step into the local AI movement — excited to start building, experimenting, and learning. Huge thanks to everyone contributing to the DGX Spark knowledge base. Interesting side note: these were out of stock everywhere in Thailand (even though many websites still showed them as available). My wife reached out to the MSI distributor, who connected her with the last two units. She got them for 156,900 THB (~$4,680) each. Also, currently in weeks 4 of a 6–8 week wait for a 16” M5 Max MacBook Pro with 128GB to add to the mix. Looking forward to contributing wherever I can. The journey starts now. #DGXSpark #LocalAI #NVIDIA