Sydney, Australia
- Tweets 2,359
- Following 688
- Followers 17,628
- Likes 2,154
Comparing MiMo-V2.6-Flash in DS4 to the Xiaomi API. More work to be done on long context :) Still cool that a single M3 Ultra can compete on decode performance. Also, prefill on M5 Ultra should be much better.
4
812
Pre-orders for M5 Ultra 256GB already at 16-18 weeks shipping. Hopefully this is not bad news for the 512GB version...
2
1
16
2,356
You can run the full DeepSeek V4 Pro 0813 in native precision at a fairly decent speed, locally on 2x Mac Studios using DS4. Also worth noting that this model beats Mythos Preview at CyberGym (83.3 vs 83.1)
2
7
38
5,041
Some throughput benchmarks on one and two machines. Tensor parallelism via RDMA over Thunderbolt improves generation 1.5x and prefill 1.8x.
529
Per-token divergence from the full model, by span. It picks the same next token 95.5% of the time on tool calls and 87.4% on the final written answer, with the thinking spans at 78.1% carrying most of the cost.
2
3
781
Relative perplexity on 8 codebases across 5 languages, against the full 1.56TB release. The Linux kernel stays within 5% of the baseline.
1
3
698
@antirez @ivanfioravanti some tweaks to the DS4 TP support in case its useful github.com/kernelpool/ds4/tr…
2
2
29
2,698
Benchmarks for the Kimi K3 2bit quant thanks to @ivanfioravanti's llm_context_benchmarks tool. Performance holds up nicely over longer context thanks to KDA/MLA 🚀
2
4
9
3,525
Here's a mlx-lm (PR #1093) vs llama.cpp (b8660) comparison for Gemma-4-26B-A4B on M3 Ultra
Direct comparison of NVIDIA RTX 5090 to M3 Ultra 👀
Small models should always be faster on the 5090. The best perf for large models is to use both together (more on that soon).
Using llama.cpp isn’t super fair given the performance is not great on Apple Silicon. MLX is better
3
2
29
8,305
Plenty of great improvements in mlx-lm 0.30.6! Here’s Kimi-K2.5 running on 2 x M3 Ultra, up to 128k context 🚀
Latest mlx-lm is out:
- New models: Kimi K2.5, Step3.5 flash, LongCat Flash lite thanks to @kernelpool
- Support for distributed inference with mlx_lm.server thanks to @angeloskath
- Much faster and more memory efficient DeepSeek v3 (and other MLA-based models)
2
10
46
8,522
Here's another one from Kimi-K2.5-3bit, running on a single M3 Ultra. I was only able to test up to 8k context without MLA absorption.
1
103
It runs a little slower, but its much better than running out of memory after about 20k tokens :) However, I did something similar to DeepSeek 3.2 earlier this week, and did benchmark that:
2
4
434
Always fun to see speed runners pull off arbitrary code execution exploits in real time 😃 piped.video/watch?v=aG7_bypp…
3
15
6,937
In Apple's defense, there's a carefully articulated warning in the MLX docs ml-explore.github.io/mlx/bui…
10
1,594
MiniMax-M2 3bit DWQ vs standard 3bit quant: +4.1 points on MMLU-Pro 📈
(Took a week to benchmark 😅)
🔗 huggingface.co/catalystsec/M…
1
1
1,792
We’re ready for round 2 of @offensive_con ! Come say hi and grab a pen and stickers at our booth :)
1
17
3,196
First day of #HITB2023AMS starting soon (iOS/OSX security panel discussion is tomorrow morning). Let’s go! @HITBSecConf @TrenchantARC
1
4
31
4,981
A bit late to the game, but how did Norway not vote this song through to Eurovision?! piped.video/YgcGN_oc93I
1
LOL. The day after Trump pulls out of the Paris agreement, Australian TV stations start airing this ad again piped.video/watch?v=7UcFY5fw…
3
Song of the day - Epistemology (Keep of Kalessin): piped.video/watch?v=mVuxJuO7…
2
2
