Kev on Core ML 🍎 After seeing this tweet, We got inspired to port Kev-0.8B (the Qwen3.5 variant) and ran Guess Who over 80 Wiki bios, on an M5 Pro, same accuracy, coreml vs original PyTorch: 🚅 37 ms vs 1.07 s per bio → all 80 in 3.0 s vs 99 s 🧠 ~2 GB vs 6.6 GB memory 📦 1.45 GB vs 1.79 GB model Model: huggingface.co/FluidInferenc… Code: github.com/FluidInference/Fl…
UPDATE: Kev-0.6B, 4B, and 8B are now available. Kev is a family of small open source Jev-like decision models you can train and run yourself. This new family is based on Qwen3 using the same LoRA + small pointer head technique as before, but scaled up. Out of domain, on data Kev never trained on: Kev-8B 79.6%, Jev 85.7%. • Drop-in TypeSafe System One API; their SDK works with one `base_url` change • Kev-4B serves on a 32 GB Mac in bf16: ~300 ms for five questions, ~40 ms on an H100 • Repeated documents hit a KV cache: 2-2.5x faster • Apache 2.0 License. Kev-4B trains in 40 minutes on one H100. Kev-8B in 83 minutes. Code, weights, evals: github.com/jaredpalmer/kev
2
4
39
2,794
over a year late but we converted SigLIP 2 to Core ML and sorted 7,349 pet photos into 37 breeds on a M5 Coreml model vs. base PyTorch model, same accuracy: • 36 s vs 102 s • 5 ms per photo vs 14 ms • 262 MB peak RAM vs 4.2 GB • 715 MB on disk vs 1.5 GB model: huggingface.co/FluidInferenc… code: github.com/FluidInference/Fl…
Introducing SigLIP2: now trained with additional captioning and self-supervised losses! Stronger everywhere: - multilingual - cls. / ret. - localization - ocr - captioning / vqa Try it out, backward compatible! Models: github.com/google-research/b… Paper: arxiv.org/abs/2502.14786
6
14
186
10,361
We have converted GLiNER2.5-Decide to coreml. ~4× faster, ~5× less Peak RAM, half the size. model: huggingface.co/FluidInferenc… code: github.com/FluidInference
Introducing GLiNER2.5-Decide, our new 340M parameter open weight, encoder-based decision model. GLiNER2.5-Decide is built for fast, deterministic classification. The model evaluates a set of user-defined typed questions and rules, and jointly decodes their answers, returning structured decisions with probability distributions and confidence scores. We evaluated the model’s performance on Fast Decisions, an unseen, internally generated classification suite based on 17 datasets testing real-world use cases across routing, triage, classification, sentiment, and content understanding. Measured against similar decision models, GLiNER2.5-Decide leads in 9 of the 17 datasets, achieving the highest average score: - GLiNER2.5-Decide: 60.1% - SemIf: 56.4% - JevK5: 57.5% - Laya: 46.6% This performance makes the model a strong fit for use cases like tool calling, model routing, browser and computer use, and LLM-as-a-judge. GLiNER2.5-Decide’s lightweight encoder architecture makes it easy to fine-tune the model for specific tasks, while being efficient enough to run locally on consumer-grade CPUs or in air-gapped environments, giving users greater control over where their data goes and where the model runs. To make building and experimenting with GLiNER2.5-Decide as easy as possible, we're also offering hosted inference. You can now use our API to run inference and fine-tune GLiNER models on specialized tasks right inside your own coding agent: agent.fastino.ai As with previous models, we’re also releasing the model weights on @huggingface under the Apache 2.0 license: huggingface.co/fastino/GLiNE…
15
66
733
54,127
Replying to @Maouswawan
Direct links, sorry about that: model: huggingface.co/FluidInferenc… Swift + demo: github.com/FluidInference/Fl… (v0.3.0, see Sources/SortAnythingDemo) Both are public and live: the Hugging Face repo is public and FluidUse v0.3.0
2
1,765
I decided to give fine-tuning a shot: trained Decision-1.0-Lex on a ~6.7k Snake moves dataset on my M5. was relatively quick without needing a dedicated GPU. the model went from a score of 12 to 36 Model: huggingface.co/FluidInferenc… code: github.com/FluidInference/Fl… Base model from vLLM Semantic Router, built by @XunzhuoLiu
1
1
11
1,583
Parakeet 🦜 Ultra 🤗 is now in FluidAudio, running🏃‍♂️‍➡️ on the ANE 🔥 Ultra vs v3 on our Core ML benchmarks (M5 Pro, ANE, back to back): • LibriSpeech clean: 2.13% vs 2.27% WER · 127× vs 129× RTFx • LibriSpeech other: 3.81% vs 4.12% WER · 110× vs 115× RTFx • FLEURS, 24 langs: 11.7% vs 14.8% WER · 135× vs 137× RTFx Model: huggingface.co/FluidInferenc… Code: github.com/FluidInference/Fl…
Today we're releasing Moondream Parakeet Redux and Moondream Parakeet Ultra. Two speech-to-text models based on NVIDIA's Parakeet. 25 languages. They run locally, wickedly fast. Redux is for CPUs. Ultra is for GPUs.
2
8
115
10,953
FluidAudio now runs Nemotron 3 Diarization on-device in Core ML. Its able to support 8 speakers with 8 different languages like in the below demo. Models: huggingface.co/FluidInferenc… Code: github.com/FluidInference/Fl… Special thanks to @NVIDIAAI for the early access.
When several people talk at once, a transcript can get messy fast. Our new Nemotron 3 Diarization model tracks who spoke when, even when voices overlap. It handles up to eight speakers, has 100M parameters, and is now available on @huggingface 🤗
2
14
200
13,158
we ported @knowledgator’s GLiClass Edge Apps v2 to Core ML on an M5 Pro. 32.7M params, ~1.4 ms per decision on CPU. it played 2048 for 2 minutes and reached the 4096 tile. code: github.com/FluidInference/Fl… models: huggingface.co/FluidInferenc…
The Jeff model, based on our GLiFormer, should be the most parameter-efficient model (a 500M-parameter model) for its performance score. And we created it right before the Jev release. Right now, imagine if we scale it 😉
4
11
103
7,707
since y'all loved the Tetris we decided to make a follow up. the model harness uses fewer bad moves offered, better wording, one deleted sleep. A continuous game lasting 35 seconds with a timer included.
Thanks for the model, we were able to port Laya to coreml with 99.5% of the ops on ANE + benchmarked too. it is now blazing fast with 3.7 ms per decision on an M5 Pro. Release: github.com/FluidInference/Fl… Models: huggingface.co/FluidInferenc…
1
2
14
2,456
Thanks for the model, we were able to port Laya to coreml with 99.5% of the ops on ANE + benchmarked too. it is now blazing fast with 3.7 ms per decision on an M5 Pro. Release: github.com/FluidInference/Fl… Models: huggingface.co/FluidInferenc…
This is NOT Jev. Open source. Runs on your laptop. Decides in ~27 ms, about 200× faster than waiting on a hosted LLM. Here it is playing Tetris by itself 👇 brainfunctioncollapse.com/la…
20
111
1,322
115,231
CUA-S1-FORMS now lives in its own repo. FluidUse reads a form in a running Mac app or browser through API, asks what belongs in each field, and types the answer into the real app. ~1 ms per decision on ANE. We plan to convert more compute-use models in the future! Model on Core ML: huggingface.co/FluidInferenc… SDK: github.com/FluidInference/Fl…
We converted it to Core ML and ran it fully on-device: 706K parameters, ~1 ms inference on M5 Pro, up to 98.2% ops on ANE and 99.95% accuracy across 24,370 decisions. we were able to get it down to int4 as well . SDK: github.com/FluidInference/Fl… Model: huggingface.co/FluidInferenc…
1
1
12
2,763
We converted it to Core ML and ran it fully on-device: 706K parameters, ~1 ms inference on M5 Pro, up to 98.2% ops on ANE and 99.95% accuracy across 24,370 decisions. we were able to get it down to int4 as well . SDK: github.com/FluidInference/Fl… Model: huggingface.co/FluidInferenc…
1/ Introducing CUA-S1: a family of System One Models, small, specialized, and built for computer use. Today we're open-sourcing CUA-S1-FORMS, the first in the family: github.com/trycua/cua
9
27
341
41,828
NVIDIA's Nemotron 3.5 ASR Streaming Multilingual is now available through FluidAudio optimized for Apple Silicon so apps can run ~40-language real-time ASR entirely on device, no cloud required. Apps shipping with it today include @SpokenlyApp, @ALTIC_DEV, @SnaplyAI, and others. FluidAudio: github.com/FluidInference/Fl… Original model: huggingface.co/nvidia/NVIDIA… @NVIDIAAIDev
3
3
19
1,348
LeafTok narrates books on-device using FluidAudio for TTS. Zero network calls during playback. Voices ship with the app. The trade-off vs cloud TTS: bigger app, fully offline.
1
2
3
110
Alex retweeted
Impressive. Very nice. Recently contributed a patch for Parakeet V2/V3 to FluidAudio that makes it match this speed, on-device, i.e. ~300x speed factor. Can transcribe an hour of audio in ~12 seconds on an iPhone. And tdt-ctc-110m is ~33% faster than that (1 hr in ~9s). github.com/FluidInference/Fl…
1
2
251