Fluid Inference retweeted
Kev on Core ML 🍎 After seeing this tweet, We got inspired to port Kev-0.8B (the Qwen3.5 variant) and ran Guess Who over 80 Wiki bios, on an M5 Pro, same accuracy, coreml vs original PyTorch: 🚅 37 ms vs 1.07 s per bio → all 80 in 3.0 s vs 99 s 🧠 ~2 GB vs 6.6 GB memory 📦 1.45 GB vs 1.79 GB model Model: huggingface.co/FluidInferenc… Code: github.com/FluidInference/Fl…
UPDATE: Kev-0.6B, 4B, and 8B are now available. Kev is a family of small open source Jev-like decision models you can train and run yourself. This new family is based on Qwen3 using the same LoRA + small pointer head technique as before, but scaled up. Out of domain, on data Kev never trained on: Kev-8B 79.6%, Jev 85.7%. • Drop-in TypeSafe System One API; their SDK works with one `base_url` change • Kev-4B serves on a 32 GB Mac in bf16: ~300 ms for five questions, ~40 ms on an H100 • Repeated documents hit a KV cache: 2-2.5x faster • Apache 2.0 License. Kev-4B trains in 40 minutes on one H100. Kev-8B in 83 minutes. Code, weights, evals: github.com/jaredpalmer/kev
2
7
70
4,647
Fluid Inference retweeted
over a year late but we converted SigLIP 2 to Core ML and sorted 7,349 pet photos into 37 breeds on a M5 Coreml model vs. base PyTorch model, same accuracy: • 36 s vs 102 s • 5 ms per photo vs 14 ms • 262 MB peak RAM vs 4.2 GB • 715 MB on disk vs 1.5 GB model: huggingface.co/FluidInferenc… code: github.com/FluidInference/Fl…
Introducing SigLIP2: now trained with additional captioning and self-supervised losses! Stronger everywhere: - multilingual - cls. / ret. - localization - ocr - captioning / vqa Try it out, backward compatible! Models: github.com/google-research/b… Paper: arxiv.org/abs/2502.14786
6
25
277
16,227
Fluid Inference retweeted
We have converted GLiNER2.5-Decide to coreml. ~4× faster, ~5× less Peak RAM, half the size. model: huggingface.co/FluidInferenc… code: github.com/FluidInference
Introducing GLiNER2.5-Decide, our new 340M parameter open weight, encoder-based decision model. GLiNER2.5-Decide is built for fast, deterministic classification. The model evaluates a set of user-defined typed questions and rules, and jointly decodes their answers, returning structured decisions with probability distributions and confidence scores. We evaluated the model’s performance on Fast Decisions, an unseen, internally generated classification suite based on 17 datasets testing real-world use cases across routing, triage, classification, sentiment, and content understanding. Measured against similar decision models, GLiNER2.5-Decide leads in 9 of the 17 datasets, achieving the highest average score: - GLiNER2.5-Decide: 60.1% - SemIf: 56.4% - JevK5: 57.5% - Laya: 46.6% This performance makes the model a strong fit for use cases like tool calling, model routing, browser and computer use, and LLM-as-a-judge. GLiNER2.5-Decide’s lightweight encoder architecture makes it easy to fine-tune the model for specific tasks, while being efficient enough to run locally on consumer-grade CPUs or in air-gapped environments, giving users greater control over where their data goes and where the model runs. To make building and experimenting with GLiNER2.5-Decide as easy as possible, we're also offering hosted inference. You can now use our API to run inference and fine-tune GLiNER models on specialized tasks right inside your own coding agent: agent.fastino.ai As with previous models, we’re also releasing the model weights on @huggingface under the Apache 2.0 license: huggingface.co/fastino/GLiNE…
17
81
866
66,038
RT @Alex_tra_memory: Parakeet 🦜 Ultra 🤗 is now in FluidAudio, running🏃‍♂️‍➡️ on the ANE 🔥 LibriSpeech clean: 2.13% WER · 127× RTFx LibriSp…
1
Fluid Inference retweeted
FluidAudio now runs Nemotron 3 Diarization on-device in Core ML. Its able to support 8 speakers with 8 different languages like in the below demo. Models: huggingface.co/FluidInferenc… Code: github.com/FluidInference/Fl… Special thanks to @NVIDIAAI for the early access.
When several people talk at once, a transcript can get messy fast. Our new Nemotron 3 Diarization model tracks who spoke when, even when voices overlap. It handles up to eight speakers, has 100M parameters, and is now available on @huggingface 🤗
2
14
200
13,199
Fluid Inference retweeted
we ported @knowledgator’s GLiClass Edge Apps v2 to Core ML on an M5 Pro. 32.7M params, ~1.4 ms per decision on CPU. it played 2048 for 2 minutes and reached the 4096 tile. code: github.com/FluidInference/Fl… models: huggingface.co/FluidInferenc…
The Jeff model, based on our GLiFormer, should be the most parameter-efficient model (a 500M-parameter model) for its performance score. And we created it right before the Jev release. Right now, imagine if we scale it 😉
4
11
103
7,739
Fluid Inference retweeted
since y'all loved the Tetris we decided to make a follow up. the model harness uses fewer bad moves offered, better wording, one deleted sleep. A continuous game lasting 35 seconds with a timer included.
Thanks for the model, we were able to port Laya to coreml with 99.5% of the ops on ANE + benchmarked too. it is now blazing fast with 3.7 ms per decision on an M5 Pro. Release: github.com/FluidInference/Fl… Models: huggingface.co/FluidInferenc…
1
2
14
2,487
🏎️🏎️
since y'all loved the Tetris we decided to make a follow up. the model harness uses fewer bad moves offered, better wording, one deleted sleep. A continuous game lasting 35 seconds with a timer included.
4
527
Fluid Inference retweeted
Thanks for the model, we were able to port Laya to coreml with 99.5% of the ops on ANE + benchmarked too. it is now blazing fast with 3.7 ms per decision on an M5 Pro. Release: github.com/FluidInference/Fl… Models: huggingface.co/FluidInferenc…
This is NOT Jev. Open source. Runs on your laptop. Decides in ~27 ms, about 200× faster than waiting on a hosted LLM. Here it is playing Tetris by itself 👇 brainfunctioncollapse.com/la…
20
111
1,323
115,462
Fluid Inference retweeted
CUA-S1-FORMS now lives in its own repo. FluidUse reads a form in a running Mac app or browser through API, asks what belongs in each field, and types the answer into the real app. ~1 ms per decision on ANE. We plan to convert more compute-use models in the future! Model on Core ML: huggingface.co/FluidInferenc… SDK: github.com/FluidInference/Fl…
We converted it to Core ML and ran it fully on-device: 706K parameters, ~1 ms inference on M5 Pro, up to 98.2% ops on ANE and 99.95% accuracy across 24,370 decisions. we were able to get it down to int4 as well . SDK: github.com/FluidInference/Fl… Model: huggingface.co/FluidInferenc…
1
1
12
2,778
Fluid Inference retweeted
We converted it to Core ML and ran it fully on-device: 706K parameters, ~1 ms inference on M5 Pro, up to 98.2% ops on ANE and 99.95% accuracy across 24,370 decisions. we were able to get it down to int4 as well . SDK: github.com/FluidInference/Fl… Model: huggingface.co/FluidInferenc…
1/ Introducing CUA-S1: a family of System One Models, small, specialized, and built for computer use. Today we're open-sourcing CUA-S1-FORMS, the first in the family: github.com/trycua/cua
9
27
342
41,876
NVIDIA's Nemotron 3.5 ASR Streaming Multilingual is now available through FluidAudio optimized for Apple Silicon so apps can run ~40-language real-time ASR entirely on device, no cloud required. Apps shipping with it today include @SpokenlyApp, @ALTIC_DEV, @SnaplyAI, and others. FluidAudio: github.com/FluidInference/Fl… Original model: huggingface.co/nvidia/NVIDIA… @NVIDIAAIDev
3
3
19
1,353
Huge thanks to NVIDIA and all the launch partners!
2
59
Snaply now supports NVIDIA Nemotron 3.5 ASR Multilingual via @FluidInference — 40 language-locales, on-device, no cloud, free to use. @NVIDIAAIDev @NVIDIAAI #NemotronSpeech #VoiceAl
1
1
94
Today's a big day for Nemotron models. Along with Ultra, we also shipped Nemotron Speech 3.5 that now supports 40 Languages and it's insanely Fast and Ultra Low latency! I collaborated with @Alex_tra_memory, @fluidinference and @ALTIC_DEV to port the model to coreML to make bring the latest Nemotron model to any macbook using FluidVoice! Give it a try and lmk what you think! Link below ⬇️
1
81
Audivize now supports NVIDIA Nemotron 3.5 ASR Multilingual via @fluidinference, adding support for 40 language-locales all on-device. Demo: audivizelabs.com Model: huggingface.co/nvidia/NVIDIA… @NVIDIAAIDev @NVIDIAAI #NemotronSpeech #VoiceAl
1
1
186
Spokenly now supports NVIDIA Nemotron 3.5 ASR Streaming Multilingual via @fluidinference — 40 locales, on-device, free, no subscription. Privacy-first dictation, now global. App: spokenly.app Model: huggingface.co/nvidia/NVIDIA… @NVIDIAAIDev #VoiceAI
1
74
Fluid Inference retweeted
Nemotron ASR Multilingual running on an iPhone 17 Pro in CoreML. Many thanks to @fluidinference for the CoreML model and to @NVIDIAAI @NVIDIAAIDev for the model itself.
3
11
381