Automation Foundation Models For Tiny Devices
San Francisco, CA
- Tweets 104
- Following 22
- Followers 3,086
- Likes 171
Pinned Tweet
We release Needle 3: A Sliceable 8-29MB automation foundation model that can match DeepSeek V4 Flash.
One set of weights, every depth from 2 to 20 layers a model of its own, 25-121M parameters at CQ2-bit, built on our Simple Attention Networks and running locally at up to 4k tokens/sec decode speed on a Raspberry Pi 5.
Needle does not chat. Every turn is a function call: give it the tools your app exposes and it picks the right ones and fills every argument from what the user said, or hand it a schema and it returns a typed record. Ask for something no tool covers and you get an empty list, not a guess.
That trade is lets 121M parameters trained on 360B tokens of structured data beat models 10x their size on mobile tool calls and match 2-3x bigger models on structured JSON extraction.
It runs on mobiles, wearables, smart home devices, small robots and microcontrollers, with prebuilt engines for macOS, Linux, Windows, Android, iOS, watchOS, tvOS, the browser and WASI hosts. Try it in your browser: cactuscompute.com/needle
104
260
2,883
404,183
Cactus Compute retweeted
Our friends from Cactus Compute (YC S25) show how Needle 2, a 14MB function-calling model, turns plain English into local actions on a Raspberry Pi 5, using the CPU alone.
Local actions, no AI accelerator, no internet connection:
raspberrypi.com/news/turn-te…
6
20
185
23,501
Cactus Needle x MicroDuck: watch our 29MB action model drive @pollenrobotics' MicroDuck.
Needle acts as the intent brain, turning user input into action.
Excited to deploy on-device once the robot ships. We estimate a ≈150ms time-to-action on the Rockchip.
3
3
18
869
You can fine-tune Needle 3 using the python package locally on your platform and get 4-bit deployable binary, the Cactus Platform buys you:
- Full depth stable fine-tuning by incorporating the original dataset to prevent unlearning.
- Cactus 2bit quantization algorithm applied on your weights, this needs original dataset for lossless quantization.
- Confidence head tuning using Cactus Hybrid algorithms v3, this is compute-intensive and runs on our infrastructure.
- Dataset augmentation and management, evals design, your personal weights repository and more.
- You can simply pass your API key to claude or codex and have it customize your Needle 3 on our pipeline.
read more: cactuscompute.com/blog/finet…
5
10
52
57,616
The small depths lose accuracy on the general benchmarks and get it back when fine-tuned to one product's tools. On DroidCall every subnetwork gains 18 to 36 points, and from 4 layers (29M parameters) up the tuned subnetwork passes DeepSeek V4 Flash. LoRA on the frozen base, merged at export, from `pip install cactus-needle` on a Mac or PC; the 2-bit post-training behind the shipped model runs on the Cactus Platform.
1
1
29
7,476
Thirteen platform folders, one engine under 1MB each, all loading the same weights: macOS, Linux on x86-64, ARM64, ARMv7, RISC-V and MIPS32, Windows x64 and ARM, Android, iOS, watchOS, tvOS, the browser as WebAssembly, and a WASI component. `needle build --platform linux-arm64 --layers 8 --out ./pi` puts the engine and the weights in a folder you copy over; inference never touches the network.
2
1
30
6,746
Every response carries a calibrated confidence score: act above your threshold, show the call and ask below it, or escalate to a bigger model. Needle in front of a private DeepSeek V4 Flash deployment is the setup we run for enterprise work. Guides, the Python reference and the playground are at cactuscompute.com/needle, thoughts always appreciated.
4
2
42
6,364
Every argument is a span of the request. The model writes a one-line derivation first ('living room' -> room; '30' -> brightness), then emits the call under a byte-level grammar compiled from your schema, so the JSON always parses and an enum cannot leave its set. Optional field with no evidence: omitted. Required field with no evidence: the call is withheld. "Except the bedroom": the bedroom call is dropped.
1
1
40
8,812
Structured extraction is the same operation with the record as the only tool. Declare the shape, pass the text, get a typed object whose every field is a span of the passage; an optional field with no span comes back as None instead of a guess. An enum field turns the record into a classifier whose output cannot leave the set.
1
1
37
7,958
Needle 3 is a Simple Attention Network.
1) The dense feed-forward layers are gone: a Monarch Hadamard MLP with 25.6K parameters per layer takes their place.
2) The knowledge they would hold sits in an engram, hashed n-gram tables read by gather at zero arithmetic.
3) 70.8M of the 121M parameters live there, so the full model does the arithmetic of a 50M one: 100 MFLOPs per token against 296 for a transformer of the same shape.
3
2
82
16,607
On Mobile Actions, 961 phone commands scored on the exact call, the 20-layer model reaches 86.0 through the shipped 2-bit binary with the confidence gate on. LFM2.5 1.2B is at 82.4, Qwen3.5 0.8B at 76.0, FunctionGemma 270M at 65.1 and Apple's on-device model at 57.6, all at f16. DeepSeek V4 Flash through its API is at 88.4, the line in every chart.
3
2
46
10,087
The intelligence ladder. Every depth from 2 to 20 layers of the same weights is a trained, deployable model: blocks 0 and 19 are always kept and the rest are added by bisection, so each subnetwork nests in the next. A watch, a Pi and a phone pick their size at deploy time with `needle build --layers n` and run on the same engine.
1
5
54
12,550
We release Needle 3: A Sliceable 8-29MB automation foundation model that can match DeepSeek V4 Flash.
One set of weights, every depth from 2 to 20 layers a model of its own, 25-121M parameters at CQ2-bit, built on our Simple Attention Networks and running locally at up to 4k tokens/sec decode speed on a Raspberry Pi 5.
Needle does not chat. Every turn is a function call: give it the tools your app exposes and it picks the right ones and fills every argument from what the user said, or hand it a schema and it returns a typed record. Ask for something no tool covers and you get an empty list, not a guess.
That trade is lets 121M parameters trained on 360B tokens of structured data beat models 10x their size on mobile tool calls and match 2-3x bigger models on structured JSON extraction.
It runs on mobiles, wearables, smart home devices, small robots and microcontrollers, with prebuilt engines for macOS, Linux, Windows, Android, iOS, watchOS, tvOS, the browser and WASI hosts. Try it in your browser: cactuscompute.com/needle
104
260
2,883
404,183
Needle’s best-kept secret isn’t function calling. It’s structured extraction.
Long-form text in → valid JSON out. Small enough to run in a browser at 14MB.
The JSON schema is guaranteed by the same constrained sampling that we use for tool calling.
github.com/cactus-compute/ne…
2
19
108
5,547
We release Needle Environments - a collection of hand-curated tool schemas to get the best Needle 2 performance.
Watch a 14MB LLM score 90%+ on held-out production tasks:
github.com/cactus-compute/ne…
5
10
95
5,579
Needle 2 was made to be fine tuned.
Show it your environment and some data samples - and the 14MB model beats DeepSeek v4 Flash.
Read more: cactuscompute.com/needle#fin…
8
13
276
13,415
Excellent deep dive into Needle from Andris and the @BetterStackHQ team!
piped.video/watch?v=M24yg6ZM…
3
17
1,725
Week 1 of Needle 2: #3 on GitHub weekly trending, front page of Hugging Face, and in production for @Pebble's new Index Ring.
Thank you for the support 🌵
If you haven't tried it yet, here's a browser playground: cactuscompute.com/needle
1
5
52
2,593
We ran 6,120 tests over five weeks to see how far we could push Needle 2.
Here's a complete playbook for getting a 14MB function-calling LLM to production accuracy:
6
8
78
3,872
7. Fine-tune
Once you saturate out-of-the-box performance, fine-tune Needle on your own dataset!
Guide here: github.com/cactus-compute/ne…
1
1
3
248