Founder, edge0.ai | On-device foundation models. High intelligence, low memory, low power | MIT TR35 linktr.ee/samuelzeng
San Francisco Bay Area
- Tweets 85
- Following 243
- Followers 4,136
- Likes 180
Pinned Tweet
A 35B language model running on an iPhone using only 1–2.5 GB of peak memory.
No cloud. No remote server. No desktop GPU.
Today, we’re open-sourcing Edge0 — a framework for running large AI models fully on-device.
395
835
10,279
720,541
Speech recognition is easy—until you ask it to listen forever.
Today we’re open-sourcing Audio8 ASR Infinite:
Ultra-low latency, unlimited audio, 24/7 transcription, no drift.
Built-in semantic turn detection keeps it listening like a human ear.
New SOTA for streaming ASR.
52
198
2,317
160,961
How does Audio8 ASR Infinite keep listening 24/7 without drifting?
In simple terms:
⏱️ No chunks. Audio and text share one clock: every 80ms, the model emits exactly one token. 12.5 decisions per second.
🎛️ Delay is a dial, not a constraint. 240–560ms: spend latency, buy accuracy. Same weights either way.
🪟 A 30s rolling window keeps memory flat. Old context is dropped, and the positions rotate back to zero—exact, not approximate. Hour 10 behaves like minute 1.
💬 Turn-taking is semantic, not acoustic. It can tell a thinking pause from an actual end of turn.
2
1
44
5,336
We benchmarked Audio8 ASR Infinite against two streaming baselines at the same operating point: 480 ms delay, 80 ms clock.
Lower is better.
🐉 Chinese is where it pulls away.
AISHELL-1 CER: 1.75 vs 16.80 for Voxtral-Mini-4B-Realtime.
AISHELL-4: 2.89 vs 16.46.
🇺🇸 English is closer—and we lose on clean speech.
Voxtral leads on LibriSpeech test-clean: 2.21 vs 3.04. We prioritized Chinese during post-training.
📊 Four-set average: 3.62 vs 10.25 vs 9.52.
⏱️ Nemotron-3.5 was measured at 560 ms, a longer delay than ours, giving it an easier operating point.
2
1
29
3,887
A 35B language model running on an iPhone using only 1–2.5 GB of peak memory.
No cloud. No remote server. No desktop GPU.
Today, we’re open-sourcing Edge0 — a framework for running large AI models fully on-device.
395
835
10,279
720,541
Edge0 Technical Breakdown
Edge0 is an open-source, streaming MoE inference framework for running large models on devices with tight memory budgets.
💾 SSD expert offload: MoE expert weights are memory-mapped on SSD and streamed on demand. Hot experts stay cached in memory, while long-tail experts are prefetched layer by layer.
🔮 Prerouter routing prediction: Edge0 predicts the next token’s expert routes one step ahead, overlapping SSD reads with the current forward pass.
🧩 Parallel LoRA: LoRA adapters are applied at runtime rather than merged, so a single read-only base model can serve multiple adapter sets.
Put simply: the full model lives on storage; only the weights needed for each step are in active memory.
Report：github.com/Edge0-AI/edge0/bl…
7
10
208
22,800
Thank you for all the attention on Edge0.
1. We’ve updated the Edge0 technical report: github.com/Edge0-AI/Edge0/bl…
2. Phone and PC apps: We’ve received a large number of requests for native phone and PC apps. In the next version — coming very soon — we will open-source the app layer, including the phone app and the desktop app. Anyone will be able to experience the model running directly on their own device.
3. Larger models: We are now training larger models (e.g. >100B), with the goal of making them easy to run on PCs and laptops. They may be released together with the following versions.
A 35B language model running on an iPhone using only 1–2.5 GB of peak memory.
No cloud. No remote server. No desktop GPU.
Today, we’re open-sourcing Edge0 — a framework for running large AI models fully on-device.
4
11
154
27,875
Over the past two months, we’ve open-sourced multiple generations of on-device Audio models to the community through Audio8.
The current portfolio includes:
• ASR: 0.1B, 0.3B, 0.6B, and 3B
• TTS: 0.1B, 0.3B, and 0.6B
These releases are designed for practical local inference, with deployment targets spanning phones, PCs, and other resource-constrained devices.
22
40
317
89,817
Our ASR releases, sorted by model scale, with deployment variants grouped together:
- 0.1B / 0.3B line
Audio8-ASR-0.1B
huggingface.co/Audio8/Audio8…
iOS ANE
huggingface.co/Audio8/Audio8…
ONNX Runtime
huggingface.co/Audio8/Audio8…
- 0.6B line
ARK-ASR-0.6B
huggingface.co/Audio8/ARK-AS…
INT8 ONNX
huggingface.co/Audio8/ark-as…
3B line
ARK-ASR-3B
huggingface.co/Audio8/ARK-AS…
Our TTS releases, sorted by model scale:
- 0.1B line
ONNX INT8
huggingface.co/Audio8/audio8…
Preview 0.1B
huggingface.co/Audio8/Audio8…
0.3B line
GPA
huggingface.co/Audio8/GPA
0.6B line
Preview 0.6B
huggingface.co/Audio8/Audio8…
ONNX INT4
huggingface.co/Audio8/Audio8…
The Hugging Face page also includes GPA-v1.5, an additional 1B-scale TTS release:
huggingface.co/Audio8/GPA-v1…
3
1
24
1,172
We’ll continue open-sourcing more Audio8 versions, model sizes, deployment formats, and capabilities.
Thank you to everyone who has downloaded the models, tested them locally, and shared feedback with us.
On September 10, we’ll make a major open-source release:
Edge0 — a breakthrough in on-device AI.
1
5
701
Something big is coming on September 10. A major open-source release from us.
35B parameters. One iPhone. No cloud.
We trained Edge8-35B, an ultra-sparse MoE with a jointly trained dynamic expert planner, and built an SSD-streaming inference engine around it.
In this demo: 44 tok/s, ~1.06 GB peak memory.
A truly usable large-model stack for on-device AI.
Model, runtime, and paper: open source soon.
2
1
9
2,670
Audio8 TTS Preview 0.1B — now on Hugging Face as ONNX INT8
~100M params · 11 languages · zero-shot voice cloning
Runs on CPU only (~0.4 GB RAM) — no PyTorch, no CUDA
44.1 kHz · OpenAI-compatible API · Apache 2.0
Try it 👇
huggingface.co/Audio8/audio8…
#TTS #VoiceCloning #ONNX #OpenSource
23
73
688
37,245
The world’s smallest Transformer-based TTS model?
We’re open-sourcing Audio8 TTS Preview 0.1B — an approximately 170M-parameter multilingual speech model with zero-shot voice cloning that delivers surprisingly strong, cloud-level quality in a dramatically smaller footprint.
38
112
1,057
75,197
The model, codec, tokenizer, processor, and inference code are now available:
Model:
huggingface.co/Audio8/Audio8…
Try it, test the voice cloning capability, and share your feedback.
3
9
65
4,495