Run frontier MoE models on hardware you already own.
Colibrì is a pure C inference engine that runs massive Mixture-of-Experts models by treating VRAM, RAM, and NVMe as one memory hierarchy.
MoE models have hundreds of billions or trillions of parameters, but only a small subset of experts activate per token.
Colibrì moves weights across storage tiers based on which experts are needed.
Hot experts stay in VRAM or RAM. Cold experts live on NVMe and stream in when the router selects them.
It tracks routing patterns, keeps frequent experts closer to compute, and prefetches the next layer before it's requested.
The result: limited VRAM changes speed, not which model you can run.
Key capabilities:
- Memory multitiering: VRAM, RAM, and NVMe as placement tiers
- Expert streaming: Load routed experts from storage instead of keeping all in memory
- Routing-aware caching: Per-layer LRU, hot-expert pinning, and one-layer-ahead prefetching
- Heterogeneous execution: CPU, CUDA, Metal, NUMA memory, partial or full expert residency
- Frontier-scale support: Qwen3.6 (35B) to Kimi K3 (2.8T), plus GLM-5.2, DeepSeek V4 Flash, and others
100% open source.