Voice agents need to respond in real time but the models powering them are trained on information from the past. We built an architecture with our friends at @Linkup_platform to give a voice agent access to the live web without adding enough latency to break the conversation.

Running Qwen3.6 with DFLASH speculative decoding on a single NVIDIA B200, we measured: • 300–510ms model time-to-first-token • ~700ms web retrieval • <1 second perceived time-to-first-audio • <2 seconds to a complete grounded response

We just released Memory + GPU Snapshotting in beta on Cerebrium. For some workloads, this reduces GPU cold starts by 80%+ by restoring fully warmed CUDA workloads in seconds instead of rebuilding them from scratch on every scale-up.

The hard part is making this work for real CUDA workloads. GPU checkpointing is not just “dump memory and reload it.” Real workloads hold references to sockets, IPs, filesystem paths, NVIDIA driver state, multiprocessing workers, local RPC files, CUDA work, and hardware/runtime compatibility.

Today we’re sharing how we built Thalamus: Cerebrium’s highly available distributed router for global AI workloads. It helps power low-latency, resilient AI applications across fragmented compute for companies like @tavus, @superwhisper, @useCamb_AI Read more: cerebrium.ai/blog/thalamus-o…

From there, Thalamus ranks clusters by hot/warm/cold capacity, latency, health, cost, cache locality, and customer constraints like data residency. It doesn’t just pick “the best” cluster. It makes weighted routing decisions based on the cluster most likely to serve the request quickly. We also log every decision so we can replay production traffic, simulate routing changes, and understand what would have happened before shipping changes.

From there, Thalamus ranks clusters by hot/warm/cold capacity, latency, health, cost, cache locality, and customer constraints like data residency. It doesn’t just pick “the best” cluster. It makes weighted routing decisions based on the cluster most likely to serve the request quickly. We also log every decision so we can replay production traffic, simulate routing changes, and understand what would have happened before shipping changes.