The biggest bottleneck in the GPU industry at the moment is not chips, CoWoS, or even power. It's credit. Sharing some thoughts on compute markets, residual value, and where compute financing is heading. Substack link in the comments.

Wafer beat Cerebras on latency for @ycombinator's AI Office Hours. GLM-5.2 on Wafer averaged 379 ms versus 674 ms for Gemma 4 31B on Cerebras. that's 44% lower latency with a much larger model. YC wanted people to get startup advice from AI versions of its partners at conversational speed. after testing lightweight Gemma and OpenAI models, they moved to a dedicated Wafer endpoint. Wafer agents tuned the serving setup for YC’s request rate, cache usage, and prompt and response lengths. users spent 2.5 minutes longer talking to its AI partners on Wafer compared to other providers. read how YC built the experience and landed on Wafer 🧵 link in thread

@brilliantorg 3x’d their tokens per second and cut inference costs by 50%, by making its AI tutor fast enough to answer in real time instead of speculatively prefetching answers. To keep Koji, their AI tutor, responsive during math and coding lessons, the Brilliant team was speculatively prefetching AI-generated responses so students wouldn’t have to wait. On Wafer's dedicated GLM-5.2 endpoint, Brilliant reached ~250 ms time to first token and 300+ output tokens per second. This was a 3x throughput increase over their previous provider. That speed let Koji answer learners in real time, so the Brilliant team could stop pre-generating responses and remove the prefetching layer. Read how Brilliant made the switch! 🧵 Link in Thread

we launched the most comprehensive ai performance engineering repo in the world now we'll be posting every single resource follow and save to keep up with the series. links in thread 🧵 part 5: Roofline: An Insightful Visual Performance Model for Floating-Point Programs and Multicore Architectures the authors introduced a model for relating arithmetic throughput, memory bandwidth, and data reuse. applied to AI workloads, it gives a way to evaluate batching and kernel optimizations against the hardware's limits. the paper's concepts connect to these AI performance decisions: operational intensity measures floating-point work per byte of DRAM traffic after cache reuse. during dense transformer decoding, a linear layer processing a small token batch can read a large weight matrix for little arithmetic. compute and bandwidth roofs bound throughput. if weight reads limit an inference kernel, reducing transferred bytes can lower its memory-time bound. increasing peak arithmetic throughput alone leaves that bound unchanged. the ridge point marks the minimum intensity needed for peak compute throughput to be possible. batching tokens that share a weight matrix can increase work per weight byte, moving the matrix multiplication toward that threshold. computation ceilings account for limits in instruction parallelism and operation mix. for neural-network matrix multiplication, check Tensor Core use and compare throughput with the roof for the kernel's execution path and precision. bandwidth ceilings account for memory access patterns and data placement. for GPU tensor operations, changing the layout or thread-to-data mapping to coalesce scattered accesses can improve useful bandwidth. data reuse raises intensity when it reduces DRAM traffic for the same arithmetic work. a tiled matrix multiplication can reuse operands on chip; measure whether that reuse reduces DRAM byte traffic. the memory level determines which bytes to count. if a matrix multiplication reuses operands from L2, compare its L2 traffic with L2 bandwidth and its DRAM traffic with DRAM bandwidth. the authors demonstrated the model with 4 floating-point kernels on 4 multicore systems. for AI performance work, use it to assess weight reuse, memory layout, and arithmetic execution before choosing an optimization to test.

Nope. OpenAI and Anthropic trending towards a duopoly would be negative for AI infrastructure demand and a disaster for the application layer.

who is the first GPU prop trading firm? dm me

Inference providers hate this man... Your LLM deployment can deliver 30 to 50% better performance after initial deployment. If your current inference provider isn't learning your workload and tuning the stack continuously after deployment, you need another provider. Wafer's continual inference agents do that work. Wafer's continual inference agents profile your production workload to identify where scheduling, decoding, or GPU memory pressure limits performance. The agents benchmark changes to batching, serving engines, and GPU kernels against your workload. Each candidate must preserve correctness and meet your reliability targets before it ships. The agents keep profiling as traffic changes, identifying new bottlenecks and further opportunities to improve performance per dollar. Watch @gpuemi explain how that work continues after deployment in his conversation with theCUBE.

Why am I so focused on networking over raw compute? Look at GPU utilization 53% of surveyed data center operators report running at only 50–70% capacity during peak demand. GPUs aren't sitting idle because they lack compute power, they're sitting idle because they're waiting for data. They wait on storage, they wait on CPUs, and above all, they wait on other GPUs to send results back across the network. If you add more compute to a cluster without upgrading the pipes, you just get diminishing returns on your capex. $LITE $COHR $AAOI $CRDO

can someone make a group buy discord for compute like they did for Kanye leaks

very well written and completely agree

Let Down son or Weird Fishes daughter?

Nearly half a year of silence. We spent it studying one problem: how far RL can scale. MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks. Streaming the run: mimo.xiaomi.com/rl/

@ycombinator was testing lightweight models for its ai office hours. the goal was useful startup advice at conversational speed. they moved to glm-5.2 on a dedicated wafer endpoint. the wafer agents tuned the serving setup around their prompts, cache usage, and traffic. yc then tested it against gpt-4.1 mini on openai and gemma 4 31b on cerebras. wafer had the lowest average llm latency at 379ms. users on wafer talked to the ai partners for 2.5 minutes longer on average! read how yc found the right setup 🧵 link in thread

@ycombinator built AI versions of its partners to help more people work through their startup ideas. For its Office Hour Simulator, the team needed useful answers delivered quickly enough for a spoken conversation. They had been testing lightweight Gemini and OpenAI models before moving to GLM-5.2 on a dedicated Wafer endpoint. YC then compared that deployment with GPT-4.1 mini on OpenAI and Gemma 4 31B on Cerebras, evaluating answer quality, latency, and conversation duration. The Wafer configuration delivered 31% lower average LLM latency than OpenAI and 44% lower than Cerebras. Users spent 2.5 minutes longer talking to YC’s AI partners when using Wafer. Read how YC built the experience and landed on Wafer 🧵 link in thread

vibes definitely made up for how much I lost thanks @CloudflareDev for cohosting!!

DeepSeek-V4.1-Flash is live on @OpenRouter!! pick @wafer_ai as your provider (ss taken 9.12.26)

@wafer_ai poker night. nyc. sept 14. 8pm. 40 seats. give me ur money. link in thread.

hi my name is gpumich and i'm co-hosting poker night with the @wafer_ai boys. come forfeit your chips to me luma.com/ala26o53