The Bottleneck is South of the Model @Bhorowitz (Ben Horowitz), @martin_casado (Martin Casado) and @RaghuRaghuram (Raghu Raghuram) (@a16z), interviewed by @ErikTorenberg (Erik Torenberg) (The a16z Show) The model stopped being the constraint about 3 years ago. What binds now is everything underneath it: chips, memory, interconnect, power, cooling, and further down to reinforced concrete and certified electricians. All of it was designed for a different era of computing. The largest company-building opportunities have moved back down into hardware for the first time in 20 years, and capital now buys time that engineering headcount used to have to earn. 1. South Of The Model. Raghu Raghuram says the models stopped being the bottleneck about 3 years ago, because AI is now used to make them better faster. The binding constraint sits in what he calls everything south of the model. That stack goes deeper than the usual definition of infrastructure, past servers and storage and networking, down to the copper mines. Every layer of it was built for a workload that no longer exists. 2. The Broken Man-Month. Ben Horowitz says the oldest law in startups just stopped applying. If you had a 2-year lead and a rival hired a thousand engineers to catch you, they wrecked their company; 9 women cannot have a baby in a month. Now a rival takes $3 billion, lights up a cluster, and a Grok or a Kimi shows up and is immediately real. Money buys time in a way it never has, and Horowitz says everyone is still psychologically adjusting to that. 3. Infinite Demand, Physical Margin. Martin Casado says AI inverts what a business worries about. Demand is effectively infinite, so the open question is whether you can serve it profitably. The efficiencies that decide that live in the physical limits of hardware. Software margins used to arrive on their own once the business worked, and in AI you have to engineer them into the hardware. 4. The Dark Fiber Comparison. Supply across the board is booked out to 2028, and Horowitz says the industry has reached multi-day auctions for a few thousand GPUs. The leading memory vendor told the Hot Chips conference that today's demand alone would take 3 years of capacity to fill. The 1999 buildout was speculative: most of the fiber went in the ground and stayed dark, because the users to consume it did not exist yet. Every GPU built today is pre-sold, and people resell them for 4 times what they paid. 5. Autocatalytic Tokens. "Nobody likes to use AI more than AI." Reasoning is inference, chain of thought is inference, long-running agents are inference, and writing a GPU kernel with AI is AI consuming AI. Each step from chatbot to reasoning to agents multiplied the tokens per task by an order of magnitude, and Horowitz expects token demand to grow close to 1,000% a year. Engineering had a natural governor in the mythical man-month, and token consumption has none. 6. One ASIC Per Model. Casado does the math. A frontier model costs $3 to $5 billion to train, so inference has to pay that back and realistically twice that, call it $10 billion. Save 20% on $10 billion and you have saved $2 billion, which is roughly what a custom ASIC costs to build. Model weights are fixed where software is dynamic, so building a chip per model now pencils out, which tells you how bespoke this hardware layer is about to get. 7. Agents As Employees. Casado describes 3 framings the industry has moved through: add AI to a product as a search bar, then chat with it and have it chat back, then make it an extension of you that holds your keys and passwords. The current framing gives the agent its own computer and its own browser and treats it as an employee. Casado now asks whether an agent can do a task before he does it himself, including email triage, where it knows to check with him first. Horowitz is blunt that nobody has cracked this: these employees burn tokens and produce nothing, forget things, invent things, and create security problems. 8. The End Of AC Power. Rack power is going from 5 to 10 kilowatts to 100 to 150 kilowatts, roughly 70x the compute density. AC power stops working at that level, so data centers move to DC, which needs its own cooling and is dangerous at 800 volts. About 2% of US electricians are certified on DC power, and Meta now runs a free program to train more. AI is supposedly taking every job and is about to create a generation of electricians. 9. The Gigawatt Gap. New data centers will need about 44 gigawatts of additional power by 2028 against maybe 25 gigawatts of expected grid additions. A gigawatt runs roughly 50,000 homes; Horowitz grew up in Flagstaff, Arizona, a town of 40,000 to 60,000 people that uses less than one. You cannot compress these lead times by working weekends, because permits, transformers and turbines are all short while demand grows 10x a year. New companies now chase GPUs in Mexico and Australia, so blocking data centers here exports the jobs and the economic upside with them. 10. Silver Bricks. Alex Rampell once pitched Facebook and Dan Rose told him he could collect a lot of silver bricks, but Facebook had so many gold bricks it could not pick them all up. Horowitz puts Nvidia in that position and calls it the law of markets. The silicon incumbents are worth multiple trillions, so 5% of their market is still a massive private company that they have no reason to chase. Markets fragment as they expand and consolidate only when growth slows, which is how Arista, Cisco and Juniper happened. 11. Systems Founders. The share of deals from top founders involving hardware went from about 3 to 5% to north of 20 or 30%, and first rounds run into the hundreds of millions before there is a product. Raghuram says these have to be systems founders: architect the chip, then work out who manufactures it and who supplies them, the way Jensen Huang thinks through the whole supply chain before designing anything. That is why the founders skew older, and why Elon Musk and Travis Kalanick both built software companies first. Horowitz says the industry and academia defocused hardware for 20 years, so the biggest legacy of SpaceX may be the founders it produced rather than the rockets.

Awesome to see the @wafer_ai (software) platform outperform a chip vendor on real-time inference.

Exceptional founders possess two traits that look like weaknesses but are actually strengths. First, they are contrarian thinkers. They challenge everything: investors, employees, customers, conventional wisdom. That contrarian energy is what lets them see around corners and push through resistance. But the same trait makes them hard to work with, slow to trust, and sometimes unable to delegate. Second, they are obsessed, to the edge of sanity. The best founders are consumed by the problem in a way that looks unhealthy to normal people. They don't clock out. They don't have hobbies in the conventional sense. That intensity compounds, but it also burns people around them and can mask real personal costs. The best founders I've worked with have an almost unhealthy attachment to being right about their vision of the world. They can't let it go. They'll sacrifice money, relationships, comfort, sleep, all of it, because something inside them needs to see this thing exist.

Exactly right. Still true.

ARR growth in isolation without looking at the burn is basically irrelevant. It means that the company might be entirely dependent on continually tapping capital markets for survival. Now in rare cases, that’s perfectly fine, but for most companies, that are not 1 => 50 in a year growers, they need to operate and grow efficiently, such that they don’t die when the markets turn.

This is an awesome customer story (I’m a proud @brilliantorg investor and a huge supporter of their mission). We heard the same themes over and over again when speaking to @wafer_ai customers.

The Consupocalypse Consumer commerce is having its own SaaSpocalypse moment thanks to Muse, Instinct and personal agents. (eg: Month to date Booking share price is -21%, Expedia is -18%, and Airbnb is -18%) IMO, when the dust settles, companies that will be left standing will have some or all of the following: 1. Differentiated physical (not just digital) infrastructure and fulfillment 2. Hard to access supply 3. Powerful local network effects 4. Proprietary data loops that allows better personalization than a general purpose agent can 5. Differentiated Membership / reward programs that drive true loyalty 6. Consumer trust We will go beyond theoretical moats discussions and truly see which moats are durable.

I would tell my 25 year old self three things: 1. Build relationships with people and don't be transactional. 2. Have high agency, be an independent thinker, build your own conviction and don't wait for consensus or approval. 3. Focus on the small set of things (in both work and life) that do actually matter.

Very excited for this conversation on Oct 8 with @namibaral - please register if you'd like to attend!

Thank you @lizwessel for the kind words!

A big thank you to @gokulr for building usetranscribe I’ve become its power user, and the summary and section insights have become my go-to before watching any podcast or interview. Really appreciate you keeping it free to use!

Remote work is exceptionally hard for startups, which by definition need to innovate and learn / invent / build new things together. It’s well suited for companies that are maintaining or sustaining existing products, but not for disruptive innovation where in-person can 10x the speed of iteration and learning.

The best investors amplify the sense of destiny of founders. Great example from Mike Moritz below, where he compared a fledgling Yahoo to Apple.

Two patterns i've observed in generational founders: 1. Perpetual / persistent dissatisfaction with the status quo. 2. Ambition / TAM that expands as the company grows.

📈📈

SOTA inference engine that beats Apple’s MLX on Apple Silicon inference. kudos @0xSigil 🫡