Most people who use AI and invest in AI Infrastructure operate at a high abstraction level and chase trends. However, developing a deep and broad understanding of AI Infrastructure is difficult because domain-specific technical data lives in isolated silos and isn't translated well to a broad audience. As a result, engineers often don't have system context of what they are designing and investors often don't have a decent mental model of what they are investing in. My goal is for investors and engineers to think more like system architects, because architecture decisions have financial implications and influence day-to-day execution of engineering teams. Over the past year, I've travelled to the semiconductor industry's flagship conferences and learned how these subsystems worked from the industry's foremost experts. I've written a complete suite of mental models on the subsystems that comprise AI Infrastructure: - Mixed-Signal ICs (SerDes, ADCs, PLLs, Bandgaps) - High Speed Optical/CPO - High-Power PMICs - High-Speed Signal Integrity - Advanced Packaging - AI for Chip Design / EDA - AI Accelerators - Memory (HBM and SRAM) Link to the website is in my bio and in the comment below 👇

The AI infra industry mistakes working at a higher abstraction level for having higher intellectual value.

Just wrapped up day 0 of PDCs at IMAPS Symposium! One key observation about hybrid bonding: I noticed there is a clear difference between how the broader investment community treats it (It has high potential to replace regular C2/C4 bonds) and how engineers talk about it. I sat in seminars with packaging experts, including John Lau and Tom Strothmann, former VP of Besi NA, who both note that hybrid bonding is not the preferred solution for all designs now, but could see increased demand in the future. One major bottleneck for adopting D2W Hybrid bonding for HBM is throughput per machine. Hybrid bonding is slow with BESI 8800 CHAMEO Ultra Plus pushing out ~1500-2000 units per hour. If you do the math for 16 layers, this comes out to ~100 HBM units per hour, very slow compared to TCB and MR-MUF.

First day of IMAPS PDCs done! Packaging conferences have the best venues

I can hear John Lau getting excited in the room next to me 😅

Co-packaged optics investors frequently over-index initial CapEx and vastly underestimate the OpEx from reliability and field serviceability down the line.

I don’t really read into doomsday headlines, but this was quite funny

100% agree. I took John Lau's seminar at ECTC and will never forget it

More widespread deployment of co-packaged optics will run up thermals of compute in a feedback loop: -More data moved = more compute done -More compute done = more thermals that needs to be removed and affect the reliability & refractive index of the optical devices themselves This is especially troublesome in DWDM modulators close to the chip that need precise thermal control

A sub-100nm misalignment in Fiber Array Units (FAU) and hybrid bonds can spell the difference between bleeding-edge compute and wasted CapEx. The bottleneck for scaling out co-packaged optics isn't optical engine or laser design—it’s the supply chain that enables the packaging, testing of known-good die (KGD) and precision alignment. Next week I'll be attending the IMAPS and SiPh Summit where I'll be learning about these issues firsthand and sharing expert takeaways on the technical challenges to manufacture and integrate CPO at scale.

How HBM BW is running into a signal integrity physics wall, limited by packaging: siliconcodesign.com/p/a-comp…

Bottlenecks never disappear; they just shift to another layer of the stack. Before I came to the AI Infra Summit, I was biased into thinking that shifting to the optical domain to speed up scale-up and memory BW will completely solve the problem of low compute utilization by moving data faster. However, after sitting through keynotes and expert panels in the data movement track, I realize that I failed to appreciate the system-level complexity that goes into moving and orchestrating massive amounts of data from concurrent users. I realized how "improvements" tend to add additional complexity burden and move bottlenecks to other parts of the system. More scale-up and memory BW helps to push performance, but isn’t the only means to improve overall cluster data throughput. Next week, I'll be releasing my 3-part post that explores how a token goes through a network and the challenges with scaling performance and increasing utilization. It synthesizes from a wealth of knowledge from domain-specific experts from major players such as Credo, Astera, AMD, Microsoft, Broadcom, Marvell, as well as emerging companies like UpscaleAI, iPronics, and Salience Labs. Make sure you follow and subscribe to my Substack, "Silicon-Co-design" so you'll be the first to know about it.

A mental framework to understand the current state of the art packaging trends of co-packaged optics

Everyone talks about the need for a system approach in AI infrastructure, but in order to do so, one needs a decent mental model of the system itself. The problem is that most system-level knowledge tends to be gatekept behind domain-specific silos and $3,000 conference passes. IC Engineers are actively disincentivized from seeking those out since many companies narrowly define roles and fail to justify conference expenses for them. Without a mental framework, dealing with system-level issues often runs the risk of staying shallow. As a result, most engineers treat system-level issues not as an elaborate, delicate balance of trade-offs, but high-level requirement management. Most of it involves gathering feedback from domain-specific experts over whether certain requirements are feasible or not. My website helps combat technological and career stagnation by giving engineers sufficient system context across the AI infrastructure stack. I broke through these barriers this past year developing these working mental models from industry-leading conferences and hope that people benefit from it.

In corporate environments, first-principles thinking is frequently treated as a political liability rather than an architectural virtue.