These are so so so good. I hardly watch YouTube, but savor these to enjoy on the weekend. Thanks for publishing these!

Nice!

AI should teach like this.

non-trivial error @ 57:30: the number of functions from 4 bit to 1 bit is 2^16 (65536), not 2^4 (16)

Love this, fantastic follow up

wow

This sounds fascinating! I'm looking forward to exploring the connections between logic gates and advanced computing. Can't wait to learn more from your insights!

the transcript is the same amount of code as a 1300 line php app

u already know it’s gonna be a banger

this is god's work! genuinely amazing

where do memory bandwidth limits enter since macs are cheap but moving data dominates

The most interesting part the "data tax" and how systolic arrays in TPUs solve it. Objective of a chip is to do maths but to do that it needs to have data to do the math on. CPU/GPU works like a personal chef who has to walk to the pantry (registers - data stores), grab a tomato (data), walk back (muxes), chop it, then walk back to the pantry for an onion. Most of the chef's time is spent walking , not chopping. TPU works like an assembly line (Systolic Array). One worker stays in one spot with their knife (the weight). The tomatoes (data) slide down a conveyor belt. The worker chops as they pass by and slides them to the next person. No one has to walk anywhere, so the "chopping" (math) happens at maximum speed. By using this "assembly line" design, AI chips can perform quadratically more math while only increasing the "communication" (data movement) by a tiny, linear amount. This is the primary reason modern AI chips are so much faster than general-purpose computers.

getting so spoiled with all these blackboard episodes recently, huge investment in the dwarkesh cinematic universe

Love the jump from logic gates to architecture tradeoffs. That bridge is where a lot of product intuition comes from. Founders use GPUs every day and still treat them like magic boxes. This kind of lecture helps.

Do one of these with @karpathy

Awesome format! I really enjoy the super technical deep dives please keep them coming

I still have the first two blackboard sessions in my watchlist to finish over this long weekend 😂 But keep them coming — this series is incredible! Thank you!

this is like EE-->Compsi 4 years in 80 minutes lol

I tried - really. This didn’t clear up anything for me. (It’s me, not you guys) 😗

Your data movement chapter is the one that scales into the real problem. At datacenter size the multiply is basically free. Moving the bits is the whole power bill. That is why GPU vs TPU comes down to dataflow and memory locality, not the logic gates.

AI infrastructure sounds abstract until you walk it down to gates, arrays and memory movement. The future keeps pretending to be software while behaving like hardware.

releasing pods faster than the speed of light, love it💯

Really enjoyed this @dwarkesh_sp episode with @reinerpope from @MatXComputing. The chip design breakdown was so good 🧠 Main opportunities: • AI chipmakers • memory bandwidth • low precision compute • TPUs and accelerators • data centre infrastructure • FPGA and ASIC design • companies reducing data movement costs Full summary 👇 driftnote.net/s/6si3y1r5t5

I asked sonnet 3.7 to build a react simulation for systolic array multiply, in like 2 hours, it gave me a fully interactive simulation. That thing is quite interesting, see easy we can observe these algorithm really works today

Does DP invest in every company..whose people he interviews ? 👀

love you're content dwerkesh this is the kind of format i expected from lex fridman

Dwarkesh on a generational run

these are great

the blackboard is an elite addition

The “Behind the Interface” book explains all that in a much easier way - basically how to build a computer from silicon - one layer at a time a.co/d/0i7FOqVh

good sir this is priceless content, mad respect

I dont watch a lot of podcasts....but this one is just too precious to not watch! Less goo!!

This format is very good

Legendary

Sharp tour through cores and brain-inspired design