A fun podcast I hosted about personal AI hardware with Jaggie Zhu and Austin Mejia from @PLAUDAI

Top takeaways: “Plaud is really about augmenting your professional life and giving you the freedom of perfect memory.” “We index on people for whom the value of their time and presence is worth hundreds, if not thousands, of dollars.” “We’re really trying to augment conversational intelligence, which most of the time involves two or more people.” “We make uncompromising investments in our hardware so that the fail state isn’t your hardware.” “We’re one of the only solutions that actually lets you store your own data on-premise.” “We’re the only player in the space building our own hardware, our own speech-to-text pipeline, and our own infrastructure around it.” “The magic of always-on agents is that they’re one step ahead of you, but in doing so, you compromise on your privacy and your agency.” “The best device isn’t the perfect thing for someone 24/7, but the best thing for the task you have to do today.” “Glasses are an incredible, very spiky form factor, but they’re not for everyone and they’re not for every scenario.” “On-device is a necessary part of the future, from a privacy perspective and from a reliability perspective.”

I was pretty excited after watching the talk that @DrJimFan recently gave at @sequoia that emphasized the first-person view video data (sensorized human data). I think this might be the scaling path for Robotics to hit a scaling law with manageable data cost. The first signal I got conviction that first-person view video data (sensorized human data) might be the scaling path is when I saw @danfei_xu's work that when you pretrain with enough diversity, you can get sensorized human data and Teleops Robots data in the same space See the last two charts: TSNE analysis on mean-pooled tokens from the final layer of the VLM backbone. With no pre-training, it is clear that the model has disjoint representations between human and robot data. But as pretraining becomes more diverse, latent overlap increases, which correlates with performance on our generalization tasks. I wonder whether someone can accurately plot out the scaling law coefficient for this path, and use this scaling prediction to figure out the video hours/diversity needed.

The chart is from Physical intelligence website, as the collaboration between PI and @danfei_xu’s lab at Georgia tech: pi.website/research/human_to… Check out The paper: pi.website/download/human_to…

"When could we get LLM to be able to write their own on-demand Harness for new Job types, like Data scientist AI, Quant researcher AI, Material Scientist AI, etc? Could we build digital version of Peter @steipete or Boris @bcherny?" I asked this question at AGI house last Monday during a dinner with lots of cool guests (see the video). Since I have been working on open-sourcing our agent harness for data scientist AI/Quant Researcher/Financial Analyst AI (github.com/galpha-ai/Alpha-d…), I feel thatLLM has enough training materials Based on the existing open-sourced Harness code like the OpenClaw, leaked Claude code repo, etc, @OriolVinyalsML who leads Google Deepmind's research, called this idea Meta-Harness, and his prediction is that this should be doable in a quick short time framework (quarter? 1 year?)； Furthermore, he thinks that if he had to choose what matters for the next phase, his choice is: LLM > harness. I kind of agree: if LLM can write any harness for newer jobs, figure it how to manage context, design memory system like Peter or Boris, then we are one step closer to AGI.