Introducing VLM Run Gateway: run open-weight VLMs, OCR VLMs, embodied VLMs, and specialized ViTs in one OpenAI-compatible API. You don't need a trillion parameters to parse documents, understand images, or summarize videos. When every model runs on one API, changing models is as simple as changing a parameter. The visual frontier just moved. Your AI bill should too. $ uvx vlmrun gw models

Jev can't see yet, but it turns you can get @googlegemma to play Snake directly from pixels. NO this isn't yet-another Jev demo that uses game-state. It's seeing images, making decisions and responding e2e over websockets in <240ms p99 at ridiculously low costs (~$0.00007/image). We just added @typesafeai SDK compatibility to our @vlmrun gateway where you can try out a whole bunch of VLMs with System One support. Check it out, and it's free to try! Happy hacking on the weekend!

I made a computer vision tool for rock climbing analysis in 3D using iPhone LiDAR! Having learned a lot from sharing my previous rock climbing demos, I realized that a lot of rock climbing analysis is well-suited for 3D. Even something as simple as supporting videos where the person filming moves with the climber requires 3D information. To get the depth information, I used my iPhone 15 Pro’s LiDAR depth sensor through my local iPhone app. I found that the depth measurements help a lot. I think the holds activation is better, and I like the final view of all of the holds in 3D. It’s also interesting to see the distance traveled in meters. Plus, it looks cool and it feels like a video game 🙂 In short, I think this new demo is an improvement in that climbers can see the real-world distance traveled and a visualization of which hand and foot activated which hold. I recorded the video and depth measurements from my iPhone app, and I ran the rest of the analysis on my computer. I used ViTPose+ Large for pose estimation and SAM 3.1 to segment the holds, both models accessed through the @vlmrun Gateway. Let me know what you think! The analysis code is open-source on GitHub: github.com/jeremyipark/visio…

We're at @foxglove #actuate26 to connect with fellow roboticists to learn more about about how teams are scaling robotics datasets, RL, and real-world deployments. If you're still hand-labeling > 50% of your robotic data, let's chat. We're releasing a new auto-labeling VLA/VLMs that you'd definitely be interested in trying out. DM @spillai or @dineshredy if you're around

Gemini Robotics ER 2 is now available in our Orion 2 visual agent harness! Orion 2 pairs Gemini Robotics ER 2's real-world understanding with an expanded visual toolkit, turning embodied reasoning into composable, inspectable programs.