If what you say is true, the VISION and the LANGUAGE on-device could be dangerous First came AIs that could see. Like the AI in your self-driving-ish car and your phone’s face ID. But these vision-only models had no ability to use language or think. Language and “thinking” ability came with the LLM explosion. But, at first, language models couldn’t see. Then vision and language merged. Your AI has been able to see, think about what it sees, AND use language for at least a couple of years. These “vision-language models” are typically huge models in the cloud running on massive GPUs. But what about the many cases when you can’t, or don’t want to, use AI in the cloud? At @SuperfocusAI, we’re building on-device AI systems with VLMs for these situations. Our demo video below uses the case of AI on a drone to show what is possible. The challenge of on-device AI is that the chips used in most devices have only a tiny fraction of the firepower that cloud GPUs have. So you can only use a very small VLM. But you still have to pack all of the intelligence that you need into this small VLM. Imagine a scenario where the drone has to deliver supplies to a “medic truck”. The video shows what a vision-only model (top) would do in this scenario versus a VLM (bottom). Both models are running together live on a chip that is commonly used in drones. We can also do this on even cheaper, lower firepower chips. The vision-only model does a good job of identifying trucks. But it can’t think about what it is seeing, so it can’t distinguish between the trucks without further training. It is impractical to train in advance for every nuance that might arise in the moment. In comparison, the VLM just needs to be told that the medic truck is the one with the red cross on it. With that information alone, the VLM reasons about what it is seeing and correctly picks out the medic truck from the group of similar looking trucks. We’re using vision and language together in on-device AI systems to add contextual intelligence like this in all sorts of use cases. More examples of what we’ve built to come in future posts… En garde! @hsu_steve @reindsummit

Join us this Thursday for a webinar sponsored by multiple professional societies on HCV Elimination. Register for free online at liverlearning.aasld.org/Publ…… for this webinar that will take place on September 10th.

Join the AASLD HCV SIG, in collaboration with multiple professional societies, host an upcoming webinar on HCV Elimination. We are excited to feature an all ⭐️ line up of speakers, including Drs. Natasha Martin @INHepSU Norah Terrault @AASLDtweets and Frederico Garcia @ESCMID.

UAV autonomous action via edge VL model This video shows a Vision-Language model Superfocus has running on inexpensive (~$50 chipset) hardware, interfaced with the control system on a drone. The text at the top is the LLM's interpretation of what is happening in the scene, and the green text describes its actions. It can differentiate between different vehicle types, different uniforms, orient itself to capture face-recognition data, track a vehicle or person, etc. None of this requires access to the cloud - the intelligence runs locally on the drone itself. The drone does not require a remote operator or controller. It can't be jammed.

I don’t understand Abdul’s thinking here. Maybe, just maybe, the country enabling Gaza genocide and that made an unlawful and immoral attack on Iran might be selling a misleading righteous war narrative in Ukraine?

This is a small Vision-Language model running on an inexpensive phone chipset. Executes reasoning task while controlling UAV (drone) in sim world. The task is to find the human doctor near the medic truck. If desired, the drone could be instructed to hit the doctor or the truck. @SuperfocusAI

Did it take a foreigner to make the team more American? It’s hard to escape the feeling that lots of things in the country suffer from the same problems that plagued the USMNT.

Vision-Language models can run on *very* inexpensive chips now. Look for fully autonomous but inexpensive drone swarms in the near future!

Space, bitches!

This is the situation for on-device intelligence. Can be done inexpensively, but the chips are all from Taiwan or China. Think massive drone swarms...

It ain’t no fun if the Americans can’t make none. There is not a single low-cost American AI chip for running genAI models on-device. That’s one of the eye-opening things we’ve learned at Superfocus dot ai investigating on-device AI, and a point I made in my @EmbVisionSummit talk last week in the Valley. These are the chips in phones, tablets, smart glasses, wearables, drones, robots etc. The cheapest American on-device AI chips cost $100+ at volume, likely more. The lowest NVIDIA GPU chip and the top Qualcomm smartphone chips. A $100 chip means a ~$500 product. At least. Chinese and Taiwanese on-device AI chips that can do the exact same thing cost $25, maybe less, at volume. A $25 chip means a ~$125 product. There are lots of use cases where you can’t, or wouldn’t want to, use AI in the cloud. For those situations, why isn’t there a low-cost American on-device AI chip? Does it matter? The cheaper options from China and Taiwan are good (but not perfect) chips from solid companies. @edgeaivision @phil_lapsley @JeffBierBDTI @Rakshit_Ag @vghadiok @daveselinger @hsu_steve #EVS26

Wow - clash of the (debate) titans

You are what you do, not what you say. In the same briefing, right before this, Rubio was lying through his teeth about the state of the disastrous war with Iran that he and his Trumpista buddies are waging. His actions are the opposite of the spirit of the words he is using.

Arooj Aftab incredible Pakistani heir to Sade Was with director/video producer of this last night…sick. “Raat Ki Rani”

Two lesser-noted points in the Jan '25 DeepSeek paper caught our attn at Superfocus dot ai. 1) They produced AI models small enough to run directly on consumer electronics devices but with surprisingly strong reasoning ability. 2) They did it with relatively fewer data samples. We wanted to know if these smarter “on-device” size models are actually good enough to be useful? And how cheap of a chip could we run these models on? Why does this matter? There are lots of situations where you can’t, or wouldn’t want to, use AI in the cloud via the internet. Personal & family privacy (ex: your home). Child safety. Security. Data/IP privacy (factories). No or unreliable signal (commercial/defense drones). Low latency (robot/vehicle safety). No or outdated physical/cloud infra (industrial, warehouses). Plus, you don’t have to pay compute costs for on-device AI! We’re sussing out many of these use cases, but because of our shared interest in education, my co-Founder @hsu_steve & I immediately thought about kids & learning. Below is a video of my daughter demo’ing our prototype AI reading buddy. We’ve rigged our little AI puck up to a pair of smart glasses from our homies at @MentraGlass . (h/t @caydengineer ) This may not be a big deal for huge models in the cloud running on GPUs. What’s significant is that we’re doing it with small models and completely on-device. No internet connection. The AI sees what my daughter sees. It listens to her and follows along as she reads. And helps her when she needs it by speaking to her. All from small AI models that we finetuned, engineered and got running fast and accurately on a chip that costs less than $25. It’s not quite a reading tutor yet, but perhaps you can see how it could become one? Or how on-device AI like this could solve other problems? Would love to know what you think, good, bad, or other!

It's also not fair to the many staffers who don't come from wealth, which disproportionally is the case for black and Latino staffers. Most of the people who spend their entire careers in non-prof & govt and who create and enforce this weird moral standard against those that go into private sector have rich parents and family money. (Which there's nothing wrong with either!)

A legend. My parents Sunday morning soundtrack in our house growing up always had a brimful of Asha in it.

Glen rice was on fire that tourney. Watch the first few possessions of the sweet 16 game against UNC. Calm and cool, casually knocking down 3 after 3 while chewing that big wad of gum