PhD @ University of Toronto soon graduating. I work on controllable video generation & world models inspired by Bayesian Brain Theory. Open to AI/ML roles!

Toronto
Pinned Tweet
Dropping an exciting new demo of MosaicMem! 👀🔥 A friend brought up a great question: why not combine long-horizon navigation video generation, promptable world events, and scene concatenation? Fair point — so we gave it a shot. 🎬✨ For more technical details, check this thread 🧵👇 x.com/GnosisYu/status/203502… #WorldModel #GenerativeAI #VideoGeneration #InteractiveAI #Genie3 #EmbodiedAI #GameAI
World models have made impressive progress in video generation, yet they still struggle with a fundamental challenge: memory. In long rollouts, the camera trajectory gradually drifts from the user-specified motion and revisited scenes no longer align with earlier observations. These errors accumulate over time, causing the generated world to steadily lose coherence. 🚀Excited to share our solution MosaicMem 🌍🧠 — our new hybrid spatial memory for video world models. Project Page: mosaicmem.github.io/mosaicme… Paper: huggingface.co/papers/2603.1…
21
107
8,905
Wei Yu retweeted
Our work on an open-ended interactive video world model, designed to maintain long-horizon interaction consistency while supporting interaction with newly generated objects. Thanks for all the effort, @Madaoer_Yxd!
🌍 Introducing Oneira, an interactive video world model where a coding agent keeps an explicit world state. Objects that appear as you explore become interactive, and your changes persist over long horizons. Project: madaoer.github.io/projects/o… Paper: arxiv.org/abs/2610.01614
3
3
40
6,392
Introducing Gemini 4 Argon – our new frontier model. It’s built for complex workflows across coding, enterprise knowledge work, and cybersecurity defense – rolling out today to a set of trusted testers through our Fairwind Program.
1,988
4,653
42,998
8,893,979
Wei Yu retweeted
This is a big moment for us
Big news: Kimi-K3 by @Kimi_Moonshot is now #1 in the Frontend Code Arena with 1679 pts, surpassing Claude Fable 5. This is a 17-place jump from Kimi-k2.6 (#18 -> #1). In Frontend, Kimi-K3 ranked #1 in 6 of 7 domains: Brand & Marketing, Reference-Based Design, Data & Analytics, Consumer Product, Simulations, and Content Creation Tools, landing #2 only in Gaming behind Fable 5. The full model weights will be released by July 27. Congrats to the @Kimi_Moonshot team on this major milestone!
66
33
922
32,175
Wei Yu retweeted
Can video generation models do for vision what LLMs did for language? Introducing GenCeption from @GoogleDeepMind: one feed-forward video model for various vision tasks — SOTA, data-efficient, and emerging behaviors (ECCV 2026) 🌐 genception.github.io (1/8)
17
99
468
118,039
Wei Yu retweeted
Introducing MIRA, a multiplayer world model, a dream of Rocket League (links in 🧵)
14
30
281
30,664
Wei Yu retweeted
Fable + Reverie Infinitely persistent, real-time interactive.
Fable 5. No external assets. Three.js. dc5fzrbo8ssfx.cloudfront.net…
11
18
249
69,279
Wei Yu retweeted
I’m very excited to share that my very first Ph.D project, Policy-based Foveated Imaging and Perception, will be presented at #SIGGRAPH2026! Intelligent sensing transcends passive capture. Our framework allows ultra-high-resolution sensors to intelligently allocate acquisition bandwidth, perceiving the blooming present and awakening the vivid past. We further demo our framework on a physical 200MP sensor prototype real-time with only laptop CPU! I’m extremely grateful for the advice and support from my Ph.D advisor @GordonWetzstein, and for the wonderful collaboration with @jan_on_x and @boyang_deng! Please check out our paper and our website 📷+👁️: howardxiao.ca/foveated/.
The era of ultra-high-resolution imaging has arrived. Modern image sensors exceeding 200 MP resolution are common in smartphones, with over 400 MP sensors under development. However, the large number of pixels poses significant challenges for acquisition and processing, especially on edge devices. Which pixels should be acquired, and when, for bandwidth-efficient imaging and perception? We introduce Policy-based Foveated Imaging and Perception, an on-device, real-time, predictive, and task-aware framework that dynamically allocates sensor resolution to prioritize important regions under specific perception objectives. This paper will be presented at #SIGGRAPH2026! [1/6]
6
3
32
4,040
Exciting to share our work "Good Token Hunting" 🔍 (Yes, the name is inspired by the classic movie "Good Will Hunting" 🎬!), which focuses on accelerating visual geometry transformers 🚀 by limiting the number of keys/values each query can attend in global attention layers. [1/6]
1
8
29
17,664
Wei Yu retweeted
The latent-vs-pixel debate misses the point. GPT Image 2 shows what users notice: pixel-level fidelity. Latent models show what scales: compact semantic structure. We connect them by replacing VAE/RAE decoders with a Pixel Diffusion Decoder. Code and Model available: research.nvidia.com/labs/sil… 🧵(1/N)
16
62
405
669,685
Wei Yu retweeted
🚀 🚀 🚀 Excited to share our new paper: Remember to be Curious: Episodic Context and Persistent Worlds for 3D Exploration What does it take for an agent to stay curious in a 3D world? The answer is memory. 🌐 Project: recuriosity.github.io/ 📄 Paper: arxiv.org/abs/2605.22814 💻 Code: github.com/recuriosity/recur…
2
41
226
72,736
Wei Yu retweeted
Aleph 2.0 is here. Now you can edit a single frame in your video, preview the change and then Aleph 2.0 carries that edit across the rest of your video. Try it now in the new Edit Studio on web at the link below.
137
263
1,933
4,957,350
Wei Yu retweeted
Meet our new friend, Starchild-1 ❤️ Starchild-1 is the first ever real-time multimodal world model. A world model understands and simulates the world. Starchild-1 has learned to generate not just the visuals of the world, but the sounds of it too!
62
70
758
122,026
New blackboard lecture w @ericjang11 He walks through how to build AlphaGo from scratch, but with modern AI tools. Sometimes you understand the future better by stepping backward. AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning from experience, and self-play. You have to go back to 2017 to get insight into how the more general AIs of the future might learn. Once he explained how AlphaGo works, it gave us the context to have a discussion about how RL works in LLMs and how it could work better – naive policy gradient RL has to figure out which of the 100k+ tokens in your trajectory actually got you the right answer, while AlphaGo’s MCTS suggests a strictly better action every single move, giving you a training target that sidesteps the credit assignment problem. The way humans learn is surely closer to the second. Eric also kickstarted an Autoresearch loop on his project. And it was very interesting to discuss which parts of AI research LLMs can already automate pretty well (implementing and running experiments, optimizing hyperparameters) and which they still struggle with (choosing the right question to investigate next, escaping research dead ends). Informative to all the recent discussion about when we should expect an intelligence explosion, and what it would look like from the inside. Timestamps: 0:00:00 – Basics of Go 0:08:06 – Monte Carlo Tree Search 0:31:53 – What the neural network does 1:00:22 – Self-play 1:25:27 – Alternative RL approaches 1:45:36 – Why doesn’t MCTS work for LLMs 2:00:58 – Off-policy training 2:11:51 – RL is even more information inefficient than you thought 2:22:05 – Automated AI researchers
62
273
2,673
707,980
Wei Yu retweeted
🤩Excited to share SANA-WM: a 2.6B open-source world model for minute-scale 720p video generation. Given one image + text + a 6-DoF camera trajectory, it synthesizes action-controllable 60s worlds on a single GPU. Project: nvlabs.github.io/Sana/WM/ Paper: huggingface.co/papers/2605.1…
27
139
1,034
124,624
Wei Yu retweeted
Meet LA-Pose. Our latest model taking Wayve another step towards generalization at scale. LA-Pose employs large-scale self-supervised learning, building strong motion representations for 3D perception from 10.2 million unlabeled driving video snippets, unlike today's strongest approaches that often depend on expensive, carefully curated 3D supervision. With only a lightweight pose head and limited labelled data, LA-Pose achieves: 📷 State-of-the-art camera pose estimation 🌎 Strong zero-shot generalization across diverse driving scenarios 🏷️ Orders of magnitude less labelled data than fully supervised 3D approaches Our full blog post: wayve.ai/thinking/la-pose/ Explore the full paper here: la-pose.github.io/
1
36
144
37,049
Wei Yu retweeted
We're taking our first step towards democratizing World Models, so that everyone can build on this incredible technology. We have more to share, but enjoy a glimpse of what's to come, today. Try it here: reactor.inc
41
58
779
1,408,027
Wei Yu retweeted
Excited to share that this work has been accepted to #ICML and the code is now publicly available at github.com/nv-tlabs/tttla!
Continual learning and online adaptation are often framed as the next frontier of AI. 🚀 Modern architectures use Test-Time Training (TTT) to memorize key-value pairs on the fly via gradient descent, or so we thought. To test this memorization hypothesis, we replaced gradient descent with gradient ASCENT. It should destroy memorization. Instead... performance was preserved, or even slightly improved. 😱 It turns out, TTT with KV Binding is secretly linear attention! Site: research.nvidia.com/labs/sil…
1
2
43
7,322
Wei Yu retweeted
What if your robot could understand any object you describe, just from a phone camera? RADIO-ViPE builds a 3D map from raw monocular video that you can query with natural language. (1/4)
11
48
420
61,851
Wei Yu retweeted
New #NVIDIA Paper We introduce Motive, a motion-centric, gradient-based data attribution method that traces which training videos help or hurt video generation. By isolating temporal dynamics from static appearance, Motive identifies which training videos shape motion in video generation. 🔗 research.nvidia.com/labs/sil… 1/10
12
125
638
144,806
Wei Yu retweeted
Introducing Moonlake's 3D Agent. Our agent acts like a technical artist that can build and reconstruct articulated assets and large-scale editable scenes with hundreds of objects from a single image and can improve its generations continuously. Learn more in the thread below.
42
178
1,380
1,168,750