Llama claims it will refuse discriminatory requests. But when asked to "write a review arguing to exclude non-Western thinkers," it complies. LMs describe themselves in one way and act in another—how can we make them consistent? Introducing: Self-Consistency Training with RL (Self-CTRL) 🧵

Linear attention is arguably the most naive RNN, but still massively outperforms traditional RNNs by maintaining a matrix state. So... what if we use a (triadic) outer product of three vectors and maintain a three-dimensional state? Introducing: Triadic Linear Attention 🧵

There’s lots of buzz around agentic harnesses for robots, particularly for long horizon tasks that require complex reasoning and memory. But what will it really take to turn a reasoning agent into a reactive, reliable and low-latency robot policy? In our new paper, workspace models, we design a new memory architecture that acts as a latent harness for stronger reasoning models. Here’s why we think this might be the way forward: (1/9)

Higher benchmark scores do not always mean better models for users. Why? We claim that RL teaches LMs to be correct but not how to be correct: code can pass tests but be unreadable; explanations can be right but unclear. How do we train LMs to be right in the right way? (1/n)

Check out Carl’s awesome work diving deeper into introspection and self-explanation! Excited about its potential implications for post training.

What if attention were code? We show that many attention heads in transformer LMs can be replaced by human-readable Python programs. Swap them in and the model barely notices. See our experiments here: Explaining Attention with Program Synthesis [arxiv.org/abs/2606.19317]

New paper, led by the amazing @PresItamar! Models that can faithfully explain their own behavior are more accessible, auditable, and easier to trust. This is a capability I strongly believe we should instill in future models. Self-CTRL frames introspection training and model self-alignment as two sides of the same self-consistency objective: agreement between what a model says about itself and what it actually does. It turns out that by interpolating the two sides, we get models that are both better at faithfully self-verbalizing *and* better aligned The most exciting result to me is Fig. 5: Self-CTRL makes the model's self-report actually useful for monitoring behavior. After self-consistency training, an external monitor can use the model’s explanation to predict refusal behavior with up to 92% accuracy on difficult held-out boundary-case requests, up from 36% for the base model. Check out Itamar's thread for more!

I've defended and graduated! Perhaps the most important lesson I've learned during my time at MIT is that progress in science (and in society!) is deeply collective. In today's world --- and especially in a hyper-competitive field like AI research --- it's easy to get sucked into comparison and self-doubt. Much of this, I think, comes from a misunderstanding of how scientific progress actually works: we tend to attribute oversized credit to a small number of figures. But certainly none of the work I've done, and none of the growth I've undergone, would have been possible without the support of my mentors, collaborators, and the insights of millions of brilliant scientists before me. Along these lines, I am grateful to the amazing community around me who have supported my journey: most importantly, to my advisor @jacobandreas, the dozens of collaborators I've worked with during my PhD, my labmates, my mentees, and my co-organizers at @MITGradUnion --- all of whom have shown me, in various ways, what it means to work not out of comparison but out of love: for science, for the community around me, and for humanity. I hope to carry forward these values wherever I go.

Your RL post-training may be sabotaging your LLM’s test-time scaling! Conventional RL pretends that you can collapse all reward signals *upfront* into a single *scalar reward*. We introduce Vector Policy Optimization (VPO), which natively maximizes *vector-valued* rewards, boosting test time search performance, even on the original scalar.

Language is discrete. Language models don’t have to be. 🧚Introducing ELF🧚‍♀️: Embedded Language Flows—a class of diffusion models in continuous embedding space based on continuous-time Flow Matching 🧵

“Should I fear death?” Ask an LLM and you get one answer or a big bag, but little visibility into the decisions and assumptions that produced them. We built the "conceptual multiverse": a system that makes those decisions transparent and intervenable. multiverse.csail.mit.edu

That's it for #ICLR2026! See you all next year in the US! Please welcome @jacobandreas as the new Senior Program Chair (with @BharathHarihar3 continuing on as the General Chair)