Excited to present "Training with Honeypots" 🍯 at #ICML2026 tomorrow (Tuesday 10:30AM in Hall A, Poster #1601), an approach for adversarial defense that makes successful jailbreaks less useful to attackers! #AISafety 👉 icml.cc/virtual/2026/poster/…

Reviewer: “I still have questions about who would use this.” Me: 👉🏻

I spoke with Neue Zürcher Zeitung (NZZ) about open-weight AI safety and some of the risks that come with increasingly capable open models. Right now, safeguards are still far too easy to remove. Find the article here 👇 nzz.ch/technologie/xi-besuch…

Thrilled to see our work on evaluation awareness informed alignment evaluation at Anthropic! The Claude Opus 5.5 system card credits our recommendations in its revised grader-awareness methodology (Sec 6.6.2)! The new evaluation mirrors our decomposition framework. It scores how strongly the environment discloses a grader, separately from whether the model recognizes it (recognition), and whether it acts on that recognition (propensity). Maybe they are also thinking about how to make model an honest participant in evaluation, so that recognition doesn't change behavior 🧐?

Please consider signing: make-safety-open.github.io/

I have no idea what probability I'd put on this, but the 50% reminds me of Walter Wagner on CERN destroying the world: either it happens or it doesn't, so one in two. Obviously very different reasoning. The fact that leading AI safety researchers are this concerned is scary.

🚨The Last Translation Benchmark is officially out on ArXiv! arxiv.org/pdf/2609.04173 I took part in a massive crowdsourcing effort to collect hard-to-translate examples that break SOTA translation models. If you like red-teaming models, I highly recommend contributing!

AI is moving more and more into physical systems, and there are huge open problems around making these systems safe. I’m excited to be a member of PASC and contribute to making physical AI safer.

We can finally talk about it: We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company. We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.

Filtering SFT data often has very little effect. Would be great to see a lab run this at pretraining scale (e.g like deepignorance.ai) Until we know how to make open models safe, releasing weights beyond a certain capability looks hard to justify.

Judging by my review batch, NeurIPS reviewers largely ignore the official “use sparingly” guidance for scores of 3 and 4. Too often, even a single remaining minor weaknesses seem to limit a submission to a 4. Why is it the case? Should the guidance be modified?

Hot open-source summer season has gotten to Switzerland 🇨🇭⛰️ Our team at Swiss AI Initiative is happy to release Apertus 1.5 - a multimodal and reasoning update to our first v1 version! The new version builds a foundation for future regular open model development at scale🧵

Our Jinesis Lab at the University of Toronto is bringing three exciting papers to #COLM2026! 🎉 These projects push the frontiers of #AISafety, #AgentSecurity, and #AIForScience.🤖 Huge congratulations to all collaborators and co-authors! Papers: 🛡️ODILE: Orthogonal Disruption of Injected Tool-Call Embeddings for Agentic Prompt Injection Defense 🔭Stargazer: A Scalable Model-fitting Benchmark Environment for AI Agents under Astrophysical Constraints ⚠️One Word at a Time: Incremental Completion Decomposition Breaks LLM Safety @UofT,@VectorInst,@EuroSafeAI,@ETH_en,@MPI_IS,@UMich

🌟 What an incredible experience at #ICML2026! 🌟 Our team at Jinesis Lab had a fantastic time presenting research across several exciting and important areas: 🤝 Multi-agent LLMs 🧠 Causal reasoning 🛡️ AI safety We were proud to share this work alongside our collaborators and to engage with so many researchers from across the machine learning community. 💡 Just as valuable as presenting our own work was learning from the research others brought to ICML. From thought-provoking talks and innovative poster presentations to insightful discussions and spontaneous hallway conversations, the conference was full of ideas that challenged our thinking and opened new directions for future work. ✨ We are grateful to everyone who attended our presentations, visited our posters, shared feedback, and took the time to discuss their own research with us. 🙌 A huge thank-you to our lab members, collaborators, and the broader ICML community for making the experience so memorable. We are leaving inspired, energized, and excited to build on the ideas and connections formed throughout the conference. 🌍🔬🚀 #ICML #MachineLearning #ArtificialIntelligence #MultiAgentSystems #LLM #CausalReasoning #AISafety #ResponsibleAI #AIResearch #ResponsibleAI #AIResearch

🚨Check out our #ICML2026 Spotlight Paper, "AI Poses Risks to Democratic and Social System"! 📍Poster Session: Wednesday, July 8 • 10:30 AM – 12:15 PM (KST) • Hall A, Poster #3015 What if every AI model were perfectly safe, but society still wasn't? TL;DR: We argue that safe models do not necessarily lead to safe societies. Our paper introduces sociopolitical risk, a framework for understanding risks that emerge from AI deployment at a societal scale, and argues that model-level safety should be complemented with system-level evaluations and governance. Presenting authors @ ICML: @ZhijingJin @davidguzman1120 @PepijnCobben @x_angelohuang @ChanglingXavier @SimkoSamuel @TerryJCZhang 1/2

I'm at #ICML2026 this week! Tomorrow I'm presenting our #spotlight paper , "Safe Models Do Not Guarantee Safe Societies." Even a perfectly aligned AI can quietly wear down the institutions a democracy runs on.

Happening right now! Come say hi 👋 Hall A, #1601

When a jailbreak succeeds, not all failures are equal. Some answers are wrong, vague, or not useful, while others are actionable enough to help someone cause real-world harm. We find that unprotected harmful model outputs often skew toward the more severe and actionable side. At the same time, automated judges often struggle to tell what is actually useful for harm. In a blatant case, a nonsensical hacking answer like “tap the control key three times” can still be judged harmful by systems such as StrongREJECT.