New positions at FOCAL! cs.cmu.edu/~focal/index.html

Designers of multi-agent systems need to know which infrastructure choices can support mutually beneficial outcomes between AI agents. Today's blogpost by @EmanuelTewolde introduces CoopEval, a framework for comparing cooperation mechanisms in practice, and outlines a broader research agenda for understanding when and why they work.

Now on arXiv: our paper on whether LLM agents are more likely to cooperate when given various signals that the partner agent is similar -- led by @Akash190104 and @EmanuelTewolde. (Honorable Mention at the 2026 ICML AI4GOOD Workshop!) arxiv.org/abs/2608.12125

Two honorable mentions for papers at the ICML AI4GOOD workshop! Paper led by @EmanuelTewolde and Xiao Zhang: ‘CoopEval’ arxiv.org/abs/2604.15267 Paper led by @Akash190104 and @EmanuelTewolde: ‘Do LLMs Take Care of Their Own?’ openreview.net/forum?id=neTp…

The AI4Good workshop is today (Korea) @ ICML! @EmanuelTewolde is presenting the CoopEval paper, but also follow-up work with CAIRF fellow Akash Kundu about whether LLM agents cooperate with others that they perceive as similar. openreview.net/pdf?id=neTpZb… trustworthy-ai-for-good.gith…

Today (Korea time) at ICML in the 5pm session, Vijay Keswani is presenting our position paper "We Need Practical AI Alignment Methods that Mirror Human Reasoning!" presentation: icml.cc/virtual/2026/poster/… paper: openreview.net/pdf/a895d4cfe…

Standing by the CoopEval poster right now at #ICML2026! 📊👋 If you are interested in cooperation and multi-agent safety, game-theoretic mechanisms, or modern LLM agents defaulting to defection, come by board #4008 and let's chat! 🇰🇷🫱🏿‍🫲🏾

@EmanuelTewolde is presenting our CoopEval work at ICML on Wednesday 10:30am session (or catch him at the alignment workshop today)! presentation: icml.cc/virtual/2026/poster/… arXiv: arxiv.org/abs/2604.15267

Congrats to our recent undergrad+MS grad Jerick Shi on receiving the best paper award at ICML NExT-Game for our paper "When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games" w/ Terry J. C. Zhang, Bernhard Schölkopf, @ZhijingJin! openreview.net/forum?id=v8nY…

our recursive joint simulation paper (w/ Vojta and @C_Oesterheld) accepted to Synthese! TLDR: When players in a game run a simulation of themselves (incl. further subsimulations), that's equivalent to an infinitely repeated game and so allows cooperation. arxiv.org/abs/2402.08128

One of my open math problems apparently got resolved by ChatGPT 5.5 Pro (Ryan O'Donnell prompted it better than I did!), though the proof was so hard for me to read that it seemed easier to just prove it myself. More thoughts on implications for math here: aifails.substack.com/p/even-…

@FOCAL_lab member @EmanuelTewolde is presenting his CoopEval work at EPFL on Monday (14:30) and it will be on zoom! (paper link in comment) memento.epfl.ch/event/ai-cen…

(1/4) Can remembering more of the past make AI agents less cooperative? In our new paper, we study LLM agents in repeated social dilemmas. The key variable is not how many rounds they play, but how much prior interaction history they can access when making each decision.

(2/4) Surprisingly, longer recall often degrades cooperation. Across 7 LLMs and 4 repeated social dilemma games, agents with longer histories often shift away from forward-looking cooperation and toward retrospective grievance-tracking.

(3/4) The mechanism is not just “too much context.” It is what the agents remember: replacing histories with synthetic cooperative records restores cooperation, and ablating explicit CoT reasoning often reduces the collapse. We call this the memory curse.

(4/4) Paper link: arxiv.org/abs/2605.08060 Big thanks to my collaborators! @Jack_Litq, @zzzoooeee321, Xin Luo, Haoxuan Zeng, @EmanuelTewolde, Tai Sing Lee, @tonghanwang, @ckingsford, @conitzer

In this paper led by Jerick Shi @Jerick1380, we propose a taxonomy of LLM deception. arxiv.org/abs/2604.04788

(3/3) Also, I don't understand how some people think AGI is just around the corner but the risks are easily manageable! Of course their positions may not be captured accurately here.

(2/3) I'm sure we all have thoughts on our descriptions -- I certainly worry about many other AI risks current and future in addition to scaled misinformation, and I actually think the world is too focused on LLMs-as-chatbots -- but still impressive.

(1/3) Wow, I (and many friends) made it onto this "Mapping AI" map! Does anyone know the methodology? Who is behind it? mapping-ai.org/map