On the job market this year! Open to faculty as well as RS/MTS positions and primarily interested in the science of language models. Please reach out if you are hiring and you think I would be a good fit.

The UT comp ling faculty (@jessyjli, @kanishkamisra, @kmahowald & myself) are looking for PhD students to join us! sites.utexas.edu/compling/ 🤘 We are an interdisciplinary group with interests across linguistics, cogsci and NLP. Apply by 12/1: liberalarts.utexas.edu/lingu…

my iclr 27 papers to review are some of the strongest evidence against rsi I've seen to date. just the most dreadful claudeslop.

It was great to speak with you at COLM! 🦙 Looking forward to the full episode release as well (Besides this paper, I also talk a bit about my experience in the PhD and what tools I use for research!)

If you’re applying to CMU’s School of Computer Science @SCSatCMU and are interested in the Language Technologies Institute’s PhD or MLT programs @LTIatCMU, come to our info-session on November 11! We’ll have faculty there to answer any questions you may have about the department and application process

Recently, @_emliu et al.'s paper on "The Implicit Curriculum Hypothesis" won an outstanding paper award at COLM: arxiv.org/abs/2604.08510 This is of course due to the ingenuity and hard work of the authors, but it's also thanks to an @NSF ACCESS grant of compute resources, for which we are very grateful. Just wanted to explicitly point this out as a positive story of public support for computational resources for open research!

🧵Still cleaning pretraining corpora with a heuristic HTML scraper plus dozens of hand-written filters? We replaced the whole stack with ReScraper, one unified 0.6B model that beats the strongest curation baseline at every scale 📈 🌐 Blog: cxcscmu.github.io/ReScraper

Introducing 𝐒𝐩𝐞𝐜𝐮𝐥𝐚𝐭𝐢𝐯𝐞 𝐃𝐞𝐜𝐨𝐝𝐢𝐧𝐠: 𝐇𝐨𝐰 𝐈𝐭 𝐄𝐯𝐨𝐥𝐯𝐞𝐝, 𝐖𝐡𝐞𝐧 𝐈𝐭 𝐒𝐭𝐚𝐲𝐬 𝐋𝐨𝐬𝐬𝐥𝐞𝐬𝐬, 𝐚𝐧𝐝 𝐖𝐡𝐚𝐭'𝐬 𝐍𝐞𝐱𝐭. An interactive tutorial @Madisonkanna and I built for the NeurIPS Education Track. Every LLM you use generates one token at a time. Autoregressive decoding is the bottleneck for inference. Speculative decoding accelerates this, and today it runs under nearly every hosted LLM. It is a cornerstone topic to learn in the LLM stack. Blog: neurips2026-speculative-deco…

Tired of AI slop and spaghetti code? We study how to measure and reduce slop in long-horizon coding agents in this work, which will be presented at the #COLM Workshop on Agent Behavior (October 9, 10 - 11:30 AM at Continental Ballroom) 🧵👇

When LLM demand exceeds capacity, which requests should go first? Model providers already offer a few priority tiers. But even within a tier, requests compete for compute, and users value latency differently. Your coding agent might be blocked due to a request that belongs to an unattended background job. Why not let the market decide? In our new paper, Inference Auctions, users bid for request priority, and our auction allocates that priority to those who value it most. Unfortunately, naively selling to the highest bidders destroys prefix-cache locality, increasing SGLang's average latency by up to 12x in our experiments. Our fix is to run an auction over depth-first search traversals of the request prefix tree, ordering subtrees by average bid. This maximizes social welfare while retaining cache optimality, with VCG payments incentivizing truthful bidding, and the resulting request order preserves key performance metrics (cache hits and mean TTFT) of the underlying scheduler. w/ @keegan_w_harris @sidpra5ad @nhaghtal and Michael Jordan Paper: arxiv.org/abs/2609.40070 Project page: inferenceauctions.com

🚢Marin 535B-A23B training has crossed the halfway mark! Some observations and learnings so far:

I sadly won't be participating in #COLM2026 (instead will be teaching the CMU AI Agents course), but my wonderful students will be there. Say hi to them, especially the ones on the job market this year.

Honoured to receive an outstanding paper award for our work! Couldn’t have done it without all my coauthors @KaiserWhoLearns @millicent_li @wordscompute @lltjuatja @JentseHuang and my advisor @gneubig 🫡

⚠️ User simulators are increasingly used to evaluate AI agents in multi-turn interaction. But we find that looking like real users ≠ reproducing real evaluation outcomes. 📢 We introduce Calibrated User Embeddings (CUE) to bridge this gap across LLM simulators. 🧵 1/8

AI scientist agents are great at optimizing and finding solutions through endless trial and error. But is that really what science is about? As Terrance Tao eloquently put, the role of math and science should be more than that – they are "lighthouses" that guide and inspire exploration through understanding and insights. Can AI agents discover novel insights through experimentation? To study these gaps, we worked with domain experts to introduce EurekaBench, a benchmark spanning 6 science domains that evaluates an agent’s ability to discover genuinely new scientific insights. (1/n)

Thrilled to announce HalluWorld 😵‍💫 🌎 , a controlled benchmark + methodology to verifiably measure how LLM agents hallucinates while in an environment. Now accepted at #NeurIPS2026 Evals & Datasets!! We start with the thought that Reality is vast, dynamic, fractally unfolding. W/o a stable reference and controllable complexity, verifiably measuring and inducing situations that lead to Hallucination a.k.a deviation from observed Reality gets tough. This is even more so if we want to go from single turn Q/A setting --> Agents hallucinating while solving a task once placed in an environment. HalluWorld puts models inside environments where we know exactly what is true at every step: Gridworlds 🕹️ , Chess ♟️, and Terminal 👩‍💻 tasks. What's more? Having such environments makes things neatly modifiable to am up support different levers of complexity, think canyons, signboards and long tunnels (Gridworlds) ; arbitrarily complex game situations or mods (Chess) inter alia --> This gives us ways to craft situations that amp up cognitive pressure on the agent This is inspired and in vein of the bAbi methodology of creating questions with knowable answers from a world you can create by @jaseweston et al from 2015. Come check out more at halluworld.ai , or read our arXiv at arxiv.org/abs/2605.19341 ! It was enriching and fun working on this with @_emliu , @karanps, @stevenyfeng, @_michael_yu_, @ZhuofuTao and @shocheen from @LTIatCMU , @PatronusAI , @stanfordnlp and @OhioStateCSE over the past few months!