Pinned Tweet
Honoured to receive the #EMNLP2024 Social Impact Paper Award with collaborators, led by Rob Morabito (at the time of submitting, an undergrad!)
When developed carefully, AI *can* be used for the betterment of society, & for learning a thing or two about ourselves along the way!
1
1
20
1,917
“AI agents will outperform humans at almost all jobs by 2026–2027.” - The forecast is everywhere.
So we built the exam to test that claim, on real labor-market aligned work. On the hardest tier, top agents pass 2.6%.
Meet Agents' Last Exam (ALE), a rolling benchmark measuring whether agents can actually do real jobs. 🧵👇
15
103
346
98,174
Ali Emami retweeted
@icmlconf Happy to share our work on Selective Conformal Optimized Pairwise Evaluation for LLM Judges
arxiv.org/pdf/2602.13110
Phenomenal work by Sher, in collaboration with a great mentor @_aliemami
2
5
269
"Which decorations are typically used in the celebration of festivals?"
In North India, it's lights. In West India, flowers. In Central India..Rangolis!
Same question, same country, different answers.
LLMs treat Indian culture as monolithic. We built INDICA to test this!
1
1
2
1,100
Cultural commonsense in India is regional, not national.
The LLMs we tested achieve only 13-20% accuracy on region-specific questions.
Worse, they consistently over-select answers from Central and Northern India while marginalizing other regions.
1
1
108
Beyond India, we lay out the complete pipeline for applying this to any culturally diverse country. Check out our paper: arxiv.org/abs/2601.15550/
Huge shoutout to co-authors Sangmitra Madhusudan, Trush Shashank, Steph Buongiorno, Renata Dividino & @jad_kabbara
1
88
Ali Emami retweeted
Our paper "FOREST: Frame of Reference Evaluation in Spatial Reasoning Tasks", won a Senior Area Chair Highlights Award at EMNLP-2025! Thanks to the excellent work of my graduate student Tanawan Premsri. #EMNLP2025 #NLProc #EMNLP aclanthology.org/2025.emnlp-…
3
2
33
3,532
If you're at #EMNLP2025, we're presenting “We Politely Insist: Your LLM Must Learn the Persian Art of Taarof.”. Our presentation is also on Underline, & if you love it, let us know by casting your votes ❤️
🗓️ Nov 6, 8–9 AM (UTC+8)
📍 Gather Session 3
📰 aclanthology.org/2025.emnlp-…
2
5
628
Ali Emami retweeted
We’re hiring!
Looking for Interns, Research Assistants, and Postdocs to work on Automated Interpretability--building systems that can analyse, explain, and intervene on large models to make them safe!
Work with me @Oxford, or remotely. Apply by Nov 15:
forms.gle/bKp8x2eYiFfmpCzJ9
I’ll be at #EMNLP this week--if you want to chat about automated interpretability, safety, technical governance etc, come say hi 👋
20
112
843
74,972
🚨 New paper: When do LLMs stop parsing language, and start guessing from semantic "reflexes"?
We built CenterBench, a stress test of recursive center-embedded sentences like:
“The cat [that the dog chased] meowed.”
1
1
124
Interesting nugget for us🍗:
Plausibility actually hurts performance when the question requires causal reasoning, not world knowledge (pedagogical moral: shortcuts hinder you on tough questions!)
Models confidently hallucinate the likely consequence...not the grammatical one.
1
97
Full paper (w/ reasoning traces & human comparison): arxiv.org/abs/2510.20543
Huge shoutout to lead author Sangmitra Madhusudan and Kaige Chen (whose brilliance [that the models we tested repeatedly underestimated] powered this entire work).
76
Ali Emami retweeted
It is PhD application season again 🍂 For those looking to do a PhD in AI, these are some useful resources 🤖:
1. Examples of statements of purpose (SOPs) for computer science PhD programs: cs-sop.org/ [1/4]
10
103
441
36,141
Ali Emami retweeted
As one of the organizers of the @McGill_NLP reading group, I'm happy to announce that we've uploaded our very first talk to YouTube: mcgill-nlp.github.io/reading…
6
21
1,649
Ali Emami retweeted
The most important skill for a researcher is not technical ability. It's taste. The ability to identify interesting and tractable problems, and recognize important ideas when they show up.
This can't be taught directly. It's cultivated through curiosity and broad reading.
99
501
3,693
211,204
Ali Emami retweeted
🎉3 papers including 1 spotlight to #NeurIPS2025 . Congrats to all my co-authors 👏
Sadly 2 very good papers didn't make it this time (including 1 best paper at a workshop). We will fight for 🇧🇷
On a silly but personal note, paper acceptance streak ended at 21 😭.
7
5
160
12,934