I profess CS-ly at @UWaterloo. Previously, I monkeyed code for @Twitter, slides for @Cloudera, and scienced for @yupp_ai.
Nearby data lake
- Tweets 4,544
- Following 872
- Followers 15,331
- Likes 0
Summer passed in a blur... ๐๏ธ๐ฆฉ๐น and I forgot to congratulate @HosnaOH for finishing her Master's! Her work with @amirhkarimi_ "A Unified Perturbation Framework for Analyzing Leaderboard Stability and Manipulation" arxiv.org/abs/2605.15761 just got a +1 from NeurIPS, so yay! ๐
1
2
51
3,535
Achievement unlocked! ๐ Does this come with stat bonuses or help with saving throws? uwaterloo.ca/computer-sciencโฆ
36
9
145
6,518
My colleague @yuntiandeng about โClaudishโ ๐ค and how AI coding agents are changing the way we communicate! uwaterloo.ca/news/media/q-anโฆ
1
1
10
2,645
Jimmy Lin retweeted
1/ ๐งต
BrowseComp-Plus (ACL 2026) set the standard for disentangled agentic search eval...
๐ ๐๐ซ๐จ๐ฐ๐ฌ๐๐๐จ๐ฆ๐ฉ-๐๐ฅ๐ฎ๐ฌ_๐๐ makes it even better!
๐ Same BCP questions grounded on ๐๐ฅ๐ข๐ฆ๐๐๐ข๐ฑ-๐๐๐๐ with ๐๐๐๐ ๐๐จ๐๐ฌ.
๐ป github.com/castorini/cmass
๐ Introducing BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent.
It is a new Deep-Research evaluation benchmark built on top of BrowseComp. It features
- ๐ a fixed, carefully curated corpus of web documents
- โ
human-verified positive documents
- โ๏ธ web-mined challenging hard negatives.
With BrowseComp-Plus, you can thoroughly evaluate and compare the performance of different components in a deep-research system. e.g. GPT-5 + Qwen3-Embedding.
Code, dataset, and leaderboard links are provided at the end of this thread.
1
9
34
4,469
Jimmy Lin retweeted
In the past year, we've seen rapid progress in search agents! However, how well do these search agents work outside English?
Introducingโต๏ธMAST @๐ฅFIRE 2026, a new shared task focused on multilingual agentic search.
1
8
31
6,175
Agentic reproducibility is the unlock, I claim. To what? ๐โโ๏ธ Find out by reading my piece "Reproducibility in the Era of AI Agents" ๐ค in the June 2026 issue of SIGIR Forum. sigir.org/forum/issues/june-โฆ
1
6
30
3,819
Jimmy Lin retweeted
And with that, all TREC RAG 2026 submissions are in ๐
We had an incredible turnout this year, with 30 teams submitting more than 80 RAG runs and 110 retrieval runs!
A huge thank you to everyone who participated! We canโt wait to start evaluating your submissions ๐
5
11
4,018
I always tell this joke in my classes about IR, and it goes something like: well, short of inventing telepathic retrieval, we remain stuck with some form of query formulation, be it a few keywords (before) or a prompt (now)... little did I know... ๐
Two weeks ago, I resigned from OpenAI to join Conduit as a founding researcher, where we're training models to non-invasively read the human mind.
I've written some thoughts about what telepathy could look like by 2035 and how to get there:
2
9
1,930
๐ช Belatedly plugging a paper that attempts to bridge information nuggets and side-by-side comparisons with @arena data for complex LLM responses (tl;dr - pointwise vs. pairwise evaluations) dl.acm.org/doi/10.1145/38057โฆ Paper led by @Sahel_Sharify @Ushivani3 @beirmug!
2
3
11
1,420
This work serves as an inspiration for the evaluation methodology to be deployed at @TREC_RAG 2026 trec-rag.github.io/ - submissions due in a few days!
433
The saga continues ๐ All you need... grep? โ
BM25? โ
psssh... ๐ข๐ด Boolean! โ๏ธ
This was fun. Humans need powerful search tools, but even exact-match Boolean queries can work for agentic search. Our Boolean ranking uses only match proximity and density. No learning, vectors, or even term statistics required. arxiv.org/pdf/2607.11362
1
4
51
8,481
It's now officially ๐จโ๐ DR @xueguang_ma - who yesterday successfully defended his Ph.D. ๐ Congrats!
7
9
64
15,853
Jimmy Lin retweeted
๐กLLM agents should do more than rewrite promptsโthey should actively search for and organize useful multimodal context to support the generator.
Introducing SearchGen, which co-trains an LLM agent and a generator to expand the knowledge boundary of image generation.
Try our Live Demo: tinyurl.com/searchgendemo
Search a frozen 1M Web corpus; no live API needed.
Read our project blog: haozheh3.github.io/SearchGenโฆ
(1/N)
1
6
17
2,092
Jimmy Lin retweeted
Our paper got an oral at the CTB workshop @ ICML๐ฐ๐ท
How stable are LLM leaderboards? ๐ค
Less than you'd hope. We built a unified framework for auditing how small changes to pairwise votes ripple through leaderboard rankings
๐ arxiv.org/pdf/2605.15761
๐ hosnahoseini.github.io/leadeโฆ
2
5
20
7,103
Jimmy Lin retweeted
๐ Thrilled that our ACL 2026 paper โTo Lie or Not to Lie?โ was selected as an ACL 2026 SAC Highlight! ๐
The work asks a simple but urgent question: are LLMs equally safe around the world? ๐งต๐
@aclmeeting #ACL2026NLP #NLProc
6
16
65
4,102
Jimmy Lin retweeted
[1/3] Our agentic search work is at #ACL2026! ๐ I can't make it to San Diego in person, but luckily @zijian42chen is presenting the poster โ go say hi!
The one-line takeaway: ๐น๐๐๐๐๐ ๐๐๐๐๐๐ ๐๐๐ ๐๐๐๐๐๐. ๐งต
1
4
16
962
Jimmy Lin retweeted
What if a website helper were a program tree, not a single chatbot?
I built one for my course from 30 ProgramAsWeights programs. Each question falls through its own path of routing, search, answering, validation.
Now anyone can ask a coding agent to build one with one prompt.
1
9
18
5,271