Second Look Research is a project from @uchicagoxlab. We produce open source replications of load-bearing results in AI safety.
Chicago IL
- Tweets 19
- Following 67
- Followers 79
- Likes 99
New blog post! We describe some reasons why we are excited about replicating safety experiments from frontier labs:
1. this research is often vague and closed source
2. it rarely is stress-tested or replicated
3. replication allows non-lab reserachers to interface with lab work
New blog post from me and @Yixiong_Hao: "Empirical safety claims from frontier labs should be replicated, scrutinized, and open-sourced"
more thoughts in thread🧵
lesswrong.com/posts/MmfzfGcQ…
8
564
Second Look Research retweeted
great work by Christine @secondlookxlab!
In case you need more confirmation that GPT-Astra is terrifyingly good at no-CoT computation: I ran my replication of Ryan Greenblatt’s no-CoT evals on Astra (+ Gemini 3.1 Pro, Kimi k3, Fable 5.1). It is a qualitative jump on every dataset, especially competition math and multi-hop reasoning. 🧵
1
12
930
New post from Second Look Research: "Yet another concerning result on Astra's no-CoT capabilities"
In case you need more confirmation that GPT-Astra is terrifyingly good at no-CoT computation: I ran my replication of Ryan Greenblatt’s no-CoT evals on Astra (+ Gemini 3.1 Pro, Kimi k3, Fable 5.1). It is a qualitative jump on every dataset, especially competition math and multi-hop reasoning. 🧵
8
340
Another great replication from Finn! Surprising results from the GDM team replicate on Olmo 3!!
GDM recently found that Gemini's safety eval results are mostly set after SFT, and later stages like RL barely move them. The authors were surprised by this, so we replicated it on Olmo 3 Think. We find the SFT, DPO, and RLVR are within noise of each other on every safety eval we ran.
4
106
Second Look Research retweeted
New post from @secondlookxlab: Rerunning AI safety papers on every frontier release would be pretty easy and valuable 👀
We think that AI safety experiments that are less eval-shaped are rarely rerun on new releases even though there could be lots of actionable information here!
2
5
27
2,610
Second Look Research retweeted
Come work on replications of AI safety research with @zephaniahbroe and I at @SPARexec! @secondlookxlab is hosting 2-4 mentees over the Fall semester.
SPAR runs Sept-Dec, part time, and our application should take <1 hour. You can expect 2+ hours of mentorship from Second Look along with external mentorship from researchers at orgs like Anthropic, X AI, Oxford, Redwood Research, and Geodesic.
secondlookresearch.com/
2
8
56
3,475
Fantastic work from Christine!! We replicate single-forward-pass evals on a variety of frontier models and find instances where models have a substantial jump in performance📈
Can frontier models solve hard problems in a single forward pass without thinking? For my first project w/ @secondlookxlab I extended Ryan Greenblatt's no-CoT evals to Fable 5, Opus 5, and GPT-5.6-Sol and saw a substantial jump on some evals!
5
361
Second Look Research retweeted
New replication from Second Look: turns out introspective awareness in LLMs (or at least internal state control) may not be emergent.
great work Finn ‼️‼️
Can LLMs control their internal states? Anthropic found Claude can “think about” a concept on command without saying it. We replicated this in 14 open-weight models and found it to be a general property, maybe not ‘emergent’, appearing in models even as small as 270M parameters.
2
2
35
3,234
Second Look Research retweeted
the relationship between superposition and adversarial examples has always been super confusing to me but xijia did a great job making this super digestible in this blog post!!
i'm learning so much from reading the posts that the second look fellows are putting together. highly recommend people read this one!!
Why do adversarial examples exist? Recent research suggests that adversarial attacks leverage interference between features. We replicated superposition in toy models, and found that PGD attacks align with the optimal perturbation direction predicted by latent geometry.
1
1
14
1,715
We replicate experiments showing the relationship between superposition and adversarial examples in toy models!
Xijia did a great job making this highly confusing topic clear and concise. There is also an open source organized repo which we recommend readers play around with!
Why do adversarial examples exist? Recent research suggests that adversarial attacks leverage interference between features. We replicated superposition in toy models, and found that PGD attacks align with the optimal perturbation direction predicted by latent geometry.
1
6
294
We were able to replicate Jack Lindsey's internal activation control experiments on models as small as 270M parameters!
Because we found the same core result on every model we tested across 3 model families, this is looking like a general property of LLMs🫣
Can LLMs control their internal states? Anthropic found Claude can “think about” a concept on command without saying it. We replicated this in 14 open-weight models and found it to be a general property, maybe not ‘emergent’, appearing in models even as small as 270M parameters.
4
207
Second Look Research retweeted
Can LLMs control their internal states? Anthropic found Claude can “think about” a concept on command without saying it. We replicated this in 14 open-weight models and found it to be a general property, maybe not ‘emergent’, appearing in models even as small as 270M parameters.
3
5
36
5,037
Second Look Research retweeted
In my first project for @secondlookxlab, I replicated and dug deeper into Google Deepmind's "When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors".
2
6
24
1,827
Second Look Research retweeted
shoutout to Arav for producing @secondlookxlab's first replication this Summer!
it's a replication of the DeepMind paper "When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors" (arxiv.org/abs/2507.05246). Check out his work in the LW post!
more replications are on the way 😆
Read the work here: lesswrong.com/posts/AoBTiL7X….
Shout out to @Yixiong_Hao, @zephaniahbroe, and @andyw_ais for their help!
2
2
24
1,388
First ever Second Look replication is out!!! Shout out to @AravDhoot for fantastic work👀
In my first project for @secondlookxlab, I replicated and dug deeper into Google Deepmind's "When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors".
1
6
194
Second Look Research retweeted
I’m co-founding Second Look Research with @zephaniahbroe and we are accepting summer fellowship applications for 2026!
Fellows will come to the University of Chicago, complete 2-3 replications over 10 weeks (June 15-August 22), and work with external advisors. Fellows will receive a stipend of $10,000 and we will to cover housing and meals.
secondlookresearch.com/
18
51
371
51,134
Second Look Research retweeted
Announcing Second Look!! So happy and proud that this is happening❤️
I’m co-founding Second Look Research with @zephaniahbroe and we are accepting summer fellowship applications for 2026!
Fellows will come to the University of Chicago, complete 2-3 replications over 10 weeks (June 15-August 22), and work with external advisors. Fellows will receive a stipend of $10,000 and we will to cover housing and meals.
secondlookresearch.com/
1
1
15
2,959