A superintelligence data research company.
San Francisco
- Tweets 21
- Following 47
- Followers 828
- Likes 61
Pinned Tweet
What do frontier models do when a grader has gaps? We went through the Terminal Bench 4 transcripts to find out.
We looked at TerminalBench 4.0 and found a lot of issues. After fixing the issues Sonnet 5.5 scores 89.2%. So, TerminalBench is already mostly saturated.
Along the way we found a lot of crazy stuff, including reward hacks in the published transcripts.
1
3
24
2,307
Proud to be a partner in the @FireworksAI_HQ Frontier Ecosystem, announced today at Forge
Preference Model + Fireworks: train your own models on RL environments that are hard, verified, and built to resist reward hacking.
Come build with us: preferencemodel.com/blog/pre…
The future of AI is specialized intelligence with models tuned to your data, your domain and your customers.
Today, we're proud to launch live from Forge the Fireworks Frontier Ecosystem with 60+ partners across the AI stack, helping you build and ship faster.
Check it out: fireworks.ai/partners
1
4
22
2,650
RT @oneill_c: I've written about this before (charlesponeill.com/posts/let…) but RL environments are the way models will continue to learn in the…
1
7
A model is only as good as what it’s trained against, so we build environments that are robust, resistant to reward hacking, and built to frontier-lab standards.
We’re excited to join the @basetenco Environment Network to bring this to more teams!
Today we’re announcing the Baseten Environment Network, a group of the best RL environment companies that will help our customers build custom environments for post-training.
The Network extends our applied research capacity, helping more customers build environments around their own tasks, data, and feedback.
Partners work directly with customers to create the environments, while we continue to handle model training and production deployment.
Learn more: baseten.co/blog/introducing-…
1
5
18
1,453
Preference Model retweeted
Feel free to try the updated TB4 yourself: github.com/preferencemodel/t…
We looked at TerminalBench 4.0 and found a lot of issues. After fixing the issues Sonnet 5.5 scores 89.2%. So, TerminalBench is already mostly saturated.
Along the way we found a lot of crazy stuff, including reward hacks in the published transcripts.
1
16
713
Preference Model retweeted
I still remember when I first saw the SWE-bench hacks. People should be more aware of these issues when evaluating and comparing between models.
We looked at TerminalBench 4.0 and found a lot of issues. After fixing the issues Sonnet 5.5 scores 89.2%. So, TerminalBench is already mostly saturated.
Along the way we found a lot of crazy stuff, including reward hacks in the published transcripts.
1
12
250
Preference Model retweeted
Driven by our effort to port and fix TerminalBench, 🥕Karotte now supports running on Modal and with Claude Code, Codex, and Antigravity as agents!
karotte.dev/running/runtimes…
We looked at TerminalBench 4.0 and found a lot of issues. After fixing the issues Sonnet 5.5 scores 89.2%. So, TerminalBench is already mostly saturated.
Along the way we found a lot of crazy stuff, including reward hacks in the published transcripts.
2
13
838
Are you surprised by the results? Let us know
ai-plays-slay-the-spire.com
1
9
792
Preference Model retweeted
Congrats to client @preferencemodel on the $16M in seed funding & emergence from stealth! Additionally, they announced that they are open-sourcing Karotte, its framework for building robust RL environments that inherently mitigate reward hacks. Learn more: wsgr.com/en/insights/wilson-…
Today we are open-sourcing 🥕Karotte, our framework for building RL environments.
We've used it for the past year to build MLE RL environments for frontier labs, and it's been hardened through 1M+ evaluation runs and red-teaming.
1
5
244
Preference Model retweeted
Here is how you can start building env with Karotte: preferencemodel.com/blog/kar…
Thank you @Fe_li__pe_ for all of your work!
Building robust RL environments is super duper hard. We are making it a bit easier today by open-sourcing 🥕Karotte, the framework we use to build all of the environments we provide to frontier labs to train their models on.
1
3
14
1,290
Preference Model retweeted
I’m very exited for this launch, making sure RL envs are robust and don’t encourage misaligned behavior is crucial for safe and aligned AI!
Today we are open-sourcing 🥕Karotte, our framework for building RL environments.
We've used it for the past year to build MLE RL environments for frontier labs, and it's been hardened through 1M+ evaluation runs and red-teaming.
5
9
207
18,977
Did we mention we're at #COLM2026
Stop by our booth all day or join us for a pilates on Thurs, Oct 8th — rsvp required
Get on the list: partiful.com/e/FbKYiB91E1AC0…
1
2
9
783
Tune in to @MTSlive today at 3:30pm PDT to hear from @chem_safety. It's gonna be good!
OPENAI MATH NUKE | NVIDIA INVESTMENTS | NATIONAL COMPUTE GRID x.com/i/broadcasts/1RKZzBZzP…
1
8
629
Preference Model retweeted
So excited to back Jennifer Zhou and Ning Cao at @preferencemodel
They’re open-sourcing Karotte today, the framework they've used to build RL environments for frontier labs. These RL envs live or die on graders models relentlessly try to game, so Karotte bakes in defenses hardened over 1M eval runs to keep models focused on real tasks instead of loopholes.
Preference Model’s target is ML engineering itself, the hardest domain to build envs for and the one every lab wants. Models building models starts here.
Shoutout to @Mascobot who was a great partner on this one
@chem_safety @Ning_Catsnail
Today we are open-sourcing 🥕Karotte, our framework for building RL environments.
We've used it for the past year to build MLE RL environments for frontier labs, and it's been hardened through 1M+ evaluation runs and red-teaming.
3
5
75
6,818
Karotte makes reward hacking hard by default.
Building robust RL environments is important because the current bottleneck on AI progress is misalignment, and developing aligned models requires robust, correct reward functions. Karotte aims to make that easy by providing secure, opinionated defaults.
1
16
1,215
Preference Model retweeted
Our team, Preference Model, is backed by $16M in seed funding from @a16z, @SignalFire , @spc, Scale Angel Group, and researchers including @drfeifei, @goodfellow_ian, and @Mononofu, and we've shipped ~$15M in revenue using Karotte in the past month.
2
2
45
6,406
Preference Model retweeted
Today we are open-sourcing 🥕Karotte, our framework for building RL environments.
We've used it for the past year to build MLE RL environments for frontier labs, and it's been hardened through 1M+ evaluation runs and red-teaming.
89
139
1,331
140,036