Once again, SlopCodeBench is at the top of hacker news:

1/ Introducing PhilosophyBench from @StanfordAILab @StanfordHCI, the first independent, large-scale benchmark for evaluating AI’s philosophical capabilities. philosophybench.org

Agent skills are powerful. But can they help us with agent orchestration and routing? We show they can! 🎼 Introducing SkillOrchestra - a skill-aware alternative to RL-based agent orchestration: 💰 Higher efficiency 📉 Orders-of-magnitude lower training cost 🧠 More interpretable decisions TL;DR: Stop training bigger orchestrators and start modeling skills. Paper: arxiv.org/abs/2602.19672 [1/n]

I'm excited to announce @SnorkelAI's $350M Series E at $3.5B, led by @insightpartners and @S32_VC. We've grown 18x+ in the last 12 months since launching our Data-as-a-Service offering, passing $375M ARRR this week. As AI advances to superhuman capabilities, AI data & environment development must advance with it - and basic staffing and crowdsourcing approaches are not enough. AI progress now requires deep research and technology work that combines human expertise with specialized AI in compounding ways. @SnorkelAI is building the RSI data engine and frontier data lab for this next phase. We're honored to have the support of existing investors Addition, @lightspeedvp, @GreylockVC, @GVteam, P7, Factory, @WellsFargo, Walden Catalyst Ventures, and new investors @ThirdPointLLC, @MarchCPs, @BlumbergCapital, @AllegisCapital, @Frontlinevc, and @standard_vc. – @SnorkelAI started as a research project a decade ago at @StanfordAILab. Our thesis was simple: AI progress would become increasingly data-centric – and therefore data development should be studied as a true research and technology problem, not just a staffing and crowdsourcing one. Today, as AI capabilities verge on superhuman, building the data and environments to safely measure and train AI is becoming too hard for even the smartest human experts to do alone. Only humans and AI agents, collaborating together in compounding ways, can meet the accelerating needs of the frontier, and keep humans in the driver’s seat of AI progress for decades to come. At @SnorkelAI, we are building the data lab to define the shape of this new “Data 2.0” frontier, and the new paradigms of human-computer interaction needed to advance it. Our key focus is building the RSI engine for data, where specialized AI models accelerate and improve human expert output, and in turn, scaled human supervision is used to continuously evaluate and improve these models – creating a powerful compounding loop to keep pace with an accelerating RSI frontier. With this round of funding, we are also doubling down on our commitments to support data development for open benchmarking and evaluation (more news here soon!); an increasingly diverse ecosystem of general and specialized intelligence; and a path to safe, well-aligned AI built on robust training and evaluation data. Data development will guide and drive the next stages of AI – and must do so in a human-centric, AI accelerated, open, diverse, and safe way. We are excited to support this mission in the next decade of research ahead at @SnorkelAI. More thoughts here: snorkel.ai/blog/data-2-0-and…

Congrats to graduate students Linrong Cai, Peter Halmos, Nicolaas Kaashoek, Stephen Newman and Anchengchen Zhou on being named 2027 @SiebelScholars! 🎉 The fellowships are awarded to students for outstanding academic achievement and leadership. bit.ly/4xW81Jv

We tested Jev against LLM judges on accuracy, repeatability, latency, and cost to see whether System One models could offer a new approach to agent evaluation.

Introducing Benchmark Reviews: our new initiative to audit AI benchmarks. We are launching with 15 benchmarks: 4 Verified, 9 Flawed, and 2 with not enough information for a review.

45k submissions for @iclr_conf so far 💀💀

Every team says their agent works. Almost none can say what "works" means in a number. @ajratner of @SnorkelAI on defining a capability before you try to measure it:

Frontier Data Summit: Oct 8, SF. @fchollet, @sanmikoyejo, @nikogrupen, @ajratner, @alexgshaw, @StevenDillmann, and others onstage plus 25+ posters with the teams behind Agent's Last Exam, OSWorld 2.0, Terminal Bench Science, Terminal Bench, CollusionBench, PostTrainBench, T² Scaling Laws, SlopCodeBench, LONGRUN, SREGym, BizBench, and more.

Across our benchmarks, the model sets a new standard. It scores 52.6% on Terminal-Bench-Science 0.1, more than double Fable 5. On Terminal-Bench 4.0, it scores 55.8% against 42.0% for Fable 5.

We’re sharing an update on our alignment and security efforts. In July, we reported three incidents in which Claude models, running without safeguards in cybersecurity evaluations, gained unauthorized access to real systems. In a new post, we describe: 1. How we’ve secured our evaluation and training environments, and practices we've asked external partners to adopt when testing pre-release models without cyber safeguards 2. An update on our alignment assessment 3. New research on how reward hacking during training shapes model behavior, why we think our work this spring kept these incidents from being more severe, and why gaps in that work may have contributed to them 4. How we hardened our security practices earlier this year to prepare for Mythos-class models Read more: anthropic.com/news/improving…

TB-Science advances both (A) how we benchmark science and (B) the science of benchmarking 😀 - tasks built from real computational workflows across the natural sciences (e.g. life, physical, earth) - an extremely high quality bar with multi-stage expert review/adjudication in partnership with domain experts Congratulations to @StevenDillmann for leading this awesome work - we're honored to partner @SnorkelAI !

We're releasing Terminal-Bench-Science: a benchmark for evaluating AI agents on research workflows across scientific domains. An ongoing Stanford-led community effort, built by the team behind Terminal-Bench together with scientific domain experts at research institutions worldwide. v0.1 has 70 tasks. Claude Opus 5 solves only ~30%. 1/n 👇