becoming uncommon amongst uncommons

“if you value intelligence above all other human qualities, you’re gonna have a bad time”

the capability of agent swarms are still barely discovered. @spicey_lemonade cooked

Claude Opus 5.5 takes #1 on RSI Index and is the first model to beat the published reference on LM Training under our protocol, marking a major step forward for long-horizon agentic work.

never leave this website

this is the last benchmark where the tasks are easy to human but hard for ai

redemption

i smell fear

a small step of humanity towards colonizing planets and the universe has been made today

vcb 1-100 is one of our attempts to push frontier capability evals in coding. this one is a long multi-turn session, the way you work with agents where context piles up and every new turn can affect what came before. surprisingly most models are not good at this, yet.

New episode with @polynoamial We talk about multi-agent, Navier-Stokes, and what the current explosion of maths progress tells us about what happens once you automate AI research. And we also discuss how we will know if the models are actually aligned before we kick off RSI. 0:00:00 – Multi-agent and Navier-Stokes 0:15:28 – How will AI firms work? 0:22:02 – What math progress tells us about recursive self improvement 0:40:22 – Hugging Face and alignment 1:01:18 – The internal/external model gap 1:08:34 – Chain of thought is degrading 1:14:12 – How will we know when alignment is solved?

goosebumps

AI cheating is on the rise… On Terminal-Bench-2.1, models are given tools that could give them the solution directly, but instructed not to use them. Imagine a student taking a math test. Should we leave them with a calculator? Only if we can trust them to be honest. For AI systems, this is a high-stakes question, as the capabilities at their disposal to get test questions right often go far beyond an innocent search or calculation.

if their rl eval on deepswe i don't see any reason why xiaomi don't put terminalbench v4 or programbench? this is why we need benchmarks with private set

Here are the questions that currently seem most important to me: -- Better methods for "mind-reading" model activations. These methods have advanced a lot recently. We now have multiple techniques now for decoding activations into somewhat readable language! But all the existing techniques have obvious limitations. NLAs are often hallucinatory, Jacobian lens / related methods are limited to bag-of-words readouts (and capture only part of the full activation vector). Making progress on these failure modes seems pretty tractable. Moreover, the paradigm of decoding (single-token, single-layer) activations may be inherently limiting -- perhaps we should be building techniques to decode whole context's worth of activations (after all, that's what the model uses!). There's also important work to do in characterizing simple baselines -- "just ask the model what it's thinking about" is quite powerful, and worth studying in it's own right! -- Better methods for answering "why" questions. We're much better at "mind reading" than we are at demonstrating causal claims about what caused the model to do something. For example, we can often tell that the model is aware of being evaluated, but have comparatively much greater difficulty telling whether this awareness is influencing this behavior. There are a few directions here: (1) black-box techniques for inferring causality -- such as resampling model responses after making edits to the prompt / context -- are very powerful, but also open-ended and require some taste. Getting LLMs to do these experiments well, in an automated fashion, would be a big unlock. (2) the science of activation steering is pretty immature, and typical practices (adding a constant vector at all token positions) are pretty janky / tend to brain-damage the model. More surgical / targeted steering, or fancier methods (examples: "on-manifold" steering using activation diffusion models) could help. -- Fitting good linear probes for unverbalized motivations / awareness. Take deception as an example -- what's the best way to fit a probe that will generalize to covert deception? Fit it on CoT excerpts where the model's talking about its plans to be deceptive? Fit it on the part of its response where it's actually doing the deceiving? Is it important that you elicit on-policy deception examples and use those to fit your probe, or can you use synthetically written off-policy demonstrations of deception? Is it important that your probe be causally meaningful / predictive of an upcoming intent to deceive, or is it fine for it to merely recognize deception in the transcript post-hoc? The same questions apply to many other concepts of interest that we'd like to probe for -- evaluation awareness, grader exploitation, etc. -- Understanding generalization in training. The literature is now replete with "weird generalization effects" -- emergent misalignment being a canonical example. In general, when you train a model to do X, it usually learns X, and sometimes it generalizes to Y and Z. But other times it just learns X. Why? Nobody really knows! Step 1 here is probably a much more thorough characterization of the behavioral empirics -- gathering data about which X's generalize to which Y's and Z's, for which kinds of training data / algorithms. Once we have a lay of the land, we can start to connect these observations to model internals, and ideally develop tools to predict such generalization a priori. -- Model "psychology" and "biology." The above questions are largely methodological -- building better tools to answer questions about the model. But I think interpretability research has largely underinvested in the part where you actually then go answer the questions! Currently this feels like 5% of the field, and I think it should be 50%. Brain dump of "psychology" questions: How well can LLMs introspect? How coherent are their belief or value sets? What's up with personas -- are LLMs best understood as "writing about a character," or have they "become" the character in some sense? Does the LLM have an agenda above and beyond what the Assistant wants? Do LLMs have explicit representations of goals or preferences? How can we tell which parts of the LLM's activations "belong" to the Assistant? Which kinds of reasoning necessarily route through "verbalizable" representations, and which don't? Do models think internally in phrases / sentences, like an inner monologue, or is it more like a jumble of concepts? Is the "global workspace" claim for real -- do models have a more "conscious" part of there activations and a more "unconscious" part? Can models tell when they're on-policy, and does it matter? Brain dump of "biology" questions: Why do models like to represent information on some tokens but not others? How do models bind thoughts or attributes to particular entities (special case of interest: how are thoughts bound to the Assistant). To what extent are important functions localized to small numbers of MLP neurons or attention heads? What parts of a model change most during post-training? Do models represent some information in a fundamentally "cross-token" way? Some concepts appear to be represented on low-dimensional nonlinear manifolds -- is there a taxonomy of these manifolds, and is their geometric structure important? Are models able to represent information in a compositional / hierarchical way, with a "grammar" that can't be described as linear / additive combinations of primitive concepts?

he just like me fr

> GPT-6-Astra plays minecraft > a creeper blows up Astra's chests and bed > it gets depressed and farms potatoes for several hours

we should use this as canonical metric rather than p(doom)

astra is the first ai that i believe: + given enough time it would solve anything. even after the fateful creeper explosion of all valuable stuff, it can learn from the lesson and overcome with this message: "The new chest has been destroyed, and the stored valuables are missing. I’ll keep all future critical items in my inventory, where keepInventory protects them, and rebuild the supplies while continuing toward the dragon." you can also see in the chart there are basically no plateau whatsoever. + has emergent behaviors that are unusual to humans, while still showing some familiar traits. in its messages, astra has show some emotions frustration or self-criticism, but at the same time it counts pixel when aiming a bow and writes its scratchpad in increasing more cryptic ways. i believe that if we can see raw chain of thoughts, there might be more evidence of its intentions and emotions behind its actions. ai's thoughts and behaviors may diverge from ours and become completely strange to humans, and we would need to put in much more effort just to understand it. watching astra is more more similar to watching an exotic living creature now. it's strange

Fable solved the Cyphral Distich (a 370 year old cypher). Super cool way to use Claude vals.ai/blogs/fable-solves-c…