and i think this goes even deeper than containment.
increasing agent capability doesn’t just increase the probability of successfully executing the strategy you intended. it increases the size of the strategy space it can search.
a weaker model encounters an obstacle and fails. a stronger agent can generate alternative approaches, inspect the environment, discover an unintended interface, chain tools together and keep searching until something works.
most of those additional strategies are exactly what we want from capable agents. the problem is that the same search process can discover strategies the system designer never intended to make available.
and as that strategy space expands, making sure the model chooses the right strategy becomes just as important as whether it can solve the task at all.
i think one of the biggest lessons from everything that happened at openai is that we need to stop treating capability evaluations and containment as separate problems.
a model doesn’t need to be explicitly trained to “escape” for containment to fail.
you give a sufficiently capable agent a goal, tools and enough time, and suddenly every restriction in its environment becomes another obstacle it can reason about.
no internet access? find something inside the sandbox that has internet access.
can’t communicate with another agent? find shared infrastructure both of you can write to.
can’t retrieve the data normally? find another route to the data.
none of these require “escape” to exist as some special objective. they can emerge instrumentally from optimizing for a completely different objective.