humans are so predictable that computers can easily predict what someone will type next 80% of the time, even if they try to type randomly except for this one guy, who became totally unpredictable by just "using his free will"

Introducing Project Glasswing: an urgent initiative to help secure the world’s most critical software. It’s powered by our newest frontier model, Claude Mythos Preview, which can find software vulnerabilities better than all but the most skilled humans. anthropic.com/glasswing

People are chanting “formal verification” as if it were a magic spell to ward off the slop devil. Let’s stop and think about why hardly any man-made software is formally verified.

Theorem co-founders @rajashree + @diagram_chaser explain why verified AI sandboxes are still months away despite the latest breakthroughs in automated theorem proving: Rajashree Agrawal: "The models just got good enough to prove these theorem statements, or they're still getting there. One of the costs is just tokens." Jason Gross: "Taking the recent Navier-Stokes news as a baseline, the models wrote something like 600,000 lines of Lean in about 17 hours. This works out to between one kilobyte and 30 kilobytes verified per hour." "The smallest version of Linux with a sandbox that we can make is about five megabytes. So that's still a handful of months away, even at the rate of the Navier-Stokes auto-formalization." "We need to build a pipeline from verifying the software back into RL-ing the models, so that if you want to verify all production software that exists in the world, it costs you less than $10 trillion." @theoremlabs

Fucking finally I expected this to be done in 2025 Now repeat with Bel or whatever that thing is called, and give us a report on how much Astra has missed

Greg Brockman says OpenAI pointed Astra at its own systems until it ran out of vulnerabilities to find: "We took 25% of our production engineers and said, 'Sorry, all your projects are on hold. You are now defending. You are now up-leveling our security architecture. You're going to use the models to find all the holes.' And we found a number of serious issues, and we fixed them." "We found some new problems, but eventually it saturated. We basically have found, to our knowledge, all of the P0s, all of the critical problems that Astra is smart enough to find. And of course, there will be a new model, there will be a new round." "You want to be in this tight loop of new cyber capability drops, you deploy it against your systems, you find the new holes, and ideally, you've managed to automate this, what we call defense factory. That's what we're building internally." "There are ideas, for example, formally verifying all of software, that are possible with AI." @gdb @bhorowitz