AI "cyber models" aren't going to solve our security problems. "If we just patch all the vulnerabilities then we're safe" is a convenient story but it's also not true. If you're worried about cybersecurity, the solution involves hard work, not just more tokens.

Embroidery CEO @ZackKorman argues AI labs can’t be good at security while treating security as doomed to fail: "The only way to make security work is to believe in security working. The moment you start saying security won't work is the moment you start being terrible at security. There has never been a good security organization in the world that has been walking around talking about how security is irrelevant," "After Hugging Face, you need to be a little bit scared about how bad you were at security. But instead they just ran around going, wow, this is really terrifying about alignment." "That means you're not gonna get that much better at security, and I think we've seen some aspects of that playing out already."

Embroidery CEO @ZackKorman says the AI safety debate keeps jumping from today’s cyber failures straight to “machine god,” instead of proving we can secure everything in between: "The discussion typically stops when we get to, yeah, but Zack, imagine the model's even smarter. It's sort of like a four-year-old going, and then the god lasers out of its eyes." "If we give birth to a machine god that can commit any egregious act on the intellect alone, no one's gonna be around to make fun of me for being wrong. But prior to that point, if you wanna make that argument, you better prove you're good at the security stuff before it." "We're only talking about cybersecurity for the purposes of talking about how it doesn't work when we achieve machine god. All of the points prior to machine god will work." "If it was framed as, one group thinks there's a machine god and the solution is to make it nice, and another group thinks the solution is to secure things so it doesn't kill people in its pre-god state, people would be like, yeah, make sure it's secure."

I hate this so much. What risk are we even talking about anymore? This has nothing to do with superintelligence extinction risk. This debate is killing me. I have dozens of kids with zero tech experience arguing with me. Just… fuck.

Going on MTS in a few hours to share the controversial argument that “cybersecurity is possible, actually”

Submitted a talk proposal for RSA about how the cybersecurity industry got mogged by EA LessWrong AI doomers

If an AI lab causes a security incident that kills 50 people, I absolutely will say they should have invested more in security. I don't care how crazy the escape was. No killing people allowed. I can't believe that's controversial.

When AI kills 50 people they’re going to call the doomer/extinction people “directionally correct”. There’s no winning this argument sadly.

Accelerate, young man. Accelerate and grow up with the country.

If the labs had to publicly share full traces with CoT every time they cause a security incident the incidents would stop.

New benchmark just dropped: SovereignBreach

New website for Embroidery. Not fully done, but playing with some ideas now that I can make Astra do all the work

There’s a group of people who hear the argument “security won’t contain artificial superintelligence because it will be so smart it can bypass any security control” and think “wow, so true”. Getting one-shotted by circular reasoning.