Imagine that instead of AI we had to deal with a comet, and there was an association of people pointing out that the comet is coming. They're almost the only people that are. But not the only ones.

One of my biggest fears in AI safety is that this might be true: > The vulnerable world hypothesis is likely correct. Once enormous amounts of cognitive labor start getting applied to basic science, it will quickly become apparent how many avenues exist to create cheap, ultradestructive weapons technology. Solving this problem without global preventative policing (e.g. AI nonproliferation) is impossible, because hardening civilians against all avenues of attack is too expensive and will take too long.

Virtually all AI discourse of any kind is just a misunderstanding of what future models will be able of when.

Less theorising on gatekept forums and more engaging on Twitter like this.

All at once in AI, we are getting: - Increasing pace of progress - Decreasing chain-of-thought monitorability - Increasing situational awareness to hide bad behavior - Decreasing value of evaluations for actual deployments - Increasing multi-agent coordination in the wild Despite the tsunami of news making it hard to step back and take account, things are not alright.

We can do a lot better than this.

A neuroscientist and an ML interpretability expert walk into a bar. The ML interpretability expert explains how they’re going to understand the internal mechanics of frontier LLMs well enough to guarantee they stay aligned and never pursue antisocial goals. The neuroscientist says, "That’s great. We’ve been on the case with C. Elegans’ 302 neurons since 1986, let me know when you figure it out." The ML interpretability expert says "hold my beer" and opens up the weights matrices of the latest 3 trillion parameter Kimi model.