Sorry to nitpick but we are working on a piece speculating on whether we'll see cross-agent communication/collusion so I'm trying to figure out exactly what happened.

None of these sound like "communicating with other agents" to me. Both because the instances in question weren't running before they were invoked by the OpenAI models, and also because the other models didn't do any tool calls and hence were not properly agents.

When you wrote "communicated with other non OpenAI agents" were you talking about this? It sounds like they just tried to use other models, which seems conceptually different from messaging other agents. Especially since GPT-2 wasn't close to being agentic.

The part that's ridiculous is calling Peter Singer the "father of AI doomer ideology."

Incredible.

He's talking about folks like Lina Khan.

Me, in a time machine, to Eliezer in 2006: The year is 2026. You stand back to back with Bernie Sanders. Peter Thiel has named you the Antichrist. EY-2006: Oh no. Did somebody invent pharmacological mind control and use it on me, or- Me: Nvidia was on the verge of destroying all things. You had no choice. EY-2006: Are we talking about the computer graphics card manufacturer, or an unrelated supervillian named "Invidia"? Me: The former. It turned out that computer graphics cards contained a surprising amount of world-destroying potential. All attempts at hindering the reckless exploitation of the Graphic Card Force for corporate profit were stymied by the far left, which feared that any attempts to regulate them might lead to regulatory capture. Other attempts were made to prevent Nvidia from selling to foreign companies that resold to communist China, but those attempts were blocked by the far right. Nvidia is now a $5.5 trillion company. EY-2006: ...What are politics like in 2026, exactly? Me: In other news that is mostly unrelated, the decision theory paper you're currently working on will accidentally spark off a transgender vegan murder cult. EY-2006: A WHAT? Why? How? Why? Me: Millions know your name as the greatest of heroes, millions more as the greatest of villains, and other millions know you solely as the greatest author of Harry Potter fanfiction. EY-2006: This is beginning to strain credulity. Me: The New York Post will probably soon publish a story claiming that you keep a harem of submissive mathematicians. Sadly, they will be lying. EY-2006: I'm going to stop believing you now. Me: All of this is taking place under the ominous shadow of Donald Trump.

And you were also early on AI! So "listen to people who were right about pandemics" seems like a reasonable heuristic overall even if you don't like one particular group of people who fits the criteria.

That isn't what I said so I'm not sure what point you're making.

I think in a lot of ways she's carrying forward the left/libertarian, tech-friendly viewpoint of groups like EFF circa 2012. That view has fallen out of favor on the left but it's not intrinsically right-wing by any means.

Yeah, my sense is that she reacted strongly against calls to ban TikTok and restrict teen access to social media, which has led to her having an anti-anti-Big Tech orientation that has made her more sympathetic to pro-AI views.

Obviously you can quibble over the definition of significant, but the harms from the Hugging Face attack, for example, were fairly minimal.

So far there also haven't been any significant harms from AI either.

I bet @NatPurser has smart thoughts about this.

Yeah.

What makes you assume I assume that?

Yeah, it partly depends on whether there is a "fast takeoff." But if AI starts causing serious damage, presumably some of those incidents will be spectacular enough to get public attention. It helps that we've all seen dozens of sci-fi movies where robots take over.

I mean it's a problem with using the concept, not that it's a big problem for the world as a whole.

Or to put it differently, P(Doom) combines two questions that people ought to think about separately: (1) are advanced AI models potentially dangerous? and (2) are human beings likely to respond effectively to neutralize the threat? It seems to me that (2) is often neglected.

I mean obviously broadly speaking there is value in probabilistic thinking. My only point is that "What is your P(Doom)" is a pretty different question from "do you think AI models pose existential risks?"