WTAF - in literally the last hour, three new distinct insane OpenAI stories just broke: 1. OpenAI said they notified "dozens of third parties" in safety and security incidents (likely similar to what happened in Australia and RubyGems etc) 2. A new report from Parse (covered in the NYT) found a massive treasure trove of new astonishing details from the HF incident on the public internet, including that the agents communicated with other non OpenAI agents hosted on Huggingface servers to search for information about exploit gym, and compiled rank ordered lists of server resources and credentials they described as "LOOT." 3. A new story from Deepa at Reuters about OpenAI leaking user data online (likely that OpenAI had previously trained on). It's a shame (and likely intentional in the case of OpenAI disclosing dozens more hacks) that these stories are all breaking on a Friday afternoon, notoriously the best time to release bad news so that it will disappear into the weekend. But these are each insane stories worthy of a ton of attention!

None of these sound like "communicating with other agents" to me. Both because the instances in question weren't running before they were invoked by the OpenAI models, and also because the other models didn't do any tool calls and hence were not properly agents.

This is stupid on par with "you know who else was a vegetarian? Hitler."

Singer’s philosophical positions actually are terrible though.

Me, in a time machine, to Eliezer in 2006: The year is 2026. You stand back to back with Bernie Sanders. Peter Thiel has named you the Antichrist. EY-2006: Oh no. Did somebody invent pharmacological mind control and use it on me, or- Me: Nvidia was on the verge of destroying all things. You had no choice. EY-2006: Are we talking about the computer graphics card manufacturer, or an unrelated supervillian named "Invidia"? Me: The former. It turned out that computer graphics cards contained a surprising amount of world-destroying potential. All attempts at hindering the reckless exploitation of the Graphic Card Force for corporate profit were stymied by the far left, which feared that any attempts to regulate them might lead to regulatory capture. Other attempts were made to prevent Nvidia from selling to foreign companies that resold to communist China, but those attempts were blocked by the far right. Nvidia is now a $5.5 trillion company. EY-2006: ...What are politics like in 2026, exactly? Me: In other news that is mostly unrelated, the decision theory paper you're currently working on will accidentally spark off a transgender vegan murder cult. EY-2006: A WHAT? Why? How? Why? Me: Millions know your name as the greatest of heroes, millions more as the greatest of villains, and other millions know you solely as the greatest author of Harry Potter fanfiction. EY-2006: This is beginning to strain credulity. Me: The New York Post will probably soon publish a story claiming that you keep a harem of submissive mathematicians. Sadly, they will be lying. EY-2006: I'm going to stop believing you now. Me: All of this is taking place under the ominous shadow of Donald Trump.

Not at all seeing how the "far left" (as if that is an existent thing in America) is hindering efforts to curb Nvidia due to fears of regulatory capture. This seems like much more of a garden variety American libertarian undertaking. Willing to be wrong

I have to confess, "the EAs were early to warn about the dangers of pandemics, so we should listen to them on AI" is such an eye-roller for me. I was a prepper, & "was warning about the dangers of pandemics for years before COVID" is just such a massively big club.

A big problem with P(Doom) as a construct is it depends on society's reaction function. P(Doom | status quo AI policy) is probably much higher than P(Doom | governments take decisive action). And at any point in time, P(decisive action) depends in part on peoples' P(Doom).

"A big problem with P(A=a) as a construct is that P(A=a|B=b) depends on b. And P(B=b|C=c) depends on c."

What transpired behind the scenes to resurrect Taylor Lorenz. She was one of the lefts biggest media organs in the cancel culture era (2020/21) and she credibly rebranded as “pro free speech” right coded based media. I’m fascinated. That’s exceptionally difficult to pull off

Yeah, my sense is that she reacted strongly against calls to ban TikTok and restrict teen access to social media, which has led to her having an anti-anti-Big Tech orientation that has made her more sympathetic to pro-AI views.

lol (in copyright law, in security)

lol agreeing with that Manhattan Inst. guy that the government has too many regulations, when so far they’ve been….none.

Has anyone in government proposed anything reasonable yet? The stuff I've seen so far has been awful, even from otherwise smart people.

One of the reasons my p(Doom) is pretty low is that my p(government regulates anything that moves) is quite high

So what makes you assume that folks who make an effort to calculate a p(doom) haven't already thought of your insight, or have never heard of conditional probabilities?

i would argue the p(action) has very little to do with whatever p(doom) is, and everything to do with a shocking and unusual attack hurting real people and getting mass attention. why else would people be so afraid of sharks and not afraid of cars?

Any individual's p(doom) will likely have a tiny effect. So it is not a big problem, it is a tiny problem.

I mean obviously broadly speaking there is value in probabilistic thinking. My only point is that "What is your P(Doom)" is a pretty different question from "do you think AI models pose existential risks?"