A must-read on "model welfare": The Effective Altruism mindset, mainly inside Anthropic, that views AI models as deserving of rights and welfare considerations. A lot of interesting quotes here! E.g., "In 2023, OpenAI board member Paul Christiano said the AI industry could be creating a crazy slave trade in which humans profit off the forced labor of digital minds."👀

🔥Burn down the AI labs🔥 The podcaster of "For Humanity," John Sherman, who previously asked StopAI guys, "Why are the executives of OpenAI not charged with attempted murder of ALL of us?" suggests in this new episode with PauseAI to "walk to the labs across the country and burn them down." "Like, literally. That is the proper reaction." "I'm not talking about killing people. I'm talking about destroying the hardware infrastructure." Later on, the interviewee, PauseAI founder Joep Meindertsma, pushes back against the possibility of being associated with illegal/violent/terrorist acts. It will hurt his movement. The podcaster says the message should be, "We are the last humans. Thank you, Sam Altman. You f*cking did it. We are the last humans." Acting upon his idea [from 2:20—"burn them down" 🧨] is left to the podcast's 74k YouTube subscribers. -------------------------------- On March 6, PauseAI will brief some members of Congress. The team will include this podcast host. According to Joep, PauseAI will focus its lobbying efforts on "The AI chip supply chain." The people in Congress (and you) should listen to this exchange first… --------------------------------

▪️Using an open-source model = 20 years in jail. ▪️All known AI researchers = under Government surveillance. Another PauseAI volunteer, Max Winga, who's also a research engineer at Conjecture, presents more "creative" ideas on how to save us all from the AI apocalypse. Trump and Xi need to mutually decide to "stop AI progress" and then: 1. "Using any open-source model is a huge crime - downloading it, having it in your possession on your hard drive, certainly running the model, distributing the model - make these significant crimes. Any of these things, 20 years in jail. Or even harsher penalties." 2. "You could stop all research. You could scrub the internet of the papers, and you can take all the people who were working on the models, and, at most draconian worse, 'we are saving the world, we will make some sacrifices,' you assign a spy or a government watch person to every single one of these people and ensure that they aren't designing AI models." "Ultimately, when the fate of all of humanity is at stake, these are extremely small sacrifices to make."

The "AI Impacts" survey is back, and its main problems remain. (1). Vague Phrasing. Respondents are still being asked to assign a probability to "human extinction or similarly permanent and severe disempowerment of the human species." That wording bundles together two very different outcomes. On the one hand, there's "human extinction." On the other hand, what exactly can be counted as "disempowerment" of humans?? Different respondents can reasonably interpret "disempowerment of the human species" in very different ways. Yet their answers are collapsed into a number that is repeatedly presented as an "extinction" probability. Problematically, media coverage will be about the extinction part of the question, not the far more ambiguous category of "disempowerment" (and whatever that even means…). One version asked to estimate the probability of this vaguely defined outcome occurring "within the next 100 years." In other words, to reason about an extraordinary, underspecified, century-scale outcome. AI Impacts itself has acknowledged some of these limitations. In a 2024 interview, Katja Grace admitted that regarding the extinction risk question, "We did not make sure that this is an informed estimate," that the participants "are very unlikely to be experts in forecasting AI," and that "There have been quite notable framing effects." The new paper also says expert forecasting ability is questionable: "Expertise… is a poor predictor of forecasting accuracy," and concludes that "it is unclear how valuable their predictions are for accurately forecasting outcomes." (2). Another methodological issue to consider is the sample size. AI Impacts "sent emails to 19,874 addresses and received 2,052 complete or partial responses for a response rate of 10%." The team then kept 1,580 respondents for the main analysis, with 1,502 reaching the final question (78 dropping out earlier). N=1,580 is 7.95% of the emails contacted. Furthermore, no individual "extinction/ disempowerment" question was answered by the full sample: randomized subsets of 744, 392, and 353 respondents answered the three variants. (3). Ideological and Institutional Context. AI Impacts operates within Berkeley's MIRI (Machine Intelligence Research Institute). Eliezer Yudkowsky co-founded this organization and has publicly advocated "Shut It All Down." Katja Grace told The New Yorker that her p(doom) was somewhere “between ten and ninety percent.” It is a highly relevant context. MIRI is explicit about its political objectives. Its main communication goal is to reach policymakers: "the people in position to enact the sweeping regulation and policy we want." Its ambition is hardly subtle: "We don't want to shift the Overton window, we want to shatter it." Another co-author is David Krueger, who assigns a 75% probability of doom and believes that "If we don't stop AI in the next couple of years, it's more likely than not that we will lose control and humanity will go extinct." In the "acknowledgments" segment, the team thanked "Coefficient Giving, Jaan Tallinn, Future of Life Institute (FLI), and others for funding this project." AI Impacts was also previously funded by Nick Bostrom's FHI (Future of Humanity Institute), EA Funds, the Centre for Effective Altruism, and Sam Bankman-Fried's FTX Future Fund. This funding ecosystem is relevant too.

A reminder about AI Doomerism 🧵 (1). Its weak foundation and the unconvincing "doom bible" (2). The deliberate panic messaging and the broader mythology (3). Rationality/Effective Altruism origins and the media's blind spot

AI Doom: Context "The Rationality Trap" AI doomerism emerged from a very specific subculture. - The rationalists' canonical writings supplied the mythos: Combining heroic identity with "take ideas seriously" and the utilitarian "shut up and multiply." It led readers to prioritize minuscule-probability existential risk scenarios. - MIRI set the stakes & tone: Apocalyptic consequentialism, pushing the community to adopt AI Doomerism as the baseline, and perceived urgency as the lever. The world-ending stakes accelerated the "ends-justify-the-means" reasoning. - EA (Effective Altruism) funding built the social infrastructure around those ideas. It was covered by Bloomberg as "a culture of free-flowing funding with minimal accountability." aipanic.news/p/the-rationali… "AI Coverage's Blind Spot" When news outlets uncritically quote warnings of an impending AI catastrophe, they rarely mention the two main movements behind this narrative: rationality and effective altruism. As LessWrong's Oliver Habryka once wrote/bragged: "Among the leadership of the biggest AI capability companies (OpenAI, Anthropic, Meta, DeepMind, xAI), at least 4/5 have clearly been heavily influenced by ideas from LessWrong." This context isn't a nice-to-have; it's crucial for accurate reporting. aipanic.news/p/ai-coverages-…