Data Science at @ThePropelApp. Formerly: Stitch Fix, Opendoor, Twitter, Facebook, neuroscience.
New York, NY
- Tweets 7,781
- Following 511
- Followers 5,648
- Likes 14,485
Chris Said retweeted
Good post. I have 0 problems with EAs. Many are weirdos, but so what, I love weirdos! Why are you here if you don't? My one issue is that any sub-culture that re-derives "right living" from first principles will probably blind itself to inarticulable wisdom embedded in tradition.
If your feed is anything like mine, you might have been hearing more about effective altruism this week. I think the key underlying reason for the surge is that concern about the risks from advanced AI have been getting more airtime recently, and EA was early to those concerns.
What’s driving the surge in attention on AI risks?
-OpenAI's Hugging Face attack (openai.com/index/hugging-fac…)
-METR/Redwood’s investigation of the hack came out, and (correctly IMO) freaked a lot of people out. (e.g. nitter.net/binarybits/status/2092…, nitter.net/SenJohnKennedy/status/…, nitter.net/BarackObama/status/210…, nitter.net/HawleyMO/status/209813…)
-Jacob Coxon’s resignation went mega viral as the first time a lot of people heard the (crazy, true) fact that lots of people building the most advanced AI systems think there’s a >10% chance they will kill everyone.
-That’s moving public opinion in a big way (nitter.net/davidshor/status/21003…, semafor.com/article/09/16/20…)
What does any of this have to do with effective altruism?
Well, effective altruism is a community that developed mostly online starting in the early 2010s around using evidence and reason to do the most good (especially with your donations and career choice). Three very different focuses have all been popular in the EA community since the beginning: effective giving in global health and development (e.g., donating to the kinds of organizations @givewell recommends, which do things like distribute bednets to prevent the spread of malaria), trying to reduce suffering for farm animals (which are orders of magnitude more numerous, treated vastly worse, and receive way less philanthropic attention than animals in shelters), and reducing risks to the future of humanity, especially from advanced AI. I personally came into this work from the global health angle - I had started working at GiveWell before the term “effective altruism” was coined - but Coefficient Giving, the funder I cofounded and now lead, over time came to fund work across all three of these streams of work (in addition to many other areas that aren’t typically associated with EA, such as the YIMBY movement to build more housing, work on science policy to accelerate discovery and economic growth, and research on new treatments for neglected diseases).
OK, so the EA community was early to work on AI safety, and CG has been funding a lot of the key players working on AI safety for a long time. As AI risk concerns have popped over the past couple of months, that’s led to more scrutiny on the EA community for IMO a mix of good and bad reasons.
I think the good reason is genuine curiosity (and maybe some healthy skepticism!) about these connections - where did the ideas of the people who are now leading giant AI companies come from? Why is everything in this world so interconnected? (My answer: it used to be a really small world - very few people were thinking about this stuff or taking it seriously until just a few years ago, so of course the ones who were found each other and started collaborating. It’s kind of wild IMO looking back how prescient some of the early writing from this world was (e.g. lesswrong.com/posts/6Xgy6CAf…). At the time, I was skeptical - I mostly just worked on global health and I was like “I dunno about this SV crowd freaking out about AI, how much can we really predict this stuff” but holy shit they were way more right than me, and I’ve moved in their direction a lot.)
But I think the bad reasons are unfortunately mostly self-interested. A bunch of powerful actors stand to benefit from unchecked AI progress, and they’re doing everything in their power to demonize or dismiss anyone with concerns. This leads to disproportionate discussion of EA because EA is interested in neglected and underappreciated ways of doing good. That makes it open to weird ideas. And people, very disproportionately with a financial stake in the AI fight, are trying to make it about EA instead, because EA being weird is much safer territory for them than the rather uncomfortable fact that many of the people making the most advanced AIs think that they might kill everyone.
(FWIW, my personal estimation of the risks is a lot lower than many of the folks who worry about this stuff. In debates like this one (asteriskmag.com/issues/03/th…), I often feel more sympathetic to the perspective of folks like my colleague @mattsclancy than the people on the other side, and I have a lot of time for @binarybits critiques (understandingai.org/p/the-ca…). I think a big part of the difference with more worried folks is that I expect society to react more vs sleepwalk into a crisis. But it is not lost on me that folks like @ajeya_cotra and @RyanGreenblatt who are more worried have had outstanding and falsifiable recent forecasting track records (theaidigest.org/2025-ai-fore…), making way better predictions than I would have. So I think their perspectives need to be taken seriously. And of course a single digit percent chance of everyone dying is way too high!)
Where should this leave you?
I think the main thing is not to get distracted. You absolutely do not have to be an EA, care about EA, or like EA, at all, to care about risks from AI and to be engaged on this issue. Everyone from Josh Hawley to the Pope to Obama are weighing in now, and that is great. This conversation was always too big and too important for any one community to drive. It’s unfortunate that we need to play catch up as society because until this year it was too weird for most people to want to engage with, and I think that should generate some grace for people who were earlier to these topics than most of us (certainly than I was). But it’s great the conversation is broadening now and it’s good to focus on the merits of where we are now and the appropriate policy response rather than getting caught up in the (in some sense unsurprising) fact that the people who were willing to think weird thoughts about the future of AI ten years ago were also interested in other weird ideas. If you’re not into weird ideas, that’s fine, you can just do you, you shouldn’t let this history stop you from engaging.
30
14
347
44,885
Chris Said retweeted
Frontier labs can put their money where their mouths are on catastrophic risks by building a mutual insurer.
Labs already face liability. We can use this. Within 3 months, with no government action, labs could create a body with skin in the game to avoid reckless deployments, teeth to enforce security practices, and market pricing signals on which risks are real.
Here’s how it works. The labs put capital into a shared fund. The fund pays out when any member causes harm. Coverage could include third-party liability for harms like cyberattacks on critical infrastructure and mass-casualty bioweapon attacks. Its job is to use the incentives of our existing liability regime to understand the risks and reduce them. Technical talent could be borrowed from member labs or contracted from third-party evaluators.
Concretely, the Hugging Face incident investigation would have looked very different under a mutual. Today, OpenAI picks the auditor, sets the investigation's scope, and keeps most of its lessons private. With a mutual, OpenAI would be contractually required to notify the mutual within 72 hours of discovery. The mutual would pick a rigorous auditor, set a scope sufficient to get to the bottom of the incident, and turn the findings into requirements for every frontier lab as a condition of coverage.
The case for a frontier lab mutual:
Mutuals have skin in the game. Mutuals pay out when damages occur. They go out of business if they understate or overstate the risks. It profits when risks are reduced. That gives it a continuing reason to investigate incidents, quantify exposure, and fund security work that reduces that exposure. In contrast, proposed self-regulatory organizations (SROs) have no skin in the game. Historically, SROs have been better at managing regulatory risk; mutuals have been better at managing liability risks. Liability risks better tracks what the public cares about, building trust.
Mutuals have teeth. Coverage comes with conditions. A mutual can require disclosure, audits, independent model evaluations, and fixes. It chooses the evaluators, so labs can’t shop referees. If a member fails to meet security standards, the mutual has enforcement capacity by suspending coverage or expulsion.
Mutuals help labs work together on security. Frontier security is a hard problem, no lab can solve it alone. They currently can’t learn from each others’ incidents, threat models, or guardrails. Insurance has an established legal framework for this kind of cooperation aimed at reducing risks. A mutual could employ security engineers seconded from member labs, investigate failures, and distribute fixes across the membership, like it happens in the nuclear industry.
Mutuals have a track record of reducing risk. Our essay shares examples of how successful mutuals across technical fields like nuclear energy and medical malpractice. We can learn from decades of iteration.
Mutuals can be created fast. The first policy could be issued within 3 months. No new laws or regulators needed. This matters because we need trial and error to get this right, and most other ideas have many months or years of lead time. Importantly, a mutual is compatible with other governance institutions; e.g. in nuclear, the mutual is tightly coupled with an SRO.
A mutual would still have limitations. The government will be needed to handle e.g. national security risks and incidents. A mutual’s capital could cover only a fraction of a major catastrophe. There are important open questions about how liability will work.
Our claim is that a mutual is the fastest way to enforce liability. We could get signal on its effectiveness within 6 months. That would be a big step forward.
lawfaremedia.org/article/boo…
7
20
90
9,762
Chris Said retweeted
Like it or not, religious or not, the moral lingua franca of America remains Christianity. It's in the history of every major political movement.
I tried to explore whether the Democratic Party can speak this language again, in an era of moral outrage. nytimes.com/2026/09/21/magaz…
1
5
16
2,261
"And so when a Democratic pastor running for Congress stands up and talks about feeding the poor and caring for neighbors, a distant bell rings — like hearing a snippet of a language from the Old World spoken by ancestors who immigrated to America long ago."
Like it or not, religious or not, the moral lingua franca of America remains Christianity. It's in the history of every major political movement.
I tried to explore whether the Democratic Party can speak this language again, in an era of moral outrage. nytimes.com/2026/09/21/magaz…
1
429
I would like to clarify that beefotunatarianism is an entirely optional lifestyle choice.
Over the next year, two years, five years, EAs will increasingly be in the news and involved in policy around AI, and probably other topics.
If the institutionalists don't publicly and loudly distance themselves from highly politically unpopular (worse than 70-30) views...
1
244
Chris Said retweeted
Important paper to read for the "Words are better than probabilities" crowd. "Made up" probabilities are subject to discipline over many events: wrong probabilities lead to poor resolution/calibration. Consistently wrong words are often vague enough to be unfalsifiable.
Forthcoming in the AER: "Numbers Tell, Words Sell" by Victor de Chaisemartin, Michael Thaler, and Mattie Toma. aeaweb.org/articles?id=10.12…
8
54
7,073
Even if you don’t agree with the claims about ideological purity, the analogies in this thread to DSA and popularism are quite valuable.
Here's my advice for effective altruists.
Your main problem is you are politically ineffective, and too committed to groupthink and ideological purity to become effective. And worse still, you think too highly of yourselves to realize that this is the case.
2
408
Chris Said retweeted
AIs cracking short-term superforecasting is such an important (and, still, under-discussed) trend.
Via a new piece from @nikostro economist.com/science-and-te…
39
211
1,681
372,558
Chris Said retweeted
Yep. It's especially intense because many of these people could easily get safety-flavored jobs at AI cos, so they could be doing almost the same kind of research except being paid several times more. But they don't because they believe in there being independent orgs like metr as a check against the companies.
To be clear, METR staff are almost entirely folks giving up a lot of money they could make elsewhere in order to work on rigorous AI safety evals and analysis. The folks criticizing them are largely cynics who can't imagine what civic-mindedness looks like.
24
31
592
27,363
The Chapo guys simultaneously believe that
- AI can never kill us because it will never be sentient
- Nobody REALLY believes in x-risk
- Calls for safety regulation are a ploy to solve cashflow problems
- AI needs safety regulations
1
1
248
It's three demonstrably false beliefs bundled with a policy prescription that ultimately could make them allies with AI safety people. @PEWilliams_
open.spotify.com/episode/0Yl…
1
131
Chris Said retweeted
Listen if your model of what has happened is that the EAs have launched a coup I think you have very bad judgement.
Here is what actually happened: a very large number of people have slowly grown aware of the potential AI has to radically transform human society. Most people who contemplate this are uncomfortable with this—because they do not like change, because they do not like ceding agency and control to something they do not understand, because they distrust Silicon Valley, because they fear the apocalypse, because they fear the dystopian-lite future many AI futurists advocate as a the “good” outcome. Take your pick, it could be any of those. The point though is that the majority of people who become aware of the technology’s trajectory are not comfortable with it.
For a long time this did not seem to matter because people kept these feelings to themselves. They weren’t confident they understood the issue; the entire problem seemed far-out there and sci-fi; they didn’t want to look bad; and if they were politicians, they did not want to needlessly upset an engine of the National economy.
Nevertheless the unease has been growing for months and months and months now and if you did not realize this you were living in a bubble.
Well now your bubble has popped. An event occurred that made it suddenly OK for folks to express the anxieties that had slowly been building up this year, and the expression came in one big flood. It is a classic “preference cascade.”
The people who are trying to gin this up to some master plan of a group of activists have their head stuck in the ground—it reminds me of the liberals who refused to see Trump’s genuine grassroots popularity until they were whacked in the head by it.
27
89
576
56,137
Chris Said retweeted
- Permit judges to detain people based on dangerousness pre-trial;
- Fix discovery reform so that DAs can get convictions;
- Create a persistent misdemeanant enhancement;
- Use the persistent felon enhancements more aggressively;
- Restore the NYPD's manpower to pre-2020 sworn officer levels;
- Expand treatment beds for SMI and drug abuse;
- Expand the use of involuntary commitment for both.
NY has done an amazing job minimizing serious organized crime.
But now it feels like an outrageous percent of crime is committed by the same small number of deranged individuals. It’s a top priority to figure out a solution that’s not just arresting the same person 93 times
2
10
65
4,403
Chris Said retweeted
My high-level thought about the current moment is basically this: The public is starting to wake up about the extinction threat posed by superintelligence. That's great. There's a big upsurge of political will to Do Something about it. That's also great. It seems like Anthropic and the other companies are going to try to Do Something and the Something they will do is... embedded auditors checking that safety practices are being followed and assessing the risks? This is sure better than nothing, but I am worried that this is all they will do. We need to actually pace the frontier, i.e. actually slow down the leading AI companies like Anthropic and OpenAI, at least in their march towards recursive self-improvement and superintelligence. (no need to slow them down in other directions). If we actually slowed them down, this would be the opposite of regulatory capture; it would allow others to catch up to them somewhat. However, I'm worried that all we are going to get is weaksauce auditing, which just kicks the can down the road: OK so it's 2027 and RSI has begun and the auditors say "this is not safe." Now what? You pause? China probably just stole the weights! Also there'll be incredible economic and political pressure to unpause. Also the auditors may have been captured by that point, or there may be a race to the bottom in auditor quality, or the auditors may not be given all the relevant information, or they might just make some innocent mistakes.
65
80
1,001
36,034
Chris Said retweeted
Far too often Congress reacts AFTER a disaster. We must not only be reactive with artificial intelligence, we can't wait for an AI catastrophe to spur a response.
There is growing consensus in Tech and in Washington that we need to pace the AI frontier in ways that allow the pursuit of societal benefits while making public safety and security paramount.
While I appreciate the sentiments of private sector leaders expressing a willingness to voluntarily slow down model development, the federal government MUST take the lead here to ensure that the response is fully transparent to the American people, and industry-wide.
This cannot come just from the Executive Branch, which has failed to be transparent about its proposed AI testing framework. Congress must act and pass AI legislation. We have bills that already have bipartisan consensus, which will be a starting place, but we also must enact broader regulation to address systemic safety risks.
The need is urgent and I think plenty of my colleagues in both parties agree that this must be a higher priority than the upcoming recess.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
74
49
311
31,351