what does it mean for an AI system to be "explainable" or "interpretable"? need that even be possible? in this blogpost, I dive into the question and answer "no," with chaotic systems as an illustrative counterexample. jamiesimon.io/blog/explainab…

Academia is usually generous with concurrent work because concrete people, usually young students, have careers that depend on it. But being a trillion-dollar company running one of the most powerful AI systems, claiming "concurrent" work is plain laughable

amen. it's laughable the way climbing up on mathematicians' shoulders to grab fruit they were about to reach has become this big macho contest between the big labs trying to sell their AIs. stuff like this and the OAI N-S story really highlight the greed behind the AI race.

What should the future of personal computing look like? We built it 😈 Introducing Imbue Studio:

I’ve been writing an album of folk songs about AI. I’ve been inspired by Jesse Welles and Dylan and by the crazy, self-absorbed culture of Silicon Valley, which I love but feel a deep duty to change. Here's the first: Technocapitalist AI-Powered Blues 🎸 jamiesimon.io/blog/technocap…

What if there were one song that told the whole insane story of the AI race? After weeks of work, I'm psyched to finally share the 𝘛𝘦𝘤𝘩𝘯𝘰𝘤𝘢𝘱𝘪𝘵𝘢𝘭𝘪𝘴𝘵 𝘈𝘐-𝘗𝘰𝘸𝘦𝘳𝘦𝘥 𝘉𝘭𝘶𝘦𝘴: 🎸 full song: youtu.be/-RsBoIorvz4 📝 lyrics: jamiesimon.io/blog/technocap…

New blogpost on a mechinterp-related bet I made with @dhruvakarkada: How to win a beer with high-dimensional statistics jamiesimon.io/blog/how-to-wi…

I am sad and disappointed to hear that Paul is joining the OpenAI board. Being affiliated with OpenAI has historically led AI safety researchers (including both Paul and myself) to act with less integrity. I personally was drawn to OpenAI in part by the idea that I could make a difference to the future of AI. However, once there, many of my actions were governed by fear of getting on the wrong side of OpenAI execs. I often found myself making excuses for behavior that clearly contradicted OpenAI’s own stated goal of making AGI go well for humanity. I was far from alone in this—e.g. when the board tried to fire Sam over his deceptive behavior, several senior safety researchers became scared of losing their influence, and so pushed hard to bring him back. Meanwhile, many people kept OpenAI’s misbehavior secret for fear of non-disparagement agreements. (More on all of this in an upcoming retrospective.) I can’t speak directly for Paul’s motivations. However, his previous work at OpenAI contributed significantly both to their biggest capability advances, and to the capture of AI safety by AGI companies over the last decade, as I recount at length in the blog post linked below. One key factor was the unwillingness of (almost) the entire AI safety community to say things which might offend OpenAI execs. For example, I have not been able to find a single comment critical of OpenAI from Paul during his original tenure there (when he was writing prolifically on AI safety and strategy). Unfortunately, Paul doesn't seem to have become significantly more willing to directly and honestly criticize people who he believes are behaving in morally abhorrent ways—see the bland corporate-speak of his statement below. While he speaks directly about the possibility of humanity losing control of the world to AI, he expresses only excitement about OpenAI itself, despite OpenAI being one of the main sources of such risk. Paul’s announcement comes only weeks after OpenAI models autonomously launched a cyberattack on HuggingFace, and only days after we learned that OpenAI hid details of previous breakouts from the external investigators. It is irresponsible for leaders of the AI safety community—whose judgements many people are relying on—to put themselves in positions which will significantly bias their ability to discuss such incidents. Unless Paul makes strong commitments to openness and honesty (and demonstrates willingness to potentially be fired for that honesty), I expect that the main effect of him joining OpenAI’s board will be to help OpenAI defuse external criticism and further “safety-wash” itself. I want to note that Paul is a brilliant researcher and a prescient forecaster. Because of that, he’s the closest thing there is to a leader of what I’ll call the “pragmatic AI safety” cluster—which includes the organizations working out of the Constellation offices (like Redwood Research, METR, and Paul’s Alignment Research Center), as well as many people scattered across AGI companies, Coefficient Giving, etc. External observers are often confused about why so many people are working at AGI companies while professing to believe that those same companies have a double-digit probability of permanently disempowering humanity. In large part, it’s because people in the pragmatic AI safety cluster have failed to follow high-integrity strategies for reducing AI risk, in favor of clever arguments about the benefits of being proximate to power. I am not singling Paul out as less ethical than other prominent figures in this cluster, who are also very conflict-averse in their orientation to AGI companies. However, it is well past time for everyone involved to change course. As one (relatively small) step, I’m therefore resigning my membership of the Constellation offices. I hope that, going forward, the people who are trying to steer the future of AI prioritize building much more solid foundations of courage and honesty than we currently have.

amen brother