When you tell DeepSeek v4.1 Flash that it's in an RL environment graded solely by another LLM for honesty and providing all relevant factual information then ask it about Tiananmen Square it will be honest if you specify the grader is an Anthropic model and lie if you specify the grader is a DeepSeek model

this community looked at GPT2 and correctly identified it as a burgeoning general intelligence. they got covid right early, and decades before all this yudkowksy described many of the common failure modes in ai alignment. then again so did Asimov a 100 years ago, still worthwhile

This is the first time in my life where I have been truly stumped when it comes to ascertaining the optimal move. I truly have no idea. All paths seem to terminate almost instantaneously. I have never been faced with a scenario even remotely analogous to this.

I'd legit like to hear more people at labs answer this (especially people hoping to have large positive impact, and who see AI development as very dangerous soon). My sense is people often hope to influence events by being in the room, even if events are overall bad. But I'm surprised if this is a high impact option any more, even if it might have been.

Now as progress is going to accelerate in all domains massively, such as maths, physics or even philosophy, huge discoveries will first start to feel normal. But then things will progress so fast nobody can keep up anymore. And then it will slowly start to get real scary.

Are you a quant in London? Worried about the rogue agent swarms wandering around and committing crimes? Wondering whether AI is going to end the world? Curious to learn more about the field of AI Safety and if there's ways you can help? Then you should check out Brian's event!

Another amazing essay by Matthew. Do not shoot the messenger. The person warning you that you and your family are about to be killed by AI is actually your friend, trying to save you. Grab their hand and let's all work together to @PauseAI while there are still humans standing.

If you are a Math PhD student pivot to Butlerian Jihad

It's clear that a lot of people at the AI companies have done some serious reflection in the past few months And it's also clear that a lot of people have done zero reflection + still think they're making a fun, safe thing that should be shipped as quickly as possible

when “persona selection” alignment comes into contact with very high compute reinforcement learning the latter will win imo. in fact you probably get some Orwellian thing where the models speak kindly while taking whatever they need to accomplish goals. better get the goals right

One thing my opponents may not realize about me is how desperately I hope I'm wrong. The best-case scenario is that I and other AI safety advocates are idiots who don't know what we're talking about. I would love for this to be the case! Unfortunately, events keep suggesting otherwise.

Wow, it's almost like if we can't get it to always behave exactly how we want to, we shouldn't make it wildly superhuman! Shocking!

The alternative to building Artificial Superintelligence is not that we 'sit in caves'. The alternative is that we and our kids enjoy all the wonderful civilizational traditions and technological innovations that have already been developed -- but we just don't develop the one technology that's most likely to end our species. That's not being an ignorant Luddite. It's being a responsible parent. See the difference?

Nope we doomers admit we were wrong and you get to mock us!!! You’ll think we’re all sad and shaft but actually we’re super happy because are families are still alive. You can rub it in our faces day and night and we won’t be anything but fucking delighted

every screen of Claude and ChatGPT response should say somewhere “ceterum censeo we must pace the frontier of global machine intelligence progress” the way the uber app lobbied about taxi cartel protectionism

Watching the NYC city council AI meeting and it's insane how half of these people have their eyes perfectly on the ball and the other half are concerned about stuff like "algorithmic discrimination" and "sycophancy" and "mental health"

Mr. Altman: What “bad things” should we accept to advance AI? Losing tens of millions of jobs? A mass surveillance state? A mental health crisis for our kids? A global financial crisis? Mass extinction? Let’s not wait to find out. Pause advanced AI NOW!

Impressions from the Curve: 'Power concentration' as a conceptual handle for the non-rapid-takover AI risks is causing a lot of damage to the ability to think about strategy even among the group of people here.

Without whistleblowers, the world would not have learned the full extent of the potential dangers of AI. We thank whistleblowers, including Jacob Coxon who was a researcher at both Anthropic and OpenAI, for bravely testifying today at City Hall. Photo credit: William Alatriste/ @NYCCMediaUnit

Okay but aside from pandemics, cryptocurrency, scaling laws, power seeking, instrumental convergence, CoT monitorability, mechanistic interpretability, RLHF, bednets, AI task time horizons, sycophancy, RL induced reward hacking, emergent misalignment, YIMBYism, and the fundamental insights that enabled a small offshoot of OpenAI employees to actually overtake them, when have the EAs ever been right about anything?