A ranking of takes on embedded third party evaluation this past week, from worst to best: [contentless sneer against outgroup] METR bad because [huge Sankey diagram]. fact checking? timelines of relative investments that make causal sense? determining which numbers are large or small fractions of other numbers? sounds like some EA bullshit to me. just look at this diagram with all those curved lines, you can SEE the nest of snakes. METR bad because [an actual specific chain of actors who have some pairwise relationship to each other that ends with someone it would be bad for them to have strong COIs with, e.g. "METR once received funding from an org who once received funding from a person who once gave funding to a now-frontier AI company"]. It is beneath my dignity to explain how this actually affects the decisionmaking of METR; you should vaguely perform a mood affiliation and keep scrolling. Don't think too hard. guys guys guys you HAVE to embed my company/organization into the labs. I have been an unwavering supporter of external embedded auditors since 7 minutes ago when I saw this essay taking off. Look at this thing we published once that sort of looks like AI safety if you squint! Fear not, I can assure you that I have never done anything altruistic in my life and if I had I would have been ineffective at it. I observe that [politicized actor] has had [bad take]. Let me use this correct observation to score points for my side and politicize the situation further. === zero point: takes beyond this line are better than logging off === Dear [person with insane take], here is an earnest explanation of why you are wrong. METR is bad/problematic because [an actually reasonable concern, like greater cultural overlap with Anthropic than OAI or the pressure for individual employees to be on good enough terms with labs that they could later get hired], with no further suggestions for what to do about this issue. Tweets which simultaneously acknowledge that (1) more social independence from labs would be good and (2) almost everyone competent is socially connected to labs, even if they don't propose solutions. [literally any take that engages with object level assessment of AI companies done by a third party org like METR, SecureBio, Guidelight, Redwood, Nightingale, etc] === current discourse frontier: takes beyond this line are better than anyone has yet posted === I have [actually reasonable concern] with METR/Redwood/etc. To address this concern, we should do [concrete proposal that makes any sense and would solve the problem]. I am founding a new third party evaluation org / pivoting my existing org to do more third party evaluation. We're doing [thing which is both (1) not identical to METR (2) remotely useful for assessing AI companies]. Our initial work will be on [concrete specific thing that would help]. METR has problem X. We should instead use preexisting organization Y, which does not have problem X and has comparable expertise at assessing loss-of-control risks and misalignment incidents inside AI companies, for instance [past work they've done of similar quality]. (this would be an amazing take but it is impossible to post because no such org exists)

If you are good at investigating the behavior of powerful AI models, today is a great day to apply to METR: metr.work

I think there's a ton of informative stuff in this post! Highly recommend reading.

Anthropic's second Risk Report is out! I'm pretty happy with a lot of the new things we landed in this one and think it's a big improvement over our first one in a bunch of ways. Among the novel features: * Coverage of internal models * A much more structured argument around alignment risk * Claude-powered review of the alignment section * Discussion of a wide variety of incidents, safety process failures, and ways Anthropic has fallen short (see 4.8.2, 6.5, and especially 5.2) * An attempt to get somewhat quantitative about threat modeling for CB-2 risk despite enormous uncertainty * Much more detailed discussion of jailbreak risks * Two different cases of increasing assessed risk levels from "very low" to "low" on the basis of general increased uncertainty given worries about the overall degree of process assurance * Early notes on some cool new model organism work that informs the alignment assessment and more! Excited to hear thoughts on how the next one could be better.

+1. I think working at METR would have been better than the frontier AI company work I was doing a year ago. The work I'm doing now is a sufficiently good fit for me in particular that I thiiiink it looks higher impact, but I think it's plausible I'm still making a mistake!