Scott Alexander has demonstrated in his response to Steven Pinker that the smarter you are, the more effectively you can deploy fuzzy language and logical fallacies in the defense of incorrect ideas. There are certain ideas that are so bad that they require truly smart people to come up with plausible sounding incorrect arguments in their favor, and sadly, that is the sort of use that the self-described “rationalists” now apply their intellect to.
When a group has fallen into tribalism or cult behavior, defending the pivotal beliefs of the group becomes the priority, and uncomfortable truths that contradict those pivotal beliefs must be fought with every possible available resource.
If AI is not particularly dangerous, that threatens every part of Scott Alexander’s group; AI risk is the single most important value in their lives. Thus, the most important thing in the world to Scott is to suppress the heresy that AI is not particularly dangerous. The fact that the heresy is true has become something he literally cannot allow himself to notice.