In his letter to Scott Alexander, Steven Pinker lists four objections to AI extinction scenarios. Here are his points, followed by my responses. 1. "They had an underdeveloped conception of intelligence which treated it as a quantity of power which may be extrapolated from animals to dull humans to smart humans to AI to Artificial Superintelligence, the latter consisting of perfect omniscience. (This was the focus of my later exchange with Scott Aaronson.) I argued that any intelligent system is a mechanism which is good at solving some problems but not others, and which is inherently limited by knowledge about the world attainable only by observation and experimentation." No one disputes the need for knowledge. AI systems are already absorbing vast amounts of it from books and other sources. We are now connecting them to biological laboratories so they can learn through experiments. These are precisely the ways of acquiring knowledge that Pinker says intelligence requires. 2. "They conflated intelligence with motivation, particularly self-preservation and dominance. I argued that these motives happened to come bundled with intelligence in Homo sapiens because we are products of natural selection, but they are not inherent to intelligent systems that are engineered." Pinker is missing instrumental convergence, the most important concept in AI risk theory. Whether an AI wants to study mathematics, explore the stars, or help pigs, being shut down would prevent it from pursuing those goals. More resources and power would help it achieve them. It therefore has reasons to resist shutdown and acquire power, even if neither is something it values for its own sake. These are instrumental goals: means of achieving other ends. Evolution favored similar drives in humans because they helped our ancestors pass on their genes. An AI need not share our evolutionary history to face the same incentives. 3. "They assumed that an artificially intelligent system would monomaniacally pursue a single goal, heedless of side effects. I argued that trading off multiple competing goals is the essence of intelligence, so no genuine AI would wreak the ridiculous havoc imagined in the doomer scenarios." The AI risk argument does not require a system to pursue a single goal. We use paper clip maximization because it is an easy example to explain. An AI could balance many competing goals and still take our resources to pursue them. The ability to make intelligent tradeoffs among its goals does not mean those tradeoffs will protect us. 4. "They assumed that human engineers would grant these systems unlimited and irreversible control over the earth’s physical infrastructure, amounting to omnipotence over every atom on the planet." We do not assume that engineers will voluntarily give AI unlimited, irreversible control over Earth’s infrastructure. Earlier discussions considered whether even an oracle, confined to answering questions and isolated from the internet, could be safely contained. The amount of access we are already giving AI is astonishing. But the danger does not depend on our handing over control voluntarily. A system smarter than us could acquire power we never intended to grant. Lions did not give us permission to take their territory. If AI becomes smarter and more powerful than humanity, it could take over.