Penrose's argument, in a nutshell, is that since humans can understand Goedel's theorem, consciousness is uncomputable and physics incomplete. I've always found it quite charming, the idea that understanding Goedel's theorem is an indicator for consciousness, seeing how few people actually understand Goedel's theorem.
In any case, the argument from Goedel's theorem is why, I believe, Penrose thinks that the measurement process in quantum mechanics is relevant for human consciousness. Because it's the only place in the foundations where you can fumble in some new physics that is plausibly relevant in the human brain. Hence the story with the microtubules. And if you believe that, then AI can't be conscious because it's algorithmic. Though presumably a quantum computer could be conscious, if it emuluates the quantum process that is supposedly relevant for consciousness.
(Penrose's model for wavefunction collapse by gravity is totally computable and actually does not fit that narrative.)
Also, I believe Penrose's argument from Goedel's theorem to be technically wrong. It is possible to algorithmically understand (and even prove) Goedel's theorem without violating Goedel's theorem because the theorem itself is a meta-statement about the Goedel sentence(s) and not the sentence itself. In some sense, Penrose's argument is wrong for the same reason Goedel's theorem exists in the first place, that you can make meta-statements about statements. In fact there have been computer-generated symbolic proofs of Goedel's theorem long before the AI boom.
It's not that I have an obsession with Penrose in particular, it just so happens that my interests have a big overlap with his. I have an interview with Penrose in my 2022 book "Existential Physics" on these and related problems.
nitter.net/realBigBrainAI/status/…