I don't think people realize just how much AI's latest breakthroughs in math have shocked mathematicians, myself included.
The main thing that leaves me and many mathematicians shocked is what AI is solving:
Early on – till about 4 months ago – AI was good at:
(i) solving problems that humans already solved, like Math Olympiad problems
(ii) finding out that a problem in field A was already solved by an obscure work in field B, using a different terminology (or language).
But starting with the shocking work on the Erdos Unit Distance Problem, and up till the recent advance on Navier–Stokes, a totally different AI is seen.
It is innovating.
It's bringing a new set of tools to the problem, and then proceeding to using those tools very delicately and accurately to achieve the result.
It's a bit like how a brain surgeon delicately operates on the exact region that needs fixing.
This is absolutely shocking and unexpected. It raises the question of what exactly constitutes mathematical intuition and brilliance. Many a mathematician have written about it (I love Henri Poincare's treatise "Mathematical Creation"). But we don't understand how mathematicians reach their breakthroughs.
Now, it's the same with AI, we don't understand how it goes about reaching these proofs but the end result is objectively new, and powerful.
This creativity, insight, ingenuity, is what's shocking to me.