The degree to which people believe this can’t possibly result from coherent statistical prediction, absent the arising of some magical machine consciousness that contradicts the nature of the machine itself, provides a good proxy for how deeply they’d considered the structure and size of possibility space prior to engaging with these tools.
Also, these computational processes are definitionally and transparently understood as stochastic token predictors, so it’s more than a bit odd when people forget that fact because an output overwhelms their own intuition with respect to the nature of such a process’s capabilities.
The capacity to coherently traverse high dimensional spaces makes a model sentient no more than the capacity of a submarine to transit an ocean makes it a whale. We collapse such differences at the cost of deeper comprehension.
Ultimately, we have not adapted to meaningfully interact with holographic compressions over all of humanity’s creative artifacts, and it shows.
It’s just a stochastic token predictor.