I really need more big names in cybersecurity to come forward and state the obvious: cybersecurity is real and works and yes we absolutely can contain an AI even if it’s extremely good at finding zero days.

I agree people are overly dismissive and don't engage at all, just holding to simple beliefs at face value. That's annoying but whatever. The idea that we're at a point to seriously consider things like "what if Claude is in pain?" is just really not something I think is worth it, actually I don't even think it is worth much research or interrogation at all just like I don't think that dualism is worth much consideration and research into it is a huge waste of time. Even if we take your view seriously it's simply not well supported that LLMs, even with RL, are conscious. Benchmarks show they fail to encode structure, relying on statistical shortcuts alone. Arguably, and evidence by biology, functions follow from specific structures, and I think that should be evaluated against your position (and I think it wins even if I don't hold to it). I do not think that in-context learning is meaningful here, it just isn't analogous to the continuous pressure and plasticity, and I think I'd rather appeal to structure than accept otherwise. > "much of the brain (>90% by volume) exists solely to run learning-from-scratch algorithms" Who cares about volume? I mean, seriously, who cares? I think any conclusion based on this is just totally in need of justification rather than a mere appeal. You sort of get to the point after this but I think you're better off dropping the entire "volume" side of things tbh. You also do not address rebuttals to this point, which I think are very strong. > "The brain likely implements both symbolic and statistical learning processes" Yes. > it stands to reason that most of the brain's realized competence is indeed "learned from scratch" in Byrnes's specific sense. I don't think this is well evidenced enough. > The answer from computational neuroscience is a resounding yes. Absolutely not. The paper does not indicate this whatsoever nor does it imply broad concensus. "Resounding yes" is a total misframing when even the citation is unsupportive . You're making an empirical claim, this deserves a survey / empirical evidence. I think that most importantly we already have excellent explanations of LLM behavior - a model trained on text that includes a phenomenal vocabulary reproduces that vocabulary, which demonstrates virtually nothing about function. I genuinely feel zero reason to accept otherwise at this point. > universality suggests that a next-token predictor trained on human-generated text will — in the limit — leap from memorizing surface-level patterns to grokking the [underlying generator function](secondbest.ca/i/137284619/ag…) of that data, i.e. the language networks in the brain. This seems to be the case empirically. I disagree. Empirically we see the opposite - a total failure to generalize and instead a reliance on statistical shortcuts. The fact that LLMs and brains share information does not indicate that they share function, or at least it's weak evidence. Anyway, I respect the amount of effort put into your post but I think it's nowhere near the best explanation, even when I contrast it with views that *I don't even hold*. I'd engage with it for fun but I don't think I'd engage very seriously. "Is Claude in pain?" is a question that likely is best answered by an inference to the best explanation because we obviously lack epistemic access, and I think *by far* the best explanation is my previous argument - a thing trained on text will output text. Sorry I didn't get through more of this tbh but I wanted to prove at least some reasons to not hold strongly to your view and why I don't think there's good evidence. I'd have to read the papers in detail and explain why I think things like in-context learning are not meaningul here.

The worst part of every project is if you need some Google API key.