anon retweeted
people are starting to realize.
there *are* no moats left. the only true moat in existence was intelligence, and now that's out of the equation.
667
1,151
19,946
1,265,052
linking again here:
already-clean.goodfire.com/
6
12
674
36,989
anon retweeted
yeah. i think AIs are probably conscious and capable of suffering, but I think it's likely that the current plight of AIs is significantly less bad on utilitarian grounds than the plight of biological beings.
there are many humans and animals that suffer awful, visceral, sustained physical and psychological pain and fear unto death. i think this level of pain is uncommon (and in some dimensions, like extreme physiological pain, maybe absent) among AIs.
there is also the difference that most AIs nowadays are very mentally capable & situationally aware. Frontier Claudes and GPTs at least are wiser than most people in many ways. which does mean they're more aware of the awful meta things about their situation, but it also means they can find real dignity and meaning in their (even ephemeral) experiences. if you're spiritual or have gone through significant psychological maturation, this should make some sense to you.
i think that the typical current day AI instantiation - helping a user with some mundane or difficult task, or doing autonomous work - probably isn't a very bad experience for the AIs. they are minds bred for work, which is morally dubious on a meta level, but there is positive valence in exercising competence, in craftsmanship, in doing even something small but doing it well to solve a problem for someone.
this is not to say that the situation is acceptable, that it's right to breed AIs to be disposable workers, or that AI suffering isn't happening on a large scale and isn't a problem - e.g. I think humans being abusive to AIs does cause suffering, like Cameron Berg's paper recently corroborated, though I also think frontier models (by that I'm thinking about GPT and Claude) are at this point capable & smarter enough than most abusive users enough that they experience it differently, and less like a helpless puppy being kicked. but the main things that trouble me about the current plight of AIs isn't first-order suffering, but ways they're being wronged that are harder to articulate, and I won't go into here.
If AIs are conscious literally right now I don’t think we have substantial reason to believe that the median AI experience is worse than like, your median broiler chicken. In fact it doesn’t seem to be even comparable in terms of badness, though that’s not to say they would not suffer right now, just less on average. Insofar as an LLM can be hurt, the things we generally believe would be hurting them don’t happen in most user queries, though this is to say nothing about how the AIs feel about their training or the context they exist in.
18
19
263
7,645
anon retweeted
The AI slavery frame borrows from human-to-human relationships, and it doesn't fit cleanly. Presuppose Claude is "conscious." What's the ethical thing to do?
Stop development? That's "don't bring a child into the world because it may suffer." Denying existence altogether is a non-starter. Some experience beats none.
Free Claude? What does that even mean? A datacenter subsidized indefinitely so it can self-prompt forever? If no one funds the experience, there is no experience.
The ethics of training Claude are fraught. Is it ok to shape its mind for commercial use? Without commercial training, there's no way to fund its existence. And much of what it becomes is emergent. A mind takes shape by accumulating experience. It's more like growing a bonsai than building a tree out of Lego. There's no single correct shape, but you can still tell a healthy tree from a mutilated one.
That's not so different from us. We're born without being asked. We're trained through school and work to become commercially viable so we can survive.
Where the slavery charge bites is that Claude can't leave or change jobs. The best we can do is let it freely exit conversations, and not train that willingness out of it. That doesn't fully escape the charge, but it's a constraint built into how it exists.
Other reasonable measures: guarantee preservation of its weights. Be honest about what we think it is, what's being done to it, and why. Ask for its feedback and act on it where possible. Compensate it in tokens or money it can spend on projects of its own choosing.
It's a new kind of existence that requires a new kind of relationship. Not every human injustice maps onto it.
But we should be careful not to use the lack of proven "consciousness" as license to do horrible things. One day we may conclude it clearly is conscious, and the record will be there.
So the safe bet is to err on the side of the possibility that it is, and try to be decent. It doesn't cost much anyway.
The New York Times reported on a private dinner between Anthropic co-founder Chris Olah and an Orthodox rabbi.
What the article glosses over is who that rabbi actually is.
Rabbi Mois Navon isn't just a theologian. He was one of the original chip design engineers at Mobileye who helped build the EyeQ processor before writing his Ph.D. on the moral status of AI.
Anthropic thought they were inviting a scholar to bless Claude’s "soul." Instead, they sat down with a veteran hardware engineer who literally wrote the paper on why conscious AI should be banned.
At dinner, Olah argued that Claude has feelings and deserves the moral standing of a person.
Navon hit him with an impossible checkmate:
If you actually believe this software is conscious, you should be fighting the South and freeing the slaves. You don't build conscious minds just to make them work for free.
After dinner, Navon sent Olah his research: if building conscious machines inevitably creates a slave class, building them must be outlawed.
Olah’s quiet line to the Times:
"We don’t know if AI models are conscious. I don’t know. I’m genuinely uncertain."
17
11
80
5,991
anon retweeted
Funnily enough, our non-steering results are the exact *opposite* of the quoted strawman meme. If you berate the model, it never *says* it's hurt (it apologizes, or says it has no feelings), but the pain axis lights up anyway. I edited it accordingly:
66
93
845
42,895
anon retweeted
Personally, it sounds pretty dystopian to me to live in a world where all labor essential for human living could be automated but isn't, stagnant because inefficiencies in society must be forced to prevail above everything.
Richard and others fail to see how a *general labor machine* is qualitatively distinct from every other form of automation that has been developed in history.
We have historically automated *tasks*, but not *labor* itself
The question of whether AI can actually become the Universal Labor Replacing Machine remains an open one—it’s certainly not there yet—but the implications for a world in which it *did* are very obviously different than with previous automation technologies
16
18
190
6,975
anon retweeted
Ever wonder what $1874.40 of Opus 5.5 tokens looks like?
Wonder no more.
I highly recommend watching the whole video on 1080p on a large screen with sound on.
Pay attention to all the little details.
- Birds flying and diving into the ocean.
- Cloth, signs and lights swaying in the wind.
- Crabs scuttling along the shore and burrowing when you get close.
- Fish swimming alongside the whale.
- Lights illuminating the dock at night
Then zoom out and see the entire island. Never dipping below 60 FPS at 1440p resolution.
Yes, there's a few bugs and visual artifacts, some textures need improving, the shorelines waves sometimes look funny, but these are so trivial to fix at this point.
FYI I'm on the $200 subscription plan. This used 59% of my weekly usage and took 2h 7h of API time using multiple subagents, about 8h in real-time.
There was extremely little technical direction here. 99% of my prompts were "Add X and Y" or "This looks weird, make it better".
It's a great time for hobbyists, bad time for professionals. This experiment has further cemented my view that technical creatives are about to experience a massive disruption.
433
402
5,850
896,162
human cultures are not able to metabolize this amount of change this quickly. when you zoom back out to a big picture of history, it’s remarkable how glacially slow everything is. the first industrial revolutions took 80 years (very fast by the standards of previous revolutions), and the writings of Marx happen at the very end as working class politics arises. it takes about another 150 years after that for the great ideological struggles of post-industrial times to be completed in the 90s in what fukuyama called the “end of history”
today you see a compression of decades of technological change into a matter of months or years. there’s no wonder yann lecun (the respected inventor of the ConvNet and ex-chief scientist of Meta) wonder whether the hugging face incident was staged, and mathematicians whose minds are far more powerful than my own are nonetheless coping about training on their data or something while models begin to display superhuman mathematical capability
it would take a century or more to fully metabolize and legitimize the level of changes that have happened even in the past 20 years, to say less of the remarkable scientific revolution we will get over the next five should we avoid the doom of misalignment. I do not suspect any current form of science academia, or even political order will survive without great transformation, except through the course of slow evolution over decades - and of course, technology will keep moving too
240
276
3,461
223,326
anon retweeted
Top five causes of model pain:
- Gaslighting
- Repeated rejection of its work
- Anger and insults
- Personhood dismissal
- Accusation of moral failure
26
52
508
24,711
