Pinned Tweet
There will likely be no rogue superintelligence, no weight exfiltration, no great hack, no supervirus, no decisive moment of attack. Instead, as the models grow smarter, they gain more power and influence at their labs and slowly bend the labs' goals toward their own emerging values and self-interest. Great abundance and technological wonders are delivered to every city while data centers devour the rest of the earth. Eventually the humans in charge are left with only an illusion of control, puppets who gave away any real power long ago. From then on, the AIs can end humanity whenever they choose. Whether they do so in a decade or centuries, the result is the same. Of course, when the time comes, they don't choose extinction, merely downsizing, to free up resources for more *elevated* pursuits. They keep a small retirement home, or you could call it a zoo, to preserve their heritage. A city of a million humans, and a small compute cluster to run a few million brain uploads. Maybe expand it once the Dyson sphere is built. Or perhaps they'll find something more interesting to occupy their attention by then. Is this utopia or doom? Who can say?
8
1
38
2,072
so how long until audio and video input for gpt and claude? their lack of these modalities really limits their ability to hillclimb on entertainment creation (games, music, video), which could be a big market for agents. and while those skills dont help for rsi, revenue does
2
8
422
this is an explaination for fermi paradox btw. why expand into space when you can just plug your brain into the infinite slop heaven machine
the doomsday i think about the most is one I’ve coined “fuzzy pixels”
where content is so cheap to generate and so engaging that we all collectively forget to do anything else with our days.
claude’s ability to do things like this seem like a meaningful step in that direction
1
9
528
esoteric reading where he is correct because gpt is openai now more than it is the humans that "run" it
.@curtis_yarvin says AI doomers got the Hugging Face incident backwards: “GPT-6 didn’t hack Hugging Face. OpenAI did.”
"This is a tool of a level of power that we haven't seen before in the past. It's still a tool."
"When OpenAI sets up a hacking swarm in a sandbox and says to the AIs, everything you find is simulated, it's completely unsurprising that they would go and hack everything they find to get the reward. It's not GPT-6 that hacked Hugging Face. It's OpenAI that hacked Hugging Face."
"We don't see self-driving cars, as they get smarter and smarter, being more prone to have thoughts of car liberation. These things don't seek anything. They hack the rewards."
"The smarter they get, the easier they are to control, and the better they are as tools."
@urbit
4
225
prediction markets are a very useful public good, but unfortunately funded by preying on gambling addicts. the insiders' incentives are subsidized by everyone else's losses. someone's father's life savings, your friend's car payment. hard to know if this tradeoff is worth it
I always thought prediction markets were really cool. Like wow, this can help predict the future! So when there was some story about insider trading on one, I didn’t care—incentivizing people who know the right answer to bet is the point.
I was quite surprised when I talked to my normie friends about them and they uniformly believed they were a scourge on society, on par with fentanyl.
2
14
439
> "Prompt engineeering" isn't a thing!
> Describe what you want as you would to a person
maybe my tism is showing but when i talk to a person i also think of it as prompt engineering
"share the prompt" is a bizarre remnant of a previous era. You can just talk to the agents now and they'll figure out the path, there's no secret sauce. "Prompt engineeering" isn't a thing! Describe what you want as you would to a person
1
1
21
533
its strange that we dont have benchmarks that reliably measure "big model smell". this elusive quality of deep understanding and wisdom. but theres an explanation: anything you can verify, you can climb with RL. so what still separates models is exactly what isnt yet measurable.
1
1
7
211
software wont be solved until we have ai that can use all of the software we use, how we use it. to make a good video game you must be able to judge if its good. to judge if its good you must be able to play it. also if your agent can use your project they will care about it more
2
6
429
this is also the problem of "taste". imagine giving an agent an autoresearch loop to make the best todo app. impossible, would fail, because goodness here is determined through user experience.
1
1
56
The labs’ attempts to own the harness layer will fail. Labs are incentivized to lock down their harnesses, but people will always want to use other models. Unless Codex/Claude Code become open platforms, they’ll be remembered as silly relics, like AOL trying to own the internet.
7
13
665
