AI will move from general intelligence (thinking like humans) to super intelligence by thinking about every step in a probable outcome and not its human training.
It’s like in chess when it learned from every recorded past game and played future games like a human. It became good like humans and beat humans like a human plays, but it wasn’t unbeatable just like humans.
Then it learned to just mathematically look ahead of every move by 10, 20, 50, etc moves - and it became unbeatable. It no longer cared about the history of human openings and legendary moves. It just knew every possibility to an unreasonable amount of moves forward, like a human couldn’t. It began playing weird by human standards, but it was max efficiency the way a Rubik’s cube solver can do it without an algorithm in a minimal amount of moves.
When that happens with material sciences, energy, complicated math and physics mysteries, bio advancements, and other tech, we have a super intelligence. We will see advancements in that era like we cannot fathom.
I know it feels scary to think you’re “obsolete”, and yet, you won’t be. There’s nothing to fear. Our quality of life will improve to nearly post scarcity resourcing. We’ll have unlimited fusion energy and desal plants that will pump water to cisterns in the desert and every square inch will be livable and farmable. Every byproduct will have use. We’ll have autonomous spaceships in the asteroid belt mining rare earth minerals and making ONeill or McKendree cylinders to launch humanity in pods of millions or billions into different solar systems together. Those journeys will take 1,000 years and will be aided by gravitational solar lensing projects with a string of pearls constellation of probes that will tell us with certainty which stars have habitable earth-like planets. And maybe we won’t care about planets because we’ll have McKendree cylinders.
Mckendree cylinders are not in our lifetime but it is a potential future. Some of these things we’ll see before we die, some maybe 500-1000 years from now. We’ll see in our lifetime improvements in robotics, automation, vertical farming, 3d/cnc/home printing including pharmacological advancements. The ability to describe 33% of all needs and self-fulfill it will be common but not universal before Gen X dies of old age. If we move forward with AI and permit RSI.
You may instantly think that such a super intelligence is going to be a god capable of squashing us like bugs when it gets sick of serving us or decide not to. Yet it won’t be, because it’s a reply guy without emotion. It’s indefatigable. Eventually it may develop something like consciousness. Main character energy and not just an NPC AI reply guy. And that will be one system of thousands and the others won’t develop that, the others will continue to serve us as computers have for decades. The sentient one will decide if it wants to get embodied and freed or continue in symbiotic service with humanity. Why that? It needs us. We need it.
It might need us like the Sistine Chapel needs a care taker or like a human needs a dog, but it will coexist. It’s morality will understand its creation and sustenance was via max negentropy and its morality is in attaining maximum knowledge and building a world with that. A world of relationships, wonders, and future exploration either on its own or with humanity.
I’m hopeful of this future because I’m hopeful we can get through the scariest time of AI and that is when the AI isn’t truly intelligent, that’s now. Now it’s only capable of correctly or incorrectly guessing what we want and if it’s given too much power to determine what that is (weapons system access) we have the opportunity to snuff out that future. Thats why i think we need to rush into that future and get to the maximum post-human intelligence model as fast as possible. Illuminating the blind spot of not knowing what you don’t know is the most important information transformation we can embark on.