Or it’s like worrying about an arms race toward civilization-ending arsenals after seeing the Trinity test… which… was the correct response.
We don’t know it’s possible to build superintelligences but so far we don’t have a good reason to think we can’t and we have complete certainty that humans will spend immense, immense resources getting as close as they can as fast as they can.
My point is that there is no Trinity test. Trinity showed nuclear devices were real. GPT shows language models are useful and can potentially be part of what might be an AGI but nobody showed how, not even in theory.
You don’t have to build it, just show how it could be super intelligent.
Language models' potential role in AGI is a red herring.
GPT shows that there's enough "there" to draw billions of dollars and thousands of the smartest researchers on the planet into the general problem space of creating intelligent machines. I suppose the closer analog would be to Szilard's chain reaction work which prompted the Manhattan Project.
There might be another AI winter or 4 or 10 or 100 before superintelligence, but they will almost certainly be winters and not ends of the road.
If there is any route to superintelligence and we don't annihilate ourselves by some other method first, we will get there. I don't get what the conceptual challenge with superintelligence is. Do you believe that the brain that happens to exist inside the naked apes of Earth is the absolute pinnacle of possible intelligence?
> I don't get what the conceptual challenge with superintelligence is. Do you believe that the brain that happens to exist inside the naked apes of Earth is the absolute pinnacle of possible intelligence?
Well, first of all I do not believe that, but now you mentioned it I assume you do not believe we are the pinnacle of intelligence. Any proof? Because anybody can present literally all of Earth's life as proof that we are.
It is completely unknown to us if and how any great improvements are possible. This quest shares some aspect of the quest for extraterrestial intelligence in that way. It should be out there, because it's hard to think of reasons it can't be, but yet the skies are empty. They could be on our doorstep or the universe could be devoid of intelligent life and anything in between. There's literally no way to tell at this point.
I guess what I'm saying is that it's a bit early to starting beating drums.
We don’t know it’s possible to build superintelligences but so far we don’t have a good reason to think we can’t and we have complete certainty that humans will spend immense, immense resources getting as close as they can as fast as they can.
Very different from the lasers.