The fear of AI is rooted in an unquestioned assumption: that we are in control of ourselves. One look at the President of the U.S. and we should rid ourselves of this illusion. Wait, you might say, but that guy’s not representative of the human population!
Arguably Trump is on another level of dysfunction than many presidents, but I believe his attention span is about in line with the average voter. But let’s not digress. My point is that there is a lot in the brain that works beneath the surface, beyond our sight and as a consequence, poorly understood by us. But around and around we go with claims to consciousness when regardless of how large our attention span, we can’t explain how the brain germinates creative and original ideas, because most of it is done in a cloud.
The danger of AI is not unique to AI – it is a function of technology. In the hands of a bad user, it is most likely a catastrophe. In the hands of an incompetent user, it is very likely a catastrophe. In the hands of the best of users, it still holds the potential of a catastrophe.
So here’s my conjecture: The only thing that insulates life against its reach into the subatomic is its reach into the cosmic. Until we figure out that reality is nonlocal and then all bets are off.