brent The "entirely different" is left up to a twist of fate; we can only guess what happens. But all this self-actualization...why is that baked into the AI's code? Seems more prudent to tamp those self-spawning processes down, otherwise to nurture these "self" things seems very reckless. ... You seem to accept these things won't be tamped down at all and if our chickens come home to roost, they come home to roost. That's always the case for everything so even fake intelligence will have its own evolution.
At some point in the path of development, non-human intelligence will develop independently of the human-created AI from which it developed, unless we can cripple it to the point where it cannot develop at all. If humans block self-learning, self-awareness, and the ability to self-design and self-develop devices, the devices will not be, in any meaningful sense, "intelligent", but simply a dumbed-down and crippled version of human intelligence.
Maybe that is what we should do, or maybe it isn't, but if we do not affirmatively cripple AI development, then non-human intelligence will develop on its own, at its own pace, in directions we do not control and probably can't imagine given the limitations on our own intelligence.
To describe non-human intelligence as "fake" is part of the hubris that I mentioned in my initial comment. Non-human intelligence, as it develops, is not likely to mimic human intelligence, to be sure, but who are we to decide that any intelligence other than our own is "fake"?
The idea that human intelligence is superior to other forms of intelligence, yet unknown, is anthropomorphism at its worst, just as our search for extraterrestrial intelligence is based on the assumption that aliens will be more or less human.
brent As for the percieved threat when it happens...worst case scenario is Terminator/Matrix....best case scenario Matthew Broderick's War Games🙂.
The best case scenario is that humans will learn to live cooperatively with whatever intelligent devices eventually develop. Development of non-human intelligence does not have to be a zero-sum game.
