If that idea sounds far-fetched, then please consider Cambridge boffins fear 'Pandora's Unboxing' and RISE of the MACHINES.
Boffins at Cambridge University want to set up a new centre to determine what humankind will do when ultra-intelligent machines like the Terminator or HAL pose "extinction-level" risks to our species.Mike "Mish" Shedlock
A philosopher, a scientist and a software engineer are proposing the creation of a Centre for the Study of Existential Risk (CSER) to analyse the ultimate risks to the future of mankind - including bio- and nanotech, extreme climate change, nuclear war and artificial intelligence.
Apart from the frequent portrayal of evil - or just misguidedly deadly - AI in science fiction, actual real scientists have also theorised that super-intelligent machines could be a danger to the human race.
Jaan Tallinn, the former software engineer who was one of the founders of Skype, has campaigned for serious discussion of the ethical and safety aspects of artificial general intelligence (AGI).
Tallinn has said that he sometimes feels he is more likely to die from an AI accident than from cancer or heart disease, CSER co-founder and philosopher Huw Price said.
Humankind's progress is now marked less by evolutionary processes and more by technological progress, which allows people to live longer, accomplish tasks more quickly and destroy more or less at will.
Both Price and Tallinn said they believe the rising curve of computing complexity will eventually lead to AGI, and that the critical turning point after that will come when the AGI is able to write the computer programs and create the tech to develop its own offspring.
"We need to take seriously the possibility that there might be a ‘Pandora’s box’ moment with AGI that, if missed, could be disastrous. With so much at stake, we need to do a better job of understanding the risks of potentially catastrophic technologies.” said Price.
http://globaleconomicanalysis.blogspot.com