Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
THE ARTILECT WAR AND THE PROSPECT OF HUMAN EXTINCTION A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines Prof. Dr. Hugo de Garis Associate Professor of Computer Science and Theoretical Physics, Utah State University, Logan, Utah, 84322-4205, USA. [email protected], http://www.cs.usu.edu/~degaris Abstract This article argues the case that there is a non negligible possibility that humanity may become extinct before the end of the century due to the building of “artilects” (artificial intellects, i.e. godlike massively intelligent machines), with capacities trillions of trillions of times superior to human levels, who may consider human beings so inferior that they may choose to wipe us out as a pest. 21st century technologies, such as nanotechnology, one bit per atom memory storage, reversible computing, heatless computing, 3D circuits, quantum computing, self assembly, etc will allow humanity to build artilects, thus forcing the issue of “species dominance” to be hotly debated in the coming decades. This article argues that humanity will split into at least two major, murderously opposed, ideological groups, the “Cosmists” (based on the word “cosmos”, which is their perspective) who will be in favor of building artilects, and the “Terrans” (based on the word “terra”, the earth, which is their perspective) who will be opposed to building them. A major war between these two groups (an “Artilect War”) would probably kill billions of people. Alternatively, if the artilects are built, there is always the risk that they may decide to kill us off, for whatever reason. 1. Introduction This article argues that it is quite possible that there will be a major war (called the “Artilect War”) before the end of the current century over the issue of “species dominance”, i.e. whether humanity should or should not build “artilects” (artificial intellects, godlike massively intelligent machines). The arguments in this article are shortened versions taken from the author’s book “The Artilect War : Cosmists vs. Terrans : A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines” [deG 2005]. The author argues that 21st century technologies will allow humanity, if it chooses, to build “artilects”, with “godlike” capacities that are trillions of trillions of times above human levels, e.g. in intelligence, memory capacities, processing rates, etc. The second section of this article will introduce briefly these technologies that will allow artilects to be built. Once these technologies exist, it is likely that a major debate over “species dominance” will heat up, as humanity takes sides on the issue. This debate is described in section 3. It is likely that there will be at least two major, and probably three, ideological camps. The “Cosmists” (based on the word cosmos) will be in favor of building artilects, and their arguments as to why artilects should be built are presented in section 4. The “Terrans” (based on the word “terra”, the earth) will be opposed to building artilects, and their arguments as to why artilects should never be built are presented in section 5. A third group, the “Cyborgists” take the view that humans themselves can become artilects by adding components to their brains etc, and thus avoid the issue that divides the Cosmists and the Terrans. Their arguments are presented in section 6, along with the views of the Cosmists and the Terrans towards Cyborgism. Section 7 discusses how the author thinks the species dominance debate will heat up, and how it could boil over into the worst war that humanity has ever seen, namely, the “Artilect War” between the Cosmists and the Terrans as depicted in section 8. Section 9 attempts to answer some questions that may have occurred to readers of the previous sections. 2. Artilect Enabling Technologies a) Moore’s law Gordon Moore, one of the founders of Intel, the microchip manufacturing company, noticed a trend in the middle 1960s that the number of transistors being crammed onto a chip was doubling roughly every year or two. This trend, now immortalized by the label “Moore’s Law” has remained valid for 40 years, implying many doublings. One can now speak of “massive Moore doublings. It is thought that this trend will continue until about 2020, by which time, technology will be storing a single bit of information on a single atom. The computational capacities that such tiny scales will allow are truly gargantuan – for example, an apple contains roughly a trillion trillion atoms, hence bits. At such tiny scales, atoms can switch their state (e.g. from stable to excited and vice versa) in femtoseconds (a thousandth of a trillionth of a second). With all the atoms/bits of the apple doing this would generate a computational capacity of 1040 bit flips a second, which is approximately a trillion trillion times greater than the human brain’s estimated equivalent bit flip rate. Hence Moore’s Law allows future computers to be astronomically superior in computational capacities relative to human brains by a factor of trillions of trillions. For “physics of computation (phys-comp)” specialists, such as the author, the writing is on the wall. The days of human species dominance are numbered. A century from now it is likely that the artilects will be the dominant species. b) Reversible Computing Landauer’s Principle states that heat is generated in electronic devices when information is destroyed, i.e. bits are wiped out, as is the case with today’s computational circuits based on the NAND logic gate, with its two bits in and one bit out. With molecular scale circuitry, if we continue to compute in the same old way (i.e. irreversibly, destroying bits at every step) then the heat generated will be so great that the circuits will not only melt, they will explode. Hence future circuits have to be reversible, saving all bits. It has been known since the 1970s how to do this, yet only recently have conferences been held on this topic. c) 3D Circuitry One of the major consequences of reversible computing is that circuitry can become three dimensional (3D), because there will be no heat generation problem. Hence there will be no limit to size. Future artilects could be built the size of asteroids, hundreds of kilometers across. The computational capacity of asteroid sized artilects would be a trillion times greater than that of an apple. d) Nanotechnology Nanotechnology (the technology of the nanometer scale, i.e. a billionth of a meter) is molecular scale engineering, i.e. building machines and tools the size of molecules. Nanotech offers humanity the ability to wipe out disease and mortality, by programming zillions of molecular robots that roam the blood stream repairing diseased and aging cells. Nanotech will provide powerful new tools to many research fields, see below. e) Embryonics Embryonics is short for “embryological electronics”, the idea that molecular scale electronic components can behave in ways analogous to living cells, i.e. reproducing, differentiating, migrating, self-repairing, and self-organizing. Future electronic circuits will be capable of growing in an embryological manner, to form products, “embryofacture” (embryological manufacture). Nanotech will make embryonics practical. f) Quantum Computing Quantum computing (QC) is a new style of computing based upon the principles of quantum mechanics. Today’s classical computational (CC) style is based on classical mechanics. QC is exponentially more capable than CC. For example, using the quantum mechanical concepts of superposition, and entanglement, a quantum computer can compute 2N things (where N is the number of (quantum) bits used in the calculation) in the same amount of time that a classical computer computes 1 thing. If N is large, 2N is gargantuan. Artilects will be based on quantum computation. The above “trillions of trillions” numbers mentioned above then become hugely too small. A “QCed” artilect would then be even more godlike than a “classical” artilect. g) Brain Science There are two broad strategies for brain builders to achieve human level intelligence in their artificial brains. One is to take a purely engineering approach, doing whatever one wants, and not having to be a “slave to neuroscience”. The other approach is to copy our own brain as closely as possible. Evolution has found a way to build (molecule by molecule) a creature that is both conscious and intelligent. Hence it can be done. There is a solution implicit in our DNA, just waiting for neuroscience to discover it. Once discovered, these principles can be incorporated immediately into the artificial brains of the brain builders. Brain builders ought then to pay close attention to the lessons learned by the neuroscientists. However, neuroscience is still highly undeveloped. It is still not clear on what consciousness is, how thoughts are generated, how memory is formed etc. Most of the deep neuroscience questions remain to be answered. But with the new nanotech tools that humanity will have in the next few decades, neuroscience will be revolutionized. Nano robots will position themselves at every interneural connection (synapse) and radio broadcast their positions and local data that will be collected to build up an accurate model of the brain in a hyper nano computer. The discovery of neuroscience principles should then be rapid. The neuroengineers will then benefit from the progress in neuroscience. In time the two fields will merge. Neuroscientists will be influenced by the models being tested by the neuroengineers. h) Intelligence Theory Once sufficient knowledge is acquired from many human brains, correlations between more and less intelligent brains and their physiological characteristics ought to stimulate brain theorists to create an “intelligence theory” that will allow them to understand what intelligence is and how to generate it. Once this theoretical understanding exists, it will be possible to incorporate it into machines, to speed up the thinking process by a million fold and to expand hugely the overall capacity of artificial brains compared to biological brains by trillions of trillions of times. i) Artilect Building By incorporating all the above technologies (and others not mentioned above) into artificial brains, neuro engineers (brain builders) will be able to build artilects, the godlike machines whose capacities will be astronomically superior to our human levels. These developments will very probably occur within our current 21st century. They will have enormous social, political, ethical, and religious implications, so that it is only to be expected that a great debate on the theme of species dominance will take place shortly. One of the life goals of this author is to help that debate get off the ground. It has already started. There are already world wide interest groups on the internet with thousands of members, e.g. the “Transhumanists” (http://www.transhumanism.org) and the “Extropians” (http://www.extropy.org). 3. The Species Dominance Debate In light of the technologies presented in the previous section, the author believes it is only a question of time, perhaps a few decades from now, before artificial brain based industries will dominate the global economy. Intelligent devices, the size of molecules will be everywhere and invisible. Machines will become smarter by the year, until several billion people in the richer countries begin asking themselves the following kinds of questions. They will see their own household devices closing the IQ gap between the human level and the machine level. Many people will be asking, “Could these machines approach human level intelligence?” “Is that a good thing?” “Should humanity stop them reaching human intelligence levels?” “Can their growing artificial intelligence levels be stopped?” “Could they become smarter than we are?” “If so, could that be dangerous for humanity?” “If they become a lot smarter, how could we be sure that they would always be nice to us?” “Could there be conflict between those people who want to build machines smarter than humans and those who don’t?” One can expect that a great debate will begin. Amongst the intellectuals and academics, this debate has already begun. This author thinks that the species dominance issue will dominate our global politics this century. In a few decades, as the machines start to get a lot smarter, the general public will get involved. The media and the politicians will become active, discussing the topic of “species dominance”. “Should humanity build massively intelligent machines?” As the debate begins to heat up, people will start to take sides. The author sees two major groups and possibly a third, arising. The author labels the group in favor of building “artilects” (artificial intellects) “Cosmists”. Those people opposed to artilects being built the author labels “Terrans”. A third possible group, the “Cyborgists”, think that humans themselves can become artilects. Each of these groups will have their own reasons for their beliefs. As the debate catches the public’s attention, the media will feed the interest. Politicians will begin to formulate policy on the “artilect issue” (i.e. whether they should be allowed to become as smart as humans or not). As the IQ gap closes, the temperature of the debate will rise. Intellectuals and academics will write treatises on the issue, exploring its many complexities, both technical and ethical. In time, the issue will come to dominate peoples minds, so that the debate will take on a more violent and angry tone. People will polarize into bitterly opposed sides. 4. The Cosmist Case The Cosmists are those people who want to build artilects. They will have many strong reasons to support their desire. Here are some of them. a) God Building A fully blown artilect would be truly godlike. Its capacities, as shown in section 2, would be trillions of trillions of times above human levels. It would “think” a million times faster than we do (i.e. at electronic speeds, rather than at our slow neural chemical speeds). It would be immortal. It could change its shape and architecture in milliseconds. It could go anywhere it wants, do anything it wants, perhaps even build universes, and hence by definition, be godlike. To Cosmists, building gods would be the “destiny of the human species”, building the “next rung” up the evolutionary ladder. It would be so intoxicating a prospect that it would seem to be irresistible. b) The Big Picture Most people seem engrossed in their daily lives and activities, but from a Cosmist perspective, our human lives are of total insignificance. After all, we live on a speck planet in a speck solar system, whose star is one of hundreds of billions in our galaxy, amongst hundreds of billions of other galaxies in our universe, and perhaps there are even many universes. Our pathetic little human lives are snuffed out in a mere 80 years in a universe that is billions of years old. As humans we are nothing. However, an artilect could be immortal. It could go into deep space in search of other advanced life forms. It would be a hugely more significant being than are puny ephemeral humans. Building artilects would be consistent with seeing the “big picture” more realistically, not living in a fantasy world of our own human importance in a universe that does not give a damn about us. c) Cosmism as a Scientists Religion Most scientists in most countries are not religious, looking upon traditional religions as superstitions that make no sense in the light of modern scientific knowledge and critical thinking. However, it appears to be human nature to feel the “religious impulse”, i.e. to be concerned with the deep existential (spiritual) questions of existence, such as why there is anything rather than nothing, why are the laws of physics the way they are rather than some other way, or if the universe has any purpose. Cosmists will be able to use Cosmism as a “religion” in the sense of a set of beliefs that orients their lives, giving their lives purpose, a goal, a sense of awe, of magnificence, i.e. god building. Cosmists will be able to satisfy their religious impulses yet remain scientifically rigorous because Cosmism is based on science. A “scientific religion” will be a powerful motivator for the Cosmists, perhaps their most powerful. d) The Human Striving Argument Evolution has made human beings curious. A creature which is curious about its environment is more likely to explore and discover its characteristics more quickly. This confers survival value on the creature, similarly with the drive to go beyond the known. The Cosmists will argue that it will be virtually impossible to stop humans from wanting to build artilects. It is their human nature to build them. Human beings cannot help themselves. We are curious strivers by nature. e) Economic Inertia In time, the artificial brain (A-Brain) industry will be worth a trillion dollars a year worldwide, a situation comparable with today’s PC industry. There will be so much financial investment and human ego involved in keeping the A-Brain industry alive, that its enormous momentum will be almost impossible to stop. f) Military Inertia A similar argument holds for the military. In the next few decades it is unlikely that the planet will have a global state with a global court and police force. Hence there will be international rivalries between nation states, e.g. between the US and China, or between India and China. The minister of defense of each country will not have the luxury to allow his counterpart in the rival country get ahead in the race to build ever more intelligent weapon systems. Hence the military momentum in favor of building A-Brains will be very strong, even stronger than the economic momentum. 5. The Terran Case The Terrans are those people who are opposed to building artilects. They will have as many strong reasons to support their opposition as the Cosmists have in favor. Here are some of them. a) Preserve the Human Species The Terrans strongest argument is based on fear. They will be afraid that if the artilects are built in a highly advanced form, they will be so hugely superior to human beings, that for whatever reason, they might decide to wipe us out and be totally indifferent about it. One can make a human level analogy when we swat a mosquito. That mosquito is a miracle of nanotech engineering that modern science is still incapable of building from scratch, yet we still kill it because relative to us, we noble superior human beings, it is a pest. The artilects could become so vastly superior to us, that our fate would be a matter of utter irrelevance to them. But you may argue that since the artilects would be highly intelligent, they would realize that we humans were their parents, and hence they would respect us. Maybe, but we could not be certain. The key word in this Cosmist/Terran species dominance debate is “risk”. What is the risk that artilects, if built and attain godlike capabilities, vastly superior to human levels, might decide to get rid of us? The Terran philosophy is that the only way to be sure that this risk never arises is to ensure that the artilects are never built in the first place. But this attitude conflicts directly with the almost “religious” drive of the Cosmists to build them. The Terrans will argue that when push really comes to shove, if the Cosmists really go ahead and start building artilects, it is preferable to kill off a few million of them, so that the billions of human beings can survive, free of the risk of being wiped out by a vastly superior and indifferent species of artilects. Since the stake has never been higher, namely the survival of the whole human species, the Terrans will be ruthless. Passions will be high. More will be said on this in sections 7 and 8 on how the species dominance debate will heat up and how an “Artilect War” between the Cosmists and the Terrans might ignite. b) Fear of Difference Human beings have an evolved fear of difference. The sight of a 3 eyed “monster” evokes an instinctive rejection reflex. As early artilects begin to appear on the planet with their intelligent yet alien “minds”, many human beings will start to fear them. Fear is a very powerful motivator. c) Rejection of the Cyborgs “Cyborgs” (cybernetic organisms), i.e. part machine, part human, are human beings who add artilectual components to their brains. To the Terrans, a cyborg and an artilect are not so very different. A Terran could claim that a cyborg is just an artilect in human disguise. There is more artilectual computing capacity in a grain of sand than that of the human brain by a factor of trillions. Terrans would fear the rise of cyborgs almost as much as true (machine based) artilects. The Terrans will lump the artilects and the cyborgs into the same ideological camp, i.e. the “enemy”. d) Unpredictable Complexity This argument is more complex. It is based on the technology used by the brain builders to create artilects, namely “evolutionary engineering”. Evolutionary engineering uses evolutionary algorithms to evolve their brain circuits, using a Darwinian, “survival of the fittest” approach in which DNA like random instruction strings are used to code for the construction of neural circuits. A population of such strings compete with each other to survive into the next generation. Those with weaker performance scores on some task die off, while the better ones survive. By mutating randomly the coded instructions, some strings score slightly better and rise in the hierarchy. By looping through this algorithm many times, functional neural circuits can be evolved. However, the complexities of such circuits, even though they function, are often beyond human understanding. It is due to these complexity levels that the evolutionary engineering methods are used in the first place. Traditional blue-print, top-down design methods when faced with trillions of artificial neural connections is simply impractical. An evolutionary method is virtually essential. The Terrans will use this argument to say that evolved artilects will be unfathomable, i.e. too complex to understand. Hence as human beings, we could never be sure of the “ethics” of the artilects towards us. Their circuitry would be truly alien. If it is impossible to understand their circuitry, how could we as human beings ever trust the artilects to behave towards us as we would like. We could never be sure. e) Cosmist Inconsideration This is a selfishness argument. The Terrans will accuse the Cosmists of being supremely selfish in pursuing the construction of artilects, not only at their own risk, but at those of the Terrans as well. The Terrans will argue that there is always the possibility that the Cosmists might be wiped out by their own creations, and possibly the Terrans as well, both groups being human beings. This level of selfishness on the part of the Cosmists towards the Terrans will only make the Terrans hate the Cosmists all the more. Even if Terran pressure forces Cosmists off the planet into deep space to perform their artilectual experiments, the Terrans could argue that the artilects could decide to return to the earth and destroy the Terrans. Hence the Terrans will not even allow the Cosmists to launch themselves into deep space. 6. The Cyborgist Case The Cyborgists are a third ideological group of people who claim that the bitter conflict between Terrans and Cosmists can be avoided by having human beings themselves become artilects. This author thinks this idea is rather naïve, because the Terrans will not tolerate them, for the same reasons that they will not tolerate the artilects, i.e. both, in an advanced form, would be a threat to humanity’s survival. Many people may want to become artilects themselves, gradually adding components to their brains. Soon, they would no longer be human, since the capacities of the artilectual components they would add to their brains would dwarf their human capacities. A cyborg would rapidly become an artilect in the eyes of the Terrans. The Terrans will have the same level of hatred for the Cyborgists as the Cosmists. 7. The Debate Heats Up As the technologies mentioned in section 2 are created and merge, humanity can expect to see a rapid rise in the level of artificial intelligence of its artificial brains. Neuroscience will be feeding its newly discovered principles into the brain building industries. Artificial brain based household gadgets will be everywhere. There will be teaching robots, cleaner robots, sex robots, advice giving machines, conversation machines, companion machines, etc. The A-Brain industries will become the richest on the planet. Several billion people on the planet in the coming decades will have their own A-Brained machines and spend a lot of money on them, equivalent to the way people spend a lot of money on their cars and boats today. However, progress in the above technologies will mean that last year’s model will not be as intelligent as this year’s. As the IQ gap between artificial and human intelligence closes, billions of people will be asking themselves the questions mentioned in section 3 on the species dominance debate. Once the machines begin to become really smart, the species dominance debate will truly heat up. Passions will rise. People will take sides. Sabotage will start, so will assassinations. Brain builder companies will see their CEOs and star researchers murdered. The Cosmists will label the Terrans terrorists, and the Terrans will label the Cosmists, species murderers. Academics will write many books on the many aspects of the debate, the ethics, the politics, the technologies, the prospects, etc. 8. The Artilect War As the temperature rises in the debate so will the probability of a war between the two major groups, the Cosmists and the Terrans. This time, the war is not about the survival of a nation. It is about the survival of the whole human species, so the stake is much higher, hence the passion levels will be much higher. Since it is likely that the neurosciences will take many decades to decipher the principles of brain function, real life artilects will not be with us quickly. Their progression up the artificial intelligence scale, this author believes, will take at least several decades and probably more. The human brain has some quadrillion synapses remember. It is the most complex entity known to humanity. Nevertheless, the arsenal of tools emerging to tackle how the brain functions are formidable, as listed in section 2 on artilect enabling technologies. Artilectual progress will be initially slow but will grow exponentially in speed. This raises the question whether there will be enough time between the start of the species dominance debate and the creation of godlike artilects, for human politics to unfold. The answer to this question depends obviously on just how much time artilect development will need. In the opinion of this author, who is a professional brain builder and who studies the neuroscience literature, many decades will be needed, so the early artilects will not progress so rapidly that humanity will be caught unawares. Creating real artificial intelligence will be a hard slog. This timing factor will be of importance to the Terran strategists. The Terrans cannot wait too long if they are to assert themselves, since if they remain passive, the cyborgs and artilects will be amongst them with all their superior capacities, and hence if a conflict does arise between the Terrans and the artilects, the latter will obviously win, if the latter are sufficiently advanced. So, the Terrans need to think carefully when they should strike, and how. Now, the Cosmists will not be passive either. The Cosmists will include some of the planet’s most brilliant and far thinking minds, people of great wealth and ego. They will not passively watch their cosmist dream, their aspiration to build gods, evaporate at the hands of the Terrans. The Cosmists too will be strategizing. As the debate rages, tempers will fly, factions will polarize all the more, and more and more people will start talking about a future war. Students of the US Civil War will see all kinds of historical analogies here. Both conflicts took many decades to brew. What would be particularly potentially tragic in the case of an artilect war is the probable number of casualties. If one extrapolates up the graph of the number of people killed in major wars from Napoleonic times to a late 21st century artilect war (based on 21st century weapons), one comes to a figure in the billions, a concept this author labels “gigadeath”. If this figure seems shocking, consider the fact that about 200 million people were killed for political reasons (wars, purges, genocides, etc) in the 20th century. Just how the artilect war will be triggered is very difficult to predict, but at least one can discuss the odds of such a war happening. We consider two broad scenarios, one in which the rise of the artilect is quicker than anticipated. Then humanity will not have enough time to react. The artilects will exist, and it will be they who decide humanity’s fate. They may ignore us, and escape into deep space in search of other more advanced life forms, or whatever. In that case, the artilects would be built, and humanity would survive. On the other hand, the scenario so feared by the Terrans may arise, namely that the artilects find human beings to be a pest and exterminate us. In the second broad scenario, there is enough time for human politics to unfold (the preferred view of the author), the species dominance debate rages, tempers fly, war preparations and strategies are set, and eventually a war erupts, killing billions. Whether any humans would survive the worst war in history, with the highest passion levels, with the largest stake (i.e. the survival of the human species) in the second half of the 21st century, is debatable. The bitter irony of an artilect war is that humanity would (almost) exterminate itself at its own hands (i.e. between two human groups, the Cosmists and the Terrans) in its attempt to ensure that humanity survives against the artilects (at least from the point of view of the Terrans). The actual details of how the war starts and unfolds cannot be predicted. However, given the likely level of mutual hatred between the two main groups, the size of the stake, the grandeur of the Cosmist dream of god building, the Terran desire that the human species survives, and the efficiency of 21st century weaponry, if a war does break out, it will make the Second World War seem very small in comparison. 9. Remarks This section poses the question whether such an “artilect war” can be avoided, perhaps through government planning or intervention in some way. The author is cynical on this point. Cosmism and Terranism are two very powerful ideologies. The former is based on a cosmic dream of religious proportions and will attract billions of minds. Terranism is based on the fear of a possible extinction of the human species at the hands of the artilects. Fear and the need to survive are also very powerful motivators. The cards seemed stacked for a bitter conflict. Perhaps governments could attempt to suppress Cosmism, but probably without complete success, because it may be possible for small groups of Cosmists to build their artilects “in the basement” so to speak. If this is technically possible, then governments may have to become police states to invade the privacy of everyone in their attempts to weed out those people with Cosmist viewpoints and sympathies. As artilects approach human intelligence levels and are given more responsible jobs, sooner or later, Murphy’s law (i.e. if something can go wrong, it will) will apply. The smart machines will screw up, resulting in greater Terran hatred for the Cosmists. The strongest reason why this author is cynical that governments can prevent an artilect war is that Cosmist/Terran sympathies are so evenly divided in the population. The author has given many Cosmist/Terran talks to both academic and to some general audiences. At the end of the talks he usually asks his audience to vote – “Are you more a Cosmist or a Terran? Choose!” The vote is always around 50/50, 60/40, 40/60. At first this was interpreted as a result of the newness of the issues, but gradually it dawned upon the author that the people in the audience were as equally ambivalent about the issue as he still is. The author is publicly Terran, raising the alarm, and trying to get the artilect debate, i.e. the species dominance debate, off the ground, but privately is Cosmist. He feels that it would be a “cosmic tragedy, if humanity decides to freeze the state of evolution at its current puny level, when it could be building gods”. This article will finish with the pithy slogan found at the end of the author’s book, “Do we build gods, or do we build our potential exterminators?” [deG 2005]. 10. References [deG 2005] Hugo de Garis, “The Artilect War : Cosmists vs. Terrans : A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines”, pp 250, Etc Books, February 2005, available on www.amazon.com [Transhumanists] http://www.transhumanism.org [Extropians] http://www.extropy.org