by ARIK KERSHENBAUM

Natural selection seems, at first glance, to be so frustratingly
inefficient. Generation after generation of baby gazelles are born,
destined to be eaten by lions. Only by chance is one baby born with
longer legs, able to run faster, and so escape being eaten. Of course,
the very beauty of natural selection is that it doesn’t require any
foresight; natural selection explains life in the universe precisely
because there is no presumption of any prior knowledge. No Creator is
necessary, because the evolutionary process is guaranteed to proceed
even without any predefined rules. Life evolves—albeit slowly—without
having to know where it’s going.
But what if it were all different? What would life look like if it did know where it was going?
The 1950s physicist Anatoly Dneprov wrote quirky and characteristically Soviet science fiction. His novel Crabs on the Island tells the story of two engineers conducting an experiment in cybernetics on a deserted island. A single self-replicating robot (a “crab”) is released, and forages for the raw materials to build other robots. Soon the island is overrun with baby robot crabs. But the crabs begin to mutate. Some are larger than others, and ruthlessly cannibalize the smaller robots for spare parts to build even larger robots. How would such an experiment end? Catastrophically, of course, as is consistent with the genre, with robot crabs spreading exponentially across the entire island.
Science fiction can be terribly pessimistic, but that pessimism is unfounded. Other factors are at play. Resources are limited. Eventually, even the crabs on the island run out of materials with which to make new robots. Admittedly, humans have caused tremendous damage to our own planet, but we’ve hardly destroyed the universe. In fact, there’s no indication in the night sky that any organism, biological or artificial, has spread its influence as far and wide as we might expect if they were growing exponentially like robot crabs.
But we must also be cautious with our optimism. We rely on the age-old processes of natural selection to keep reproducing robot crabs in check; something will evolve to eat them. But what if these were intelligent organisms, plotting a way to find new resources, discovering new ways to improve themselves, their evolutionary fitness, and their ability to learn from each other and from previous generations? Could such an army of replicating artificial intelligences be possible? If so, could they be stopped? How realistic is it that alien planets may be inhabited by artificial creatures so advanced that they can bypass natural selection itself? And if that is possible, why has such a creature never evolved naturally? If we want to know whether or not we should fear alien artificial intelligence, first we have to understand what’s so special about it.
It has probably not escaped your attention that the kind of intelligent transmission of experience from one generation to the next—together with the ability to know when to use that information—is not unlike what we see in human society in the cultural transmission of ideas from generation to generation. We don’t need genetics to learn about science, we just need a school. More importantly, we don’t need to follow a religion or a political ideology indefinitely and unchallengingly, we can detect when it’s not serving our needs and change our direction. Cultural transmission of experiences is a process with spookily Lamarckian characteristics. It was the Enlightenment French biologist Jean-Baptiste Lamarck (1744–1829), who, in the century before Darwin, attempted to explain the fact that animals appear to be remarkably well adapted to their environment and had a number of ideas how this might happen, the best known of which was his two-pronged laws of inheritance—that animals develop traits that they use repeatedly, lose traits they don’t use, and pass on acquired traits to their offspring.
Nautilus for more