Modern artificial intelligence is built to mimic nature—the field’s main pursuit is replicating in a computer the same decision-making prowess that humankind creates biologically. For the better part of three decades, most of AI’s brain-inspired development has surrounded “neural networks,” a term borrowed from neurobiology that describes machine thought as the movement of data through interconnected mathematical functions called neurons. But nature has other good ideas, too: Computer scientists are now revisiting an older field of study that suggests putting AI through evolutionary processes, like those that molded the human brain over millennia, could help us develop smarter, more efficient algorithms.

1. 1
Researchers are using Darwin’s theories to evolve AI, so only the
strongest algorithms survive
By.Dr.Mahboob ali khan Phd
Modern artificial intelligence is built to mimic nature—the field’s main
pursuit is replicating in a computer the same decision-making prowess
that humankind creates biologically.
For the better part of three decades, most of AI’s brain-inspired
development has surrounded “neural networks,” a term borrowed from
neurobiology that describes machine thought as the movement of data
through interconnected mathematical functions called neurons. But
nature has other good ideas, too: Computer scientists are now revisiting
an older field of study that suggests putting AI through evolutionary
processes, like those that molded the human brain over millennia, could
help us develop smarter, more efficient algorithms.
But first, back to middle-school biology class. The concept of
evolution, famously credited to Charles Darwin and refined by
countless scientists since, states that slight, random changes in an
organism’s genetic makeup will give it either an advantage or
disadvantage in the wild. If the organism’s mutation allows it to survive

2. 2
and reproduce, that mutation is then passed along. If it doesn’t, the
mutation dies with its organism. In algorithm world, this is known as
neuroevolution. While artificial neural networks replicate the process
of learning individual concepts, neuroevolution tries to recreate the
process that built parts of the brain—i.e. the process by which only the
strong (or smart) survive.
Although neuroevolution has been around since the 1980s, the concept
is getting renewed attention as researchers dig into the archives for
different perspectives on machine learning. In the past month, Google
Brain and non-profit organization OpenAI each published unreviewed
papers on the subject, Google’s on the application of neuroevolution
principles to image recognition and OpenAI’s on using “worker”
algorithms to teach a master algorithm the best way to accomplish a
task.
Bringing biological evolution into the already complex field of AI
research can be confusing. So if it’s easier, think of algorithms as
horses. Horses learn throughout their lifetime, but they are
only evaluated on a few different metrics, like how fast they run.
Accuracy in image recognition is easy to assess as a single number, as
is the amount of time it takes a horse to run around a track. But what
actually makes that horse run faster is incredibly complicated—a vast
network of DNA that enables muscle growth, higher stamina, and even
intellect. That complexity mirrors the underlying parameters of
algorithms, or how an algorithm might be good (or bad) at image
recognition. So if you get lost anywhere in the article, just take a deep
breath and think “horses.” (This is also good life advice.)
For its research, the Google team generated 1,000 image-recognition
algorithms that were trained using modern deep neural networks to
recognize a specific set of images. Then 250 computers each chose two
algorithms and tested their accuracy by making them identify an image.

3. 3
The algorithm with higher accuracy lived, while the one that performed
poorly was “killed.” The survivor was then copied, and its clone (or
“child”) was changed slightly—just like human DNA randomly
changes during reproduction. But instead of blue eyes or a widow’s
peak, this mutation slightly altered how the new algorithm interprets
training data. The clone was then trained using the same data as its
parent, and put back into the batch of 1,000 algorithms to start the
process over again.
Google researchers found that neuroevolution could cultivate an
algorithm with 94.6% accuracy, and recorded similar (though not
identical) results during each of four repeats of the experiment.
Mutations that improved the algorithm’s image-recognition skills were
rewarded (i.e. those algorithms survived) while mutations that
decreased performance were killed off. Just like in nature.
The differences between the five sessions also illustrated a consistent
problem. Google researcher and paper co-author Esteban Real says the
algorithms kept getting stuck halfway through the process, seemingly
unsure whether to continue evolving or scrap the mutation and start
over. Real says an analogy in nature might be the evolution of wings.
“Half a wing might not help you very much,” he says, “but a full wing
lets you fly.”
Google’s team is now working on getting the evolutionary models to
explore different mutations more fully (to build whole wings). But that
gets tricky. The team only wants the algorithm to mutate in a limited
way, so it doesn’t end up with a whole bunch of extra code that isn’t
useful. Explains Real: “The worst would be having many half-wings.”
By focusing mainly on image recognition, Google tested both
neuroevolution’s capacity for tackling something the biological brain
is great at, and its ability to solve a modern problem. OpenAI, on the
other hand, used a more pure form of evolution to undertake a different
task.

4. 4
Rather than training thousands of algorithms to get better at one thing,
the OpenAI team wanted to use “worker” algorithms to train a master
algorithm to accomplish an unknown task, like playing a videogame
or walking in a 3D simulator. This technique isn’t the primary way to
teach machines how to make decisions, but a way to teach them how to
learn from specific information more efficiently, explains co-author
and OpenAI researcher Tim Salimans. The evolutionary algorithm is
able to monitor how its workers are learning, and it essentially learns
to learn—that is, to extract more knowledge from the same amount of
data.
To conduct their research, the OpenAI team set 1,440 worker
algorithms to the task of playing Atari. The workers played until they
reached Game Over, at which point they reported their scores to the
master. The algorithms that garnered the best scores were copied, as in
the Google research, and the copies were randomly mutated. The
mutated workers then went back into rotation and the process repeated
itself, with advantageous mutations being rewarded and bad ones
killed.
This approach also has its limitations, chief among them that the worker
algorithms only report one number, their high score, back to the master
algorithm. The algorithms with the best scores survived, but trying to
make the master aware of any specifically successful moveswould
require far too much computing power. (In biology, the parallel might
be an ant colony: Workers go out and find the most optimal solutions;
the queen is the central hub of information.) In other words, OpenAI
learned a lot about success, but less about scrappiness.
Back in the 1980s, neuroevolution and neural networks were similarly
sized fields of study, says Kenneth Stanley, an associate professor at
the University of Central California and a recent addition to Uber’s AI
team (through their acquisition of Geometric Intelligence, which he co-
founded).

5. 5
“There was a small community of people who thought about how
brains, which are really the only proof of concept of intelligence in
nature, get into the world,” Stanley says. “Some people thought maybe
the most straightforward way to do this would be to create a
evolutionary, Darwinian-like process in a computer that acts on little
artificial brains.”
Neural networks took off when three computer scientists—David
Rumelhart, Geoffrey Hinton, and Ronald Williams—published a 1986
paper describing an algorithm that enhanced the way those networks
learned from their mistakes, called backpropagation. The findings
greatly improved the efficacy of hand-built neural nets, but an
impending AI winter—funding was slashed for purported lack of
progress—impeded further growth. It wasn’t until the mid-aughts that
Hinton and company began publishing papers that made neural
networks too enticing for the larger computer science community to
resist, showing that backpropagation allows neural networks to grow
immensely, and to in turn understand far more complex ideas. These
networks were dubbed “deep,” and deep neural networks became the
most popular flavor of modern artificial intelligence.
“Because of that, there was some loss of awareness for neuroevolution,
which was this parallel thread of evolving brains,” says Stanley.
Back in 2002, at the start of his career, Stanley wrote an algorithm
called NEAT, which allowed neural networks to evolve into larger and
more complex versions over time. His corresponding paper has more
than 1,600 citations on Google Scholar, and has been referenced in
deep neural-network design and neuroevolution research since its
publication. In 2006, Stanley published Hyper-NEAT, an algorithm
that made neuroevolution much greater in scale, inspired by DNA’s
ability to be the blueprint for billions of biological neurons with
trillions of connections, despite only having around 30,000 genes. (Fun
fact: Hyper-NEAT’s full name is Hybercube-based NeuroEvolution of
Augmenting Topologies. I challenge anyone to name a better algorithm
anagram.) Today, Stanley says it’s gratifying to see his career’s work
back in the zeitgeist.

6. 6
Like Stanley, OpenAI and Google are now tackling two different ideas
plucked from the same field. Google’s hybrid approach combines
classic neuroevolution with the techniques, like backpropagation, that
have made deep learning so powerful today: Teach an algorithm how
to act in the world, let it evolve, and that algorithm’s child will have
most of the accrued knowledge. OpenAI’s approach was more true to
how evolution works in biology. The team only let the randomized
mutations in every generation govern how the networks improved or
failed, meaning improvement was only created through random
evolution. But both attempts had very clear goals—recognize an image,
or get a high score in a game (or make a horse run faster). How the
algorithms got there was up to nature.
“Individuals are born with the weights in their brain that they’re going
to have for their entire life,” Stanley says about OpenAI’s work. “It’s
like if we bred you, and your children, and your children’s children,
and then they knew calculus.”