If The Singularity Arrives, Will It Be By Design Or Evolution?

IF THE SINGULARITY ARRIVES,
WILL IT BE BY DESIGN OR
EVOLUTION?
Bill Worzel billwzel@gmail.com
Evolution Enterprises http://evolver.biz
Data Day Texas 11 Jan 2014
Austin, TX

Monday, January 13, 14

NATURE HAS MANY ROOMS
• Animals

solve the problem
of survival in many ways

• Most

are adapted to speciﬁc
ecological niches

• Genetics

forms the common
language of living creatures


EVOLUTIONARY ALGORITHMS
(EA) BORROW FROM NATURE
• Based

on natural selection
and population dynamics

• Create

a population of
solutions

• Preferentially

select and
recombine better individuals
to ﬁnd better solutions


AN ELEGANT SEARCH
• EAs

combine global search
with local search

• Randomly

generated
individuals test many niches

• Selection

and recombination
hones in on the best
neighborhoods


GENETIC ALGORITHMS (GA)
• GAs

encode information
and then combine and
mutate individuals

Population

• In

simplest case, encoding is
a bit string mapped to
variable values

• Initial

population of
individuals are created
randomly


P/E Trend
101001011010

SELECTION & FITNESS
• Subset

of individuals are
selected at random from
population

• Fitness

of each is calculated

pair are combined to
produce offspring

32

16

• Best


32

18

x

90

90

Fitness

CROSSOVER & MUTATION
crossover pt
• Crossover

strings

• Mutation

combines bit

changes bits

• Both

operations are
stochastic

• Offspring

replace parents or
weaker individuals in
population


101001011010
x
011101101110
=
011101111010
+
101011001110
mutation

BUILDING BLOCKS AND
SCHEMAS
• Building

block hypothesis states that GAs ﬁnd good simple
components that confer better ﬁtness on individuals

• The

Schema Theorem shows that better building blocks
accrue to produce best individuals: E(m(H,t+1)) ≥ ((m(H,t)
f(H))/at)[1-p].


CASE STUDY: AGRICULTURAL
MODELING
• Decision

support software for
farmers: With large number of
new hybrids, what to choose?

• Needed

to integrate
agronomic, weather,
economic, personal factors

• GA

not as an optimizer but as
an optionizer in a multiobjective space


RICH LITERATURE FOR GA
• Many

conferences, particularly GECCO (Genetic and
Evolutionary Computing COnference)

• D. Goldberg, Genetic

algorithms in search, optimization, and
machine learning, Addison-Wesley, 1989

• J. Holland, Hidden

Order: How adaptation builds complexity,
Helix Books, 1995

• J. Holland, Adaptation

1975


in Natural and Artiﬁcial Systems, MIT Press

GENETIC PROGRAMMING
(GP)

• GP

evolves computer programs (usually functions)

• Essentially
• Extends


a program that produces programs as its output

idea of combining bit strings to parse trees

GP OVERVIEW
Input Data

GP Parameters

?
Program
Population

Replace
Least Fit
With
Offspring

Yes
Terminate?

GP
Cycle
?

Crossover
and
Mutate


No

Select
Mating
Group

SelectTwo
Best
Programs

Output
Results

?

? = stochastic process

CONSTRUCTING TREES
• Randomly

assemble a population of function trees as
constrained by GP parameters

From: ‘A Field Guide To Genetic Programming’


CROSSOVER
(RECOMBINATION)


MUTATION



THE DEVIL IN THE DETAILS

• How

do you correct syntax errors?

• Type

coherence?

• Control

overﬁtting?

• Computationally


intensive

BUT HEAVEN’S ON OUR SIDE
• Naturally

iterative

parallel algorithm - linear speedup, mostly not

• Sub-populations

may be run asynchronously in parallel: m*n/p
where m is individuals in a sub-population, n is the number of
sub-populations, and p is number of processors

• Matches


up well with cloud computing

THE SKGP
• Uses

purely functional combinators to represent programs

• Efﬁcient, powerful, reusable
• Algorithm

code

becomes superlinear in parallel application because
of code reuse


COMBINATORS
• Applicative

algebra, derived from Lambda calculus,
binds left-to-right
• Sxyz
• Kxy
• Ix

= xz(yz)

=x

=x

• Bxyz

= x(yz)

• Cxyz

= xzy


VARIABLE ABSTRACTION
• D.A. Turner

showed that all bound variables could be
removed completely using combinators (Turner 1979, A New
Implementation Technique for Applicative Languages, Software–
Practice and Experience, vol 9, 31-49 )

• Essentially

this provides a way to create expressions that are,
combinators applied to data with no reference to variables


EXAMPLE COMBINATOR
FUNCTION
Example:
‘S(S(K +)(K 1))I’
is the function
that adds 1so
S(S(K +)(K 1)I
applied to 3 is:

S(S(K +)(K 1))I 3
S(K +)(K 1)3(I 3)
K+3((K 1)3)(I 3)
+K 1 3 (I 3)
+ 1 (I 3)
+13
4


COMBINATORS FUNCTIONS
QUICKLY BECOME COMPLEX
Here is the function for factorial:
def fac = S(S(S(K cond)(S(S(K =)(K 0)))I))(K 1))(S(S(K *)I)
(S(K fac)(S(S(K -)I)(K 1))))
Evaluation is left as an “exercise to the reader.”


THE SKGP
• Implements

programs as graphs
using both combinators with GP
to produce pure functional
(combinator) expressions

• Combinators

have the property
of being ‘structure altering
operators’

• There

is evidence that GP can
be limited in its search ability
without such a capability

Daida, unpublished based on Daida2004
Demonstrating Constraints to Diversity with a
Tunably Difﬁculty Problem for Genetic Programming


CHURCH-ROSSER THEOREM
• The

Church-Rosser Theorem says pure function evaluation
can be order independent: Regardless of order of evaluation,
result will be the same

• Because

of this, each functional piece, when evaluated, can be
stored for re-use since order of evaluation does not matter

• Because

GP shares pieces across generations, reuse gives
super-linear speed up: you don’t have to recompute each
component


CASE STUDY: MODELING THE
MODEL
• Modeling

chemical kinetics for NASA

• NASA

had a set of ﬁrst principle models used to simulate
combustion of jet fuel and its exhaust gases: accurate but
very slow

• By

using the simulator to train the SKGP, it was able to
produce a highly accurate function for predicting output gas
amounts across a wide range of values

• Functional
• Function

results was 2370x faster than running simulation

was highly accurate empirical solution of PDEs

CASE STUDY: LISTENING TO DATA
• Collaboration

with Dr. Richard Cote and USC to study
bladder cancer

• Is

there a molecular signature that matches T-stage of
tumors? No! Attempt produced complicated, poorly
performing functions

• Examining

data showed that tumors with local metastasis
were consistently misclassiﬁed

• Is

there a signature in tumor that indicates local mets? Yes!
Produced a set of concise, highly accurate, biologically
sensitive functions that could identify when a tumor had
metastasized


SOME APPLICATIONS
• Inferential

sensors (Dow Chemical)

• Financial

modeling (Analytic Research Foundation, State
Street Global Advisors)

• Antenna
• Analog
• Solid


design (NASA)

circuit layout (Solido Design)

State Memory management (NVM durance)

OPEN SOURCE SOLUTIONS
• Java: ECJ

- a well known Java implementation from one
of the well known researchers in GP

• Python: DEAP

Python

- an “all-in one package” written in

• Clojure: PushGP

- a stack-based version of GP with
many nice features, also written developed by a
respected GP researcher


PROPRIETARY

• Evolver

by Evolution Enterprises: http://evolver.biz

Modeler by Evolved Analytics: http://
www.evolved-analytics.com/

• Data


GP REFERENCES
• J. Koza, Genetic

and Kluwer.

Programming I-IV, Morgan Kauffman

• R. Poli, W.B. Langdon

and N.F. McPhee, A Field Guide
to Genetic Programming

•

<Various> Genetic Programming Theory and Practice IX1, 2002-2013

•

Mitra et al, The use of genetic programming in the
analysis of quantitative gene expression proﬁles for
identiﬁcation of nodal status in bladder cancer, BMC
Cancer, 6(159) 2006


POSSIBLE FUTURES
•

Some immediate areas of application include Smart Grid
and energy efﬁcient designs, intrusion detection, discovery of
protein-gene-SNP networks

•

Since evolutionary algorithms give a multi-dimensional
analysis in the form of a population of solutions they provide
more information than a single solution

•

EAs can continuous analyze data as it comes in, adapting to a
changing environment while still providing high performance
solutions

•

There is a bridge from functions to full programs, though
functional methods reduce the gap and could lead to
functional co-applications (an ecology of functions)


“THE BEST WAY TO PREDICT
THE FUTURE IS TO INVENT IT.”
-ALAN KAY


If The Singularity Arrives, Will It Be By Design Or Evolution?

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (11)

Similar to If The Singularity Arrives, Will It Be By Design Or Evolution?

Similar to If The Singularity Arrives, Will It Be By Design Or Evolution? (20)

Recently uploaded

Recently uploaded (20)

If The Singularity Arrives, Will It Be By Design Or Evolution?