Introductory Talk at Ben Gurion University

ASAP - Interdisciplinary Optimisation
Laboratory
Natalio Krasnogor

ASAP - Interdisciplinary Optimisation Laboratory
School of Computer Science

Centre for Integrative Systems Biology
School of Biology

Centre for Healthcare Associated Infections
Institute of Infection, Immunity & Inflammation

University of Nottingham
Ben-Gurion University of the Negev - June 23rd to July 5th 2009 - Distinguished Scientist Visitor Program - Beer Sheva, Israel 1 /41
Wednesday, 24 June 2009

Research Themes
• The IOL mission is the development of cutting-edge decision
support, optimisation and search methodologies for problems arising
in the natural sciences.

• Research activities lie at the interface of Computer Science and the
Natural Sciences, e.g. Biology, Physics, Chemistry.

• In particular, we focus on developing innovative and competitive
search methodologies and intelligent decision support systems with
an emphasis on transdisciplinary optimisation, modeling of complex
systems and very-large datasets processing.

• We have applied our expertise in Bioinformatics, Systems Biology,
Synthetic Biology, Nanoscience and Chemistry.

Acknowledgements
(in no particular order) (in no particular order)
 Peter Siepmann
 School of Physics and Astronomy
Contributors to the talks I will give at BGU

 Pawel Widera
 School of Chemistry
 James Smaldon  School of Pharmacy
 Azhar Ali Shah  School of Biosciences
 Jack Chaplin  School of Mathematics
 Enrico Glaab  School of Computer Science
 German Terrazas  Centre for Biomolecular Sciences
 all the above at UoN
 Hongqing Cao
 Jamie Twycross Funding From:
 Jonathan Blake BBSRC, EPSRC, EU, ESF, UoN
 Francisco Romero-Campero
Thanks also go to:
 Maria Franco
 Adam Sweetman Ben Gurion University of the Negev’s
 Linda Fiaschi Distinguished Scientists Visitor Program
 Open PhD Vacancy
Professor Dr. Moshe Sipper
 Open PostDoc Vacancy

Motivation
• Automated design and optimisation of complex
systems’ target behaviour
• cellular automata/ ODEs/ P-systems models
• physically/chemically/biologically implemented

• Present a methodology to tackle this problem
• Supported by experimental demonstration


Major advances in the analytical design of large and
complex systems have been reported in the literature
and more recently the automated design and
optimisation of these systems by modern AI and
Optimisation tools have been introduced.

It is unrealistic to expect every large & complex physical,
chemical or biological system to be amenable to fully
analytical designs/optimisations.

We anticipate that as the number of research challenges
and applications in these domains (and their complexity)
increase we will need to rely even more on automated
design and optimisation based on sophisticated AI &
machine learning


This has happened before in other research
and more disciplines,e.g: automated design and
and industrial recently the
•VLSI design
Optimisationdesign/optimisation been introduced.
tools have
•Space antennae design
•Transport Network
•Personnel Rostering
•Scheduling and timetabling

machine learning


and more disciplines,e.g: automatedcomplex systems are plagued with
That is, design and
•VLSI design NP-Hardness, non-approximability,
uncertainty, undecidability, etc results
tools have

machine learning


and more disciplines,e.g: automatedcomplex systems are plagued with
That is, design and
•VLSI design NP-Hardness, non-approximability,
uncertainty, undecidability, etc results
tools have

Yet, they are routinely solved by
sophisticated optimisation and design
design and optimisation based like evolutionary
techniques,
on sophisticated AI &
machine learning algorithms, machine learning, etc


Automated Design/Optimisation is not only good because it can
solve larger problems but also because this approach gives access
to different regions of the space of possible designs (examples of
this abound in the literature)

Space of all possible designs/optimisations
Automated
Analytical
Design
Design
(e.g. evolutionary)

A distinct view of the space of possible designs could
enhance the understanding of underlying system


The research challenge :

 For the Engineer, Chemist, Physicist, Biologist :

 To come up with a relevant (MODEL) SYSTEM M*

 For the Computer Scientist:

 To develop adequate sophisticated algorithms -beyond exhaustive
search- to automatically design or optimise existing designs on M*
regardless of computationally (worst-case) unfavourable results of
exact algorithms.

 To develop adequate data mining and interpretation techniques
working on both the resulting designs/optimisation and the process
itself.


Towards “Dial a Pattern” in Complex Systems


Towards “Dial a Pattern” in Complex Systems

s e
ctur
Stru
ical S
Lex
.
teC
cre
rete

Dis
d
Disc

ute
st rib
Di
Continuous (simulated) CS
How do we program?
Disc
rete
/Con
tin. (
phys
ical)
CS

Dis
cre
te/C
ont
inu
os
(Bi
olo
gic
al)


Methodological Overview

Dial a Pattern requires:

 Parameter Learning/Evolution Technology

 Structural Learning/Evolution Technology

 Integrated Parameter/Structural Learning/Evolution Tech.

 “Plastic” algorithms to continuously self-improve (without
which scalability is an issue)


Datamining, Classification and Clustering
 For the last five years we have been working on the
application of LCS/GBML methods to large-scale
datasets
 Tumor Grade Classification for Microarrays Breast
Cancer Samples
 Pre-normalised data (log-scale, min:4.9, max: 13.3)
 128 samples and ~47000 genes

3 tumour grades
1(33),2(52),3(43)
majority
class classification = 40.6 accuracy
random classification (avg): 34.4% accuracy

Ben-Gurion University of the Negev - June 23rd to July 5th 2009 - Distinguished Scientist Visitor Program - Beer Sheva, Israel 10/41

 Goal = Dimensionality Reduction
 remove irrelevant genes, reduce complexity.
 2 basic approaches:
Foldchange/variance filtering
Gene Set Analysis

 Samples Clustering
 PCA, ICA
 Supervised Learning


Protein Structure

 Varying: size, shape, structure

 “Natures Robots”

 Structure determines their biological activity

 Understanding protein structure is key to
understanding function and dysfunction


Protein Structure Prediction (PSP) aims to predict the
3D structure of a protein based on its primary
sequence

Primary Sequence 3D Structure

Ben-Gurion University of the Negev - June 23rd to July 5th 2009 - Distinguished Scientist Visitor Program - Beer Sheva, Israel /41

Evolving Energy Potentials for
PSP


Prediction Scheme


Beside the overall 3D PSP, we can predict several
structural aspects of protein residues
•Coordination number
•Solvent accessibility
•Secondary structure
•Disulfide bonding
Accurate prediction of these features can help PSP in
many ways by:
•Constraining the conformation space
•Identifying better homolog proteins
These predictions can help research in other areas,
beside the main PSP problem
•Surface prediction
•Functional prediction

Coordination Number

Two residues of a chain are said to be in contact if their
distance is less than a certain threshold

Primary Contact Native State
Sequence

CN of a residue : count of contacts of a residue
CN gives us a simplified profile of the density of packing
of the protein


Recursive Convex Hull

•Structural feature that we have
proposed recently [Stout, Bacardit,
Hirst & Krasnogor, Bioinformatics
2008 24(7):916-923;]

•We model a protein as an onion,
assigning each residue to a different
layer of the onion, i.e., its convex
hull

•The convex hull of a point set is a
metric easy and fast to compute

•Recursive Convex Hull is computed
by iteratively identifying the layers
(hulls) of a protein


How to predict these features?

Two dimensions to decide
Inputs: What input information (derived from the
protein primary sequence) is used?
Outputs: How are we modelling the feature that we
are predicting?
Predicting the actual (continuous) feature
Predicting, for instance, buried or exposed
Discretization is applied to the original feature,
dividing it into 2, 3 or 5 states


Input information

Two types of input information
Local information: From the target residue and its
closest neighbours in the chain
Ri-5 Ri-4 Ri-3 Ri-2 Ri-1 Ri Ri+1 Ri+2 Ri+3 Ri+4 Ri+5
CNi-5 CNi-4 CNi-3 CNi-2 CNi-1 CNi CNi+1 CNi+2 CNi+3 CNi+4 CNi+5

Ri-1,Ri,Ri+1  CNi
Ri,Ri+1,Ri+2  CNi+1
Ri+1,Ri+2,Ri+3  CNi+2

Global information: From the whole chain we are
predicting


Size of the problem

Dataset characteristics:

•1050 protein chains
•~260000 instances
•In the most simple representation we may have
just 10-20 discrete attributes, but with high
cardinality (20 Amino Acids)
•Depending on the representation, hundreds of
continuous attributes


Protein Structure Comparison (PSC)

Similar or not?

How? Where
similar?

Knowing the similarity helps to:
1. Infer functional information
2. Organise (classify) all proteins
3. Design new proteins with specific function


Protein Structure Comparison (PSC)

Similar or not?

How? Where
similar?

Methods:
Knowing the similarity helps to: • USM

1. Infer functional information • MaxCMO
• DaliLite
2. Organise (classify) all proteins • CE

3. Design new proteins with specific function • FAST
• TM-Align
• …


PSC: Computation time per single pair

Method Algorithm/technique Measure Time
/metric [sec]
DaliLite Distance matrices, Combinatorial, simulated AL,Z, RMSD 3.33
annealing
MaxCMO Variable neighbourhood search (VNS) AL, OL 3.32

CE Heuristics, dynamic programming AL,Z, RMSD 1.27

USM Kolmogorov complexity USM-distance 0.34

TM-Align Rotation matrix, dynamic programming AL, RMSD,TMS 0.21

Fast Heuristics, dynamic programming RMSD, AL, SN 0.07

per pair of comparison


PDB Current Holdings Breakdown (May 12, 2009)

Protein/
Protein
Exp. Method Nucleic Acids NA Other Total
s
Complexes

X-ray 46071 1142 2118 17 49348

NMR 6844 850 144 7 7845

Electron Microscopy 163 16 59 0 238

Other 110 4 4 9 127

Total 53188 2012 2325 33 57558

Source: http://www.rcsb.org


PSC- Challenges
 Lack of single gold standard methods
 Need for Consensus Based Results

 Growth of structural data
 Currentholdings of PDB >53,000
 ~5000 new structures per year

 High-throughput requirements
 Need of more scalable techniques based on
distributed/grid computing architecture


Distribution: Problem space
All-against-all comparison of a dataset of P protein structures using m different
similarity comparison methods can be represented as 3D cube.

o ds
h
et
M
Heterogeneity:
1) Each structure has different length i.e
number of residues
2) Each method has different execution time
Structures

even for same pair of structures
3) Back-end computational nodes may have
different speeds etc

4) Each method has different measures
and metrics

Structures


Intelligent load balancing
strategies
o ds
h
et
M
Heterogeneity:
number of residues
Structures


and metrics

Structures


Intelligent load balancing
strategies
o ds
h
et
M
Heterogeneity:
number of residues
Structures


and metrics

Data standardization and
Structures normalization techniques

Distribution: Nomenclature
P Number of proteins

n Number of nodes (processors)

m Number of methods (e.g. FAST, USM, …)

Average size of proteins

Average time of all methods per single pair of comparison

Row_protx Number of row proteins present on node x

Col_protx Number of column proteins present on node x

Average execution time of all methods over all pairs of proteins stored on
node x


MC-PSC: Problem Complexity
 Job complexity:

Where, P =number of proteins and m = number of methods

 Space Complexity (number of data items in the output
matrix):

Where, Sc= space complexity, P= number of proteins, Nmt= total number of
measures/metrics and 2 makes home for two protein IDs for each pair.


MC-PSC: Problem Complexity
 Time complexity:
 Given a single P4 (1.86GHz) workstation and a set of
6 methods:

Target-against-all mode:
 i.ecomparison of all structures against one designated target
structure

All-against-all mode:
 i.e comparison of all structures against all structures


Distribution: PCAM technique

Source: Designing and Building Parallel Programs, by Ian Foster


Synthetic Biology
• Aims at designing, constructing and developing artificial biological systems

•Offers new routes to ‘genetically modified’ organisms, synthetic living
entities, smart drugs and hybrid computational-biological devices.

• Potentially enormous societal impact, e.g., healthcare, environmental
protection and remediation, etc

• Synthetic Biology's basic assumption:
• Methods commonly used to build non-biological systems could also
be use to specify, design, implement, verify, test and deploy novel
synthetic biosystems.
• These method come from computer science, engineering and maths.
• Modelling and optimisation run through all of the above.


InfoBiotics
www.infobiotic.net
•The utilisation of cutting-edge information
processing techniques for biological modelling and
synthesis
•The understanding of life itself as multi-scale
(Spatial/Temporal) information processing systems
•Composed of 3 key components:
•Executable Biology (or other modeling techniques)
•Automated Model and Parameter Estimation
•Model Checking (and other formal analysis)


Automated Model Synthesis and Optimisation

 Modeling is an intrinsically difficult process

 It involves “feature selection” and disambiguation

 Model Synthesis requires
 design the topology or structure of the system in
terms of molecular interactions
 estimate the kinetic parameters associated with
each molecular interaction

 All the above iterated


 Once a model has been prototyped,
whether derived from existing literature or
“ab initio” ➡ Use some optimisation
method to fine tune parameters/model
structure

 adopts an incremental methodology,
namely starting from very simple P system
modules (BioBricks) specifying basic
molecular interactions, more complicated
modules are produced to model more
complex molecular systems.


Large Literature on Model Synthesis
• Mason et al. use a random Local Search (LS) as the mutation to evolve
electronic networks with desired dynamics

• Chickarmane et al. use a standard GA to optimize the kinetic parameters of
a population of ODE-based reaction networks having the desired topology.

• Spieth et al. propose a Memetic Algorithm to ﬁnd gene regulatory networks
from experimental DNA microarray data where the network structure is
optimized with a GA and the parameters are optimized with an Evolution
Strategy (ES).

• Jaramillo et al. use Simulated Annealing as the main search strategy for
model inference based on (O)DEs


Evolutionary Algorithms for Automated
Model Synthesis and Optimisation
EA are potentially very useful for AMSO
 There’s a substantial amount of work on:
 using GP-like systems to evolve executable
structures
 using EAs for continuous/discrete optimisation
 An EA population represents alternative models
(could lead to different experimental setups)
 EAs have the potential to capture, rather than avoid,
evolvability of models


Methods
 Evolutionary Algorithm
 GAs
 GP

 Learning Classifier Systems

 Memetic Algorithms


Related Papers
 F. J. Romero-Campero, J. Twycross, M. Camara, M. Bennett, M. Gheorghe, and N. Krasnogor.
Modular assembly of cell systems biology models using p systems. International Journal of
Foundations of Computer Science, 2009
 J.Bacardit, M.Stout, J.D. Hirst, A.Valencia, R.E.Smith, and N.Krasnogor. Automated alphabet
reduction for protein datasets. BMC Bioinformatics, 10(6), 2009
 M.T. Oakley, D. Barthel, Y. Bykov, J.M. Garibaldi, E.K. Burke, N. Krasnogor, and J.D. Hirst. Search
strategies in structural bioinformatics. Current Protein and Peptide Science (Bentham Science
Publishers), 9(3):260-274, 2008
 M. Stout, J. Bacardit, J.D. Hirst, and N. Krasnogor. Prediction of recursive convex hull class
assignment for protein residues. Bioinformatics, 24(7):916-923, 2008
 M. Stout, J. Bacardit, J.D. Hirst, R.E Smith, and N. Krasnogor. Prediction of topological contacts in
proteins using learning classifier systems. Journal Soft Computing - A Fusion of Foundations,
Methodologies and Applications, 13(3):245-258, 2008.
 P.Siepmann, C.P. Martin, I. Vancea, P.J. Moriarty, and N. Krasnogor. A genetic algorithm approach
to probing the evolution of self-organised nanostructured systems. Nano Letters, 7(7):1985-1990,
2007
 G. Terrazas, P. Siepman, G. Kendal, and N. Krasnogor. An evolutionary methodology for the
automated design of cellular automaton-based complex systems. Journal of Cellular Automata,
2(1):77-102, 2007
 N. Krasnogor and J.E. Smith. A tutorial for competent memetic algorithms: model, taxonomy and
design issues. IEEE Transactions on Evolutionary Computation, 9(5):474- 488, 2005.


Introductory Talk at Ben Gurion University

More Related Content

What's hot

Similar to Introductory Talk at Ben Gurion University

More from Natalio Krasnogor

Recently uploaded

Introductory Talk at Ben Gurion University