part A

Development of statistical theory and
algorithms for analyzing molecular data
Paula Tataru
Qualification
exam

Paula Tataru Qualification exam 2
Outline
●
Published work
o SCFGs & RNA secondary structure prediction
o Expectations for CTMCs
●
Hidden Markov models (HMMs)
●
Patterns & HMMs
●
HMMs for inferring population parameters
●
Future work

Statistical theory
Markov chains
Discrete time Markov chains
Continuous time Markov chains
Hidden Markov models
Stochastic context free grammars

Outline
●
Published work
o SCFGs for RNA secondary structure prediction
●
●
Patterns & HMMs
●
●
Future work

SCFGs & RNA secondary structure
●
Use SCFG to predict RNA secondary structure
●
KH99 best existing SCFG
●
Find a SCFG that is just as good/better

6Paula Tataru Qualification exam

Expectations for CTMCs
●
Calculate expectations for CTMCs
o Time spent in a state
o Changes between any two states
●
Three approaches
o Eigenvalue decomposition (EVD)
o Uniformization (UNI)
o Matrix exponential (EXPM)
●
Which is more accurate?
●
Which is fastest?
A A G

Outline
●
Published work
●
●
Patterns & HMMs
●
●
Future work

●
Observables
●
Hidden states
●
Initial state probabilities
●
Transition probabilities
●
Emission probabilities

Algorithms
●
Given a sequence of observations
●
Forward algorithm
o What is the likelihood of the observations?
●
Viterbi algorithm
o What is the most likely hidden explanation ?
... A T G G C C T A AT C G T ...
... C
1
C
2
C
3
C
1
C
2
C
3
C
1
C
2
C
3
N N N N ...

Applications of HMMs
●
Gene annotation (GeneMark, GeneScan)
●
Protein structure modeling (Phobius, SignalP)
●
Sequence alignment (HMMER, SAM)
●
Phylogenetic analysis (PhyloHMM, CoalHMM)

Outline
●
Published work
●
●
Patterns & HMMs
●
●
Future work

Patterns & HMMs
●
Patterns in hidden explanation are of interest
●
How many times does one pattern occur?
o restricted forward algorithm
●
What is the most likely hidden explanation containing
the pattern a number of times?
o restricted Viterbi algorithm

Patterns & HMMs
●
Keep track of the occurrences of the pattern r
●
Use a deterministic finite automaton (DFA)
●
Each state q encodes how much of the pattern r has
already been seen
●
Run the HMM and the DFA in parallel

Restricted forward
●
Forward algorithm
o
o the likelihood of the observations

Restricted forward
●
Forward algorithm
o
●
Restricted forward algorithm
o
o the distribution for the number of pattern occurrences

●
Simple gene finder
●
● Pattern r = (NN(C1
|R3
)) | ((C3
|R1
)NN)
●
Generated observed and hidden sequence of length 500

●
Simple gene finder
●
|R3
)) | ((C3
|R1
)NN)
●
Generated observed and hidden sequence of length 500

Restricted Viterbi
●
Viterbi algorithm
o
o the most likely hidden explanation

Restricted Viterbi
●
Viterbi algorithm
o
●
Restricted Viterbi algorithm
o
o the most likely hidden explanation containing the pattern a
certain number of times

Results
●
Simple gene finder
|R3
)) | ((C3
|R1
)NN)
●
Generated 100 pairs of observed and hidden
sequences of length 500, 525, …, 1500

Future work
●
Apply on real model and data (GeneTack)
●
●
Incorporate waiting time distribution
o obtained from the restricted forward algorithm
o include in the restricted Viterbi algorithm
●
●
Improve memory consumption

Outline
●
Published work
●
●
Patterns & HMMs
●
●
Future work

PSMC

PSMC
●
Relies on ancestral recombinations
●
Coalescent with recombination theory
●
Sequential Markov chain (SMC)
1 1 1 12 2 2 23 3 3 3

CoalHMM
●
Observables: nucleotides (DNA sequences)
●
Hidden states: coalescence trees
●
PSMC
o two sequences from the same population (or individual)
o coalescence trees are uniquely determined by TMRCA
… G T C T G A C …
… G A C T G C C …
T A C CG G T T G G A C C C

Inferring population parameters
●
Calculate likelihood of the data (forward algorithm)
●
Find parameters that give best likelihood
o population size back in time
o recombination rate

CoalHMM
●
Discretized time in K+1 intervals
o [0, t1
), [t1
, t2
), …, [tK
, ∞)
●
Apply on more than two sequences (same population)
o constrain coalescence events
1 2 3 4 5 1 2 3 4 5

CoalHMM
●
Notation,

CoalHMM
●
Notation,
●
Probability

Future work
●
Finalize implementing and testing the CoalHMM
●
Apply on real data
o low coverage NGS data
●
Investigate time discretization
●
Extend to more than one population
o isolation model
o isolation with migration model

Outline
●
Published work
●
●
Patterns & HMMs
●
●
Future work

Future work – concrete plans
●
Proceed with patterns & HMMs
o real model and data (GeneTack)
o waiting time distribution / space consumption
●
Finalize & extend CoalHMM
o apply on data
o extend to multiple populations
●
Stay abroad (January – June 2013)
o UC Berkeley, prof. Yun S. Song
o similar work on the sequential Markov chain

Thank you!

Deterministic finite automaton

Restricted forward

Development of statistical theory and
algorithms for analyzing molecular data
Paula Tataru
Qualification
exam
Previous 2½ years
Coming 2 years
Title of thesis

PaulaTataruQualificationexam2Outline●PublishedworkoSCFGs&RNAsecondarystructurepredictionoExpectationsforCTMCs●HiddenMarkovmodels(HMMs)●Patterns&HMMs●HMMsforinferringpopulationparameters●Futurework
Quick overlook

Statistical theory
Markov chains
Discrete time Markov chains
Continuous time Markov chains
Stochastic context free grammars
PhD project → statistical theory
All projects → Markov chains

PaulaTataruQualificationexam4Outline●PublishedworkoSCFGsforRNAsecondarystructurepredictionoExpectationsforCTMCs●HiddenMarkovmodels(HMMs)●Patterns&HMMs●HMMsforinferringpopulationparameters●Futurework
Present briefly

SCFGs & RNA secondary structure
●
Use SCFG to predict RNA secondary structure
●
KH99 best existing SCFG
●
Find a SCFG that is just as good/better
Automated search technique
Two selected grammars
Area under the graph

6PaulaTataruQualificationexam
Collaboration with a research group from Oxford University
Published this year in BMC Bioinformatics

Expectations for CTMCs
●
Calculate expectations for CTMCs
o Time spent in a state
o Changes between any two states
●
Three approaches
o Eigenvalue decomposition (EVD)
o Uniformization (UNI)
o Matrix exponential (EXPM)
●
Which is more accurate?
●
Which is fastest?
A A G
CTMCs → describe DNA evolution
State → nucleotides
Inference → certain expectations

Normalized diff
Similar accuracy for all models
EXPM slowest

Asger's Hobolth supervision
Last year BMC Bioinformatics

Basis of current work

●
Observables
●
Hidden states
●
Initial state probabilities
●
Transition probabilities
●
Emission probabilities
Sequential data with underlying hidden structure
Model for finding genes
Relate text to figure

Algorithms
●
Given a sequence of observations
●
Forward algorithm
o What is the likelihood of the observations?
●
Viterbi algorithm
o What is the most likely hidden explanation ?
... A T G G C C T A AT C G T ...
... C1
C2
C3
C1
C2
C3
C1
C2
C3
N N N N ...

Patterns & HMMs
●
Patterns in hidden explanation are of interest
●
How many times does one pattern occur?
o restricted forward algorithm
●
What is the most likely hidden explanation containing
the pattern a number of times?
o restricted Viterbi algorithm
Andreas Sand
Pattern → gene
Number of genes
Use Viterbi → very bad, no measure of certainty
Find distribution
Incorporate in prediction

Patterns & HMMs
●
Keep track of the occurrences of the pattern r
●
Use a deterministic finite automaton (DFA)
●
Each state q encodes how much of the pattern r has
already been seen
●
Run the HMM and the DFA in parallel
DFA → graph
Nodes → states

Restricted forward
●
Forward algorithm
o
2D matrix
Column → Observed up to time t
Hidden state xt

Restricted forward
●
Forward algorithm
o
●
Restricted forward algorithm
o
o the distribution for the number of pattern occurrences
4D matrix
Column → Observed up to time t
Hidden state xt
K patterns
State q

19PaulaTataruQualificationexam●Simplegenefinder●●Patternr=(NN(C1|R3))|((C3|R1)NN)●Generatedobservedandhiddensequenceoflength500
Start and end of genes

20PaulaTataruQualificationexam●Simplegenefinder●●Patternr=(NN(C1|R3))|((C3|R1)NN)●Generatedobservedandhiddensequenceoflength500
Distribution
Expectation
95% interval
Viterbi
True number
Improve Viterbi!

Restricted Viterbi
●
Viterbi algorithm
o
2D matrix
Observed up to time t
Hidden state xt

Restricted Viterbi
●
Viterbi algorithm
o
●
Restricted Viterbi algorithm
o
o the most likely hidden explanation containing the pattern a
certain number of times
4D matrix
Observed up to time t
Hidden state xt
K patterns
State q

Recover true number of patterns
Expectation → very good
Viterbi → very bad

PSMC
Use HMM for human population history

PSMC
Population size back in time

PSMC
●
Relies on ancestral recombinations
●
Coalescent with recombination theory
●
Sequential Markov chain (SMC)
1 1 1 12 2 2 23 3 3 3
Moves along the sequence
Each position → one tree
Shift in tree → recombination

Inferring population parameters
●
Calculate likelihood of the data (forward algorithm)
●
Find parameters that give best likelihood
o population size back in time
o recombination rate
Parameters determine event probabilities

CoalHMM
●
Discretized time in K+1 intervals
o [0, t1
), [t1
, t2
), …, [tK
, ∞)
●
Apply on more than two sequences (same population)
o constrain coalescence events
1 2 3 4 5 1 2 3 4 5
Infinite time

CoalHMM
●
Notation,
What is x

CoalHMM
●
Notation,
●
Probability
Sum over all possible paths
Gives probability distribution from moving from certain tree to all other trees

2 seq, fixed pop → analytically
2 seq, vary pop
3 seq → MaCS
KL → smaller, better
Compare actual distributions

40PaulaTataruQualificationexamThankyou!

part A

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (12)

Similar to part A

Similar to part A (20)

More from Paula Tataru

More from Paula Tataru (9)

part A