Applying Hidden Markov Models to Bioinformatics

4,771 views

Published on

1 Comment
3 Likes
Statistics
Notes
  • Could you plz send me the link where i can download the paper of these slides?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
4,771
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
223
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide

Applying Hidden Markov Models to Bioinformatics

  1. 1. Applying Hidden Markov Models to Bioinformatics<br />Conor Buckley<br />
  2. 2. Outline<br />What are Hidden Markov Models?<br />Why are they a good tool for Bioinformatics?<br />Applications in Bioinformatics<br />
  3. 3. History of Hidden Markov Models<br />HMM were first described in a series of statistical papers by Leonard E. Baum and other authors in the second half of the 1960s. One of the first applications of HMMs was speech recogniation, starting in the mid-1970s. They are commonly used in speech recognition systems to help to determine the words represented by the sound wave forms captured<br />In the second half of the 1980s, HMMs began to be applied to the analysis of biological sequences, in particular DNA.<br />Since then, they have become ubiquitous in bioinformatics<br />Source: http://en.wikipedia.org/wiki/Hidden_Markov_model#History<br />
  4. 4. What are Hidden Markov Models?<br />HMM: A formal foundation for making probabilistic models of linear sequence 'labeling' problems.<br />They provide a conceptual toolkit for building complex models just by drawing an intuitive picture.<br />Source: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  5. 5. What are Hidden Markov Models?<br />Machine learning approach in bioinformatics<br />Machine learning algorithms are presented with training data, which are used to derive important insights about the (often hidden) parameters.<br />Once an algorithm has been trained, it can apply these insights to the analysis of a test sample<br />As the amount of training data increases, the accuracy of the machine learning algorithm typically increasess as well.<br />Source: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  6. 6. Hidden Markov Models<br />Has N states, called S1, S2, ... Sn<br />There are discrete timesteps, t=0, t=1<br />S2<br />N = 3<br />t = 0<br />S1<br />S3<br />Source: http://www.autonlab.org/tutorials/hmm.html<br />
  7. 7. Hidden Markov Models<br />Has N states, called S1, S2, ... Sn<br />There are discrete timesteps, t=0, t=1<br />For each timestep, the system is in exactly one of the available states.<br />S2<br />N = 3<br />t = 0<br />S1<br />S3<br />
  8. 8. Hidden Markov Models<br />S1<br />S2<br />S3<br /><ul><li>Bayesian network with time slices</li></ul>Bayesian Network Image: http://en.wikipedia.org/wiki/File:Hmm_temporal_bayesian_net.svg<br />
  9. 9. A Markov Chain<br />Bayes'Theory<br /><ul><li>(statistics) a theorem describing how the conditional probability of a set of possible causes for a given observed event can be computed from knowledge of the probability of each cause and the conditional probability of the outcome of each cause</li></ul>- http://wordnetweb.princeton.edu/perl/webwn?s=bayes%27%20theorem<br />
  10. 10. Building a Markov Chain<br />Concrete Example<br />Two friends, Alice and Bob, who live far apart from each other and who talk together daily over the telephone about what they did that day.<br />Bob is only interested in three activities: walking in the park, shopping, and cleaning his apartment.<br />The choice of what to do is determined exclusively by the weather on a given day. <br />Alice has no definite information about the weather where Bob lives, but she knows general trends.<br />Based on what Bob tells her he did each day, Alice tries to guess what the weather must have been like.<br />Alice believes that the weather operates as a discrete Markov chain. There are two states, "Rainy" and "Sunny", but she cannot observe them directly, that is, they are hidden from her.<br />On each day, there is a certain chance that Bob will perform one of the following activities, depending on the weather: "walk", "shop", or "clean". Since Bob tells Alice about his activities, those are the observations. <br />Source: Wikipedia.org<br />
  11. 11. Hidden Markov Models<br />
  12. 12. Building a Markov Chain<br />
  13. 13. What now?<br />* Find out the most probable output sequence<br />Vertibi's algorithm<br />Dynamic programming algorithm for finding the most likely sequence of hidden states – called the Vertibi path – that results in a sequence of observed events.<br />
  14. 14. Vertibi Results<br />http://pcarvalho.com/forward_viterbi/<br />
  15. 15. Bioinformatics Example<br />Assume we are given a DNA sequence that begins in an exon, contains one 5' splice site and ends in an intron<br />Identify where the switch from exon to intron occurs<br />Where is the splice site??<br />Sourece: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  16. 16. Bioinformatics Example<br />In order for us to guess, the sequences of exons, splice sites and introns must have different statistical properties.<br />Let's say...<br />Exons have a uniform base composition on average<br />A/C/T/G: 25% for each base<br />Introns are A/T rich<br />A/T: 40% for each <br />C/G: 10% for each<br />5' Splice site consensus nucleotide is almost always a G...<br />G: 95%<br />A: 5%<br />Sourece: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  17. 17. Bioinformatics Example<br />We can build an Hidden Markov Model<br />We have three states<br />"E" for Exon<br />"5" for 5' SS<br />"I" for Intron<br />Each State has its own emission probabilities which model the base composition of exons, introns and consensus G at the 5'SS<br />Each state also has transition probabilities (arrows)<br />Sourece: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  18. 18. HMM: A Bioinformatics Visual<br />We can use HMMs to generate a sequence<br />When we visit a state, we emit a nucleotide bases on the emission probability distribution<br />We also choose a state to visit next according to the state's transition probability distribution.<br /><ul><li>We generate two strings of information
  19. 19. Observed Sequence
  20. 20. Underlying State Path</li></ul>Source: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  21. 21. HMM: A Bioinformatics Visual<br />The state path is a Markov Chain<br />Since we're only given the observed sequence, this underlying state path is a hidden Markov Chain<br />Therefore...<br />We can apply Bayesian Probability<br />Source: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  22. 22. HMM: A Bioinformatics Visual<br />S – Observed sequence<br />π – State Path<br />Θ – Parameters<br />The probability P(S, π|HMM, Θ) is the product of all emission probabilites and transition probilities.<br />Lets look at an example...<br />Source: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  23. 23. HMM: A Bioinformatics Visual<br />There are 27 transitions and 26 emissions.<br />Multiply all 53 probabilities together (and take the log, since these are small numbers) and you'll calculate log P(S, π|HMM, Θ) = -41.22<br />Source: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  24. 24. HMM: A Bioinformatics Visual<br />The model parameters and overall sequences scores are all probabilities<br />Therefore we can use Bayesian probability theory to manipulate these numbers in standard, powerful ways, including optimizing parameters and interpreting the signifigance of scores.<br />Source: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  25. 25. HMM: A Bioinformatics Visual<br />Posterior Decoding:<br />An alternative state path where the SS falls on the 6th G instead of the 5th (log probabilities of -41.71 versus -41.22)<br />How confident are we that the fifth G is the right choice?<br />Source: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  26. 26. HMM: A Bioinformatics Visual<br />We can calculate our confidence directly.<br />The probability that nucleotide i was emitted by state k is the sum of the probabilities of all the states paths use state k to generate i, normalized by the sum over all possible state paths<br />Result: We get a probability of 46% that the best-scoring fifth G is correct and 28% that the sixth G position is correct.<br />Source: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  27. 27. Further Possibilites<br />The toy-model provided by the article is a simple example<br />But we can go further, we could add a more realistic consensus GTRAGT at the 5' splice site<br />We could put a row of six HMM states in place of '5' state to model a six-base ungapped consensus motif<br />Possibilities are not limited<br />
  28. 28. The catch<br />HMM don't deal well with correlations between nucleotides<br />Because they assume that each emitted nucleotide depends only on one underlying state.<br />Example of bad use for HMM:<br />Conserved RNA base pairs which induce long-range pairwise correlations; one position might be any nucleotide but the base-paired partner must be complementary.<br />An HMM state path has no way of 'remembering' what a distant state generated.<br />Source: http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />
  29. 29. Credits<br />http://www.nature.com/nbt/journal/v22/n10/full/nbt1004-1315.html#B1<br />http://en.wikipedia.org/wiki/Viterbi_algorithm<br />http://en.wikipedia.org/wiki/Hidden_Markov_model<br />http://en.wikipedia.org/wiki/Bayesian_network<br />http://www.daimi.au.dk/~bromille/PHM/Storm.pdf<br />
  30. 30. Questions?<br />

×