This was a very brief introduction to the basics of learning algorithms for life scientists I was asked to give to the incoming first year students at TSRI in the fall of 2005. It covers the very basics of how the algorithms work (sans the complex math) and more importantly, how they can be appropriately understood and applied by chemists and biologists.

1. Artificial Intelligence and Learning Algorithms Presented By Brian M. Frezza 12/1/05

3. Hard Math

10. Bayesian Network: Trace H a : 100% Redhead H b : 50% Redhead 50% Not H c : 100% Not Redhead 0 Not 0 Hypothesis History Likelihood's = P( red |H a )*P(H a ) + P( red |H b )*P(H b ) + P( red |H c )*P(H c ) = (1)*(1/3) + (1/2)*(1/3) + (0)(1/3) =(1/2) Prediction: Will their next kid be a Redhead ? 1/3 1/3 1/3 P(H c ) P(H b ) P(H a )

11. Bayesian Network:Trace H a : 100% Redhead H b : 50% Redhead 50% Not H c : 100% Not Redhead 1 Not 0 Hypothesis History Likelihood's = P( red |H a )*P(H a ) + P( red |H b )*P(H b ) + P( red |H c )*P(H c ) = (1)*(1/2) + (1/2)*(1/2) + (0)(1/3) =(3/4) Prediction: Will their next kid be a Redhead ? 0 1/2 1/2 P(H c ) P(H b ) P(H a )

12. Bayesian Network: Trace H a : 100% Redhead H b : 50% Redhead 50% Not H c : 100% Not Redhead 2 Not 0 Hypothesis History Likelihood's = P( red |H a )*P(H a ) + P( red |H b )*P(H b ) + P( red |H c )*P(H c ) = (1)*(3/4) + (1/2)*(1/4) + (0)(1/3) =(7/8) Prediction: Will their next kid be a Redhead ? 0 1/4 3/4 P(H c ) P(H b ) P(H a )

13. Bayesian Network: Trace H a : 100% Redhead H b : 50% Redhead 50% Not H c : 100% Not Redhead 3 Not 0 Hypothesis History Likelihood's = P( red |H a )*P(H a ) + P( red |H b )*P(H b ) + P( red |H c )*P(H c ) = (1)*(7/8) + (1/2)*(1/8) + (0)(1/3) =(15/16) Prediction: Will their next kid be a Redhead ? 0 1/8 7/8 P(H c ) P(H b ) P(H a )

29. Viterbi Algorithm: Trace Hidden State Transition Probabilities Observable State Probabilities To From Hidden State Observable Starting Distribution Example Sequence: ATAATGGCGAGTG Exon = P(A|Ex) * Start Exon = 3.3*10 -2 Introgenic = P(A|Ig) * Start Ig = 2.2*10 -1 Intron = P(A|It) * Start It = 0.14 * 0.01 = 1.4*10 -3 0.8 0.02 0.18 It 0.01 0.9 0.09 Ig 0.2 0.1 0.7 Ex It Ig Ex 0.2 0.5 0.16 0.14 It 0.25 0.25 0.25 0.25 Ig 0.14 0.11 0.42 0.33 Ex C G T A 0.01 0.89 0.1 It Ig Ex G T A G A G C G G T A A T 1.4*10 -3 2.2*10 -1 3.3*10 -2 A Intron Introgenic Exon

30. Viterbi Algorithm: Trace Hidden State Transition Probabilities Observable State Probabilities To From Hidden State Observable Starting Distribution Example Sequence: ATAATGGCGAGTG Exon = Max( P(Ex|Ex)*P n-1 (Ex), P(Ex|Ig)*P n-1 (Ig), P(Ex|It)*P n-1 (It) ) *P(T|Ex) = 4.6*10 -2 Introgenic =Max( P(Ig|Ex)*P n-1 (Ex), P(Ig|Ig)*P n-1 (Ig), P(Ig|It)*P n-1 (It) ) * P(T|Ig) = 2.8*10 -2 Intron = Max( P(It|Ex)*P n-1 (Ex), P(It|Ig)*P n-1 (Ig), P(It,It)*P n-1 (It) ) * P(T|It) = 1.1*10 -3 0.8 0.02 0.18 It 0.01 0.5 0.49 Ig 0.2 0.1 0.7 Ex It Ig Ex 0.2 0.5 0.16 0.14 It 0.25 0.25 0.25 0.25 Ig 0.14 0.11 0.42 0.33 Ex C G T A 0.01 0.89 0.1 It Ig Ex G T A G A G C G G T A A 1.1*10 -3 2.8*10 -2 4.6*10 -2 T 1.4*10 -3 2.2*10 -1 3.3*10 -2 A Intron Introgenic Exon

31. Viterbi Algorithm: Trace Hidden State Transition Probabilities Observable State Probabilities To From Hidden State Observable Starting Distribution Example Sequence: ATAATGGCGAGTG Exon = Max( P(Ex|Ex)*P n-1 (Ex), P(Ex|Ig)*P n-1 (Ig), P(Ex|It)*P n-1 (It) ) *P(T|Ex) = 1.1*10 -2 Introgenic =Max( P(Ig|Ex)*P n-1 (Ex), P(Ig|Ig)*P n-1 (Ig), P(Ig|It)*P n-1 (It) ) * P(T|Ig) = 3.5*10 -3 Intron = Max( P(It|Ex)*P n-1 (Ex), P(It|Ig)*P n-1 (Ig), P(It,It)*P n-1 (It) ) * P(T|It) = 1.3*10 -3 0.8 0.02 0.18 It 0.01 0.5 0.49 Ig 0.2 0.1 0.7 Ex It Ig Ex 0.2 0.5 0.16 0.14 It 0.25 0.25 0.25 0.25 Ig 0.14 0.11 0.42 0.33 Ex C G T A 0.01 0.89 0.1 It Ig Ex G T A G A G C G G T A 1.3*10 -3 3.5*10 -3 1.1*10 -2 A 1.1*10 -3 2.8*10 -2 4.6*10 -2 T 1.4*10 -3 2.2*10 -1 3.3*10 -2 A Intron Introgenic Exon

32. Viterbi Algorithm: Trace Hidden State Transition Probabilities Observable State Probabilities To From Hidden State Observable Starting Distribution Example Sequence: ATAATGGCGAGTG Exon = Max( P(Ex|Ex)*P n-1 (Ex), P(Ex|Ig)*P n-1 (Ig), P(Ex|It)*P n-1 (It) ) *P(T|Ex) Introgenic =Max( P(Ig|Ex)*P n-1 (Ex), P(Ig|Ig)*P n-1 (Ig), P(Ig|It)*P n-1 (It) ) * P(T|Ig) Intron = Max( P(It|Ex)*P n-1 (Ex), P(It|Ig)*P n-1 (Ig), P(It,It)*P n-1 (It) ) * P(T|It) 0.8 0.02 0.18 It 0.01 0.5 0.49 Ig 0.2 0.1 0.7 Ex It Ig Ex 0.2 0.5 0.16 0.14 It 0.25 0.25 0.25 0.25 Ig 0.14 0.11 0.42 0.33 Ex C G T A 0.01 0.89 0.1 It Ig Ex G T A G A G C G G T 2.9*10 -4 4.3*10 -4 2.4*10 -3 A 1.3*10 -3 3.5*10 -3 1.1*10 -2 A 1.1*10 -3 2.8*10 -2 4.6*10 -2 T 1.4*10 -3 2.2*10 -1 3.3*10 -2 A Intron Introgenic Exon

33. Viterbi Algorithm: Trace Hidden State Transition Probabilities Observable State Probabilities To From Hidden State Observable Starting Distribution Example Sequence: ATAATGGCGAGTG Exon = Max( P(Ex|Ex)*P n-1 (Ex), P(Ex|Ig)*P n-1 (Ig), P(Ex|It)*P n-1 (It) ) *P(T|Ex) Introgenic =Max( P(Ig|Ex)*P n-1 (Ex), P(Ig|Ig)*P n-1 (Ig), P(Ig|It)*P n-1 (It) ) * P(T|Ig) Intron = Max( P(It|Ex)*P n-1 (Ex), P(It|Ig)*P n-1 (Ig), P(It,It)*P n-1 (It) ) * P(T|It) 0.8 0.02 0.18 It 0.01 0.5 0.49 Ig 0.2 0.1 0.7 Ex It Ig Ex 0.2 0.5 0.16 0.14 It 0.25 0.25 0.25 0.25 Ig 0.14 0.11 0.42 0.33 Ex C G T A 0.01 0.89 0.1 It Ig Ex G T A G A G C G G 7.8*10 -5 6.1*10 -5 7.2*10 -4 T 2.9*10 -4 4.3*10 -4 2.4*10 -3 A 1.3*10 -3 3.5*10 -3 1.1*10 -2 A 1.1*10 -3 2.8*10 -2 4.6*10 -2 T 1.4*10 -3 2.2*10 -1 3.3*10 -2 A Intron Introgenic Exon

34. Viterbi Algorithm: Trace Hidden State Transition Probabilities Observable State Probabilities To From Hidden State Observable Starting Distribution Example Sequence: ATAATGGCGAGTG Exon = Max( P(Ex|Ex)*P n-1 (Ex), P(Ex|Ig)*P n-1 (Ig), P(Ex|It)*P n-1 (It) ) *P(T|Ex) Introgenic =Max( P(Ig|Ex)*P n-1 (Ex), P(Ig|Ig)*P n-1 (Ig), P(Ig|It)*P n-1 (It) ) * P(T|Ig) Intron = Max( P(It|Ex)*P n-1 (Ex), P(It|Ig)*P n-1 (Ig), P(It,It)*P n-1 (It) ) * P(T|It) 0.8 0.02 0.18 It 0.01 0.5 0.49 Ig 0.2 0.1 0.7 Ex It Ig Ex 0.2 0.5 0.16 0.14 It 0.25 0.25 0.25 0.25 Ig 0.14 0.11 0.42 0.33 Ex C G T A 0.01 0.89 0.1 It Ig Ex G T A G A G C G 7.2*10 -5 1.8*10 -5 5.5*10 -5 G 7.8*10 -5 6.1*10 -5 7.2*10 -4 T 2.9*10 -4 4.3*10 -4 2.4*10 -3 A 1.3*10 -3 3.5*10 -3 1.1*10 -2 A 1.1*10 -3 2.8*10 -2 4.6*10 -2 T 1.4*10 -3 2.2*10 -1 3.3*10 -2 A Intron Introgenic Exon

35. Viterbi Algorithm: Trace Hidden State Transition Probabilities Observable State Probabilities To From Hidden State Observable Starting Distribution Example Sequence: ATAATGGCGAGTG Exon = Max( P(Ex|Ex)*P n-1 (Ex), P(Ex|Ig)*P n-1 (Ig), P(Ex|It)*P n-1 (It) ) *P(T|Ex) Introgenic =Max( P(Ig|Ex)*P n-1 (Ex), P(Ig|Ig)*P n-1 (Ig), P(Ig|It)*P n-1 (It) ) * P(T|Ig) Intron = Max( P(It|Ex)*P n-1 (Ex), P(It|Ig)*P n-1 (Ig), P(It,It)*P n-1 (It) ) * P(T|It) 0.8 0.02 0.18 It 0.01 0.5 0.49 Ig 0.2 0.1 0.7 Ex It Ig Ex 0.2 0.5 0.16 0.14 It 0.25 0.25 0.25 0.25 Ig 0.14 0.11 0.42 0.33 Ex C G T A 0.01 0.89 0.1 It Ig Ex G T A G A G C 2.9*10 -5 2.2*10 -6 4.3*10 -6 G 7.2*10 -5 1.8*10 -5 5.5*10 -5 G 7.8*10 -5 6.1*10 -5 7.2*10 -4 T 2.9*10 -4 4.3*10 -4 2.4*10 -3 A 1.3*10 -3 3.5*10 -3 1.1*10 -2 A 1.1*10 -3 2.8*10 -2 4.6*10 -2 T 1.4*10 -3 2.2*10 -1 3.3*10 -2 A Intron Introgenic Exon

36. Viterbi Algorithm: Trace Hidden State Transition Probabilities Observable State Probabilities To From Hidden State Observable Starting Distribution Example Sequence: ATAATGGCGAGTG Exon = Max( P(Ex|Ex)*P n-1 (Ex), P(Ex|Ig)*P n-1 (Ig), P(Ex|It)*P n-1 (It) ) *P(T|Ex) Introgenic =Max( P(Ig|Ex)*P n-1 (Ex), P(Ig|Ig)*P n-1 (Ig), P(Ig|It)*P n-1 (It) ) * P(T|Ig) Intron = Max( P(It|Ex)*P n-1 (Ex), P(It|Ig)*P n-1 (Ig), P(It,It)*P n-1 (It) ) * P(T|It) 0.8 0.02 0.18 It 0.01 0.5 0.49 Ig 0.2 0.1 0.7 Ex It Ig Ex 0.2 0.5 0.16 0.14 It 0.25 0.25 0.25 0.25 Ig 0.14 0.11 0.42 0.33 Ex C G T A 0.01 0.89 0.1 It Ig Ex 4.7*10 -10 3.6*10 -11 1.1*10 -10 G 1.2*10 -9 1.2*10 -10 1.4*10 -9 T 9.2*10 -9 4.1*10 -10 4.9* -9 A 8.2*10 -8 2.7*10 -9 8.4* -9 G 2.0*10 -7 9.1*10 -9 1.1*10 -7 A 1.8*10 -6 3.5*10 -8 9.1*10 -8 G 4.6*10 -6 2.8*10 -7 7.2*10 -7 C 2.9*10 -5 2.2*10 -6 4.3*10 -6 G 7.2*10 -5 1.8*10 -5 5.5*10 -5 G 7.8*10 -5 6.1*10 -5 7.2*10 -4 T 2.9*10 -4 4.3*10 -4 2.4*10 -3 A 1.3*10 -3 3.5*10 -3 1.1*10 -2 A 1.1*10 -3 2.8*10 -2 4.6*10 -2 T 1.4*10 -3 2.2*10 -1 3.3*10 -2 A Intron Introgenic Exon

44. Neural Networks: A Neuron, Node, or Unit Σ ( W )- W 0,c Activation Function Output W a,c W b,c W 0,c (Bias) W c, n a z (Bias)

45. Neural Networks: Activation Functions Sigmoid Function (logistic function) Threshold Function Zero point set by bias In In out out +1 +1

46. Threshold Functions can make Logic Gates with Neurons! Logical And W 0,c = 1.5 W b,c = 1 W a,c = 1 A B Σ ( W )- W 0,c a z (Bias) Output If ( Σ (w) – W o,c > 0 ) Then FIRE Else Don’t (Bias) 0 0 0 0 1 1 0 1 ∩

47. And Gate: Trace W 0,c = 1.5 W b,c = 1 W a,c = 1 -1.5 Off Off Off -1.5 < 0 (Bias)

48. And Gate: Trace W 0,c = 1.5 W b,c = 1 W a,c = 1 -0.5 On Off Off -0.5 < 0 (Bias)

49. And Gate: Trace W 0,c = 1.5 W b,c = 1 W a,c = 1 -0.5 Off On Off -0.5 < 0 (Bias)

50. And Gate: Trace W 0,c = 1.5 W b,c = 1 W a,c = 1 0.5 On On On 0.5 > 0 (Bias)

51. Threshold Functions can make Logic Gates with Neurons! W 0,c = 0.5 W b,c = 1 W a,c = 1 A Σ ( W )- W 0,c a z (Bias) If ( Σ (w) – W o,c > 0 ) Then FIRE Else Don’t (Bias) Logical Or B 0 1 0 1 1 1 0 1 U

52. Or Gate: Trace W 0,c = 0.5 W b,c = 1 W a,c = 1 -0.5 Off Off Off -0.5 < 0 (Bias)

53. Or Gate: Trace W 0,c = 0.5 W b,c = 1 W a,c = 1 0.5 On Off On 0.5 > 0 (Bias)

54. Or Gate: Trace W 0,c = 0.5 W b,c = 1 W a,c = 1 0.5 Off On On 0.5 > 0 (Bias)

55. Or Gate: Trace W 0,c = 0.5 W b,c = 1 W a,c = 1 1.5 On On On 1.5 > 0 (Bias)

56. Threshold Functions can make Logic Gates with Neurons! W 0,c = -0.5 W a,c = -1 Σ ( W )- W 0,c a z (Bias) If ( Σ (w) – W o,c > 0 ) Then FIRE Else Don’t (Bias) Logical Not 1 0 0 1 !

57. Not Gate: Trace W 0,c = -0.5 W a,c = -1 -0.5 Off On 0.5 > 0 (Bias) 0 – (-0.5) = 0.5

58. Not Gate: Trace W 0,c = -0.5 W a,c = -1 -0.5 On Off -0.5 < 0 (Bias) -1 – (-0.5) = -0.5

61. Layers Input Output Hidden layer