Upcoming SlideShare
×

# Ukk's Algorithm of Suffix Tree

610 views
531 views

Published on

1 Like
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

Views
Total views
610
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
14
0
Likes
1
Embeds 0
No embeds

No notes for slide

### Ukk's Algorithm of Suffix Tree

1. 1. . Algorithm of Sufﬁx Tree by farseerfc@gmail.com. Jiachen Yang1 1 Research Student in Osaka University November 12, 2011 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 1 / 25
2. 2. Part Outline.1 What is Sufﬁx Tree.2 History & Naïve Algorithm.3 Optimization of Naïve Algorithm.4 Examples & Analysis . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 2 / 25
3. 3. What can we do with sufﬁx tree?. Find properties of the stringsLinear algorithms for exact string . Find the longest common substrings of the 1matching string Si and Sj in θ (ni + nj ) time . like KMP . Find all maximal pairs, maximal repeats or 2Search for strings supermaximal repeats in θ (n + z) time, if there . Check if a string P of length m is 1 are z such repeats . . Find the Lempel-Ziv decomposition in θ (n) 3 a substring in O(m) time. . Find all z occurrences of the 2 time . . Find the longest repeated substrings in θ (n) patterns P1 , · · · , Pq of total 4 length m as substrings in time. . Find the most frequently occurring substrings 5 O(m + z) time. . Search for a regular expression P 3 of a minimum length in θ (n) time. . Find the shortest strings from ∑ that do not 6 in time expected sublinear in n . . Find for each sufﬁx of a pattern 4 occur in D, in O(n + z) time, if there are z such P, the length of the longest strings. . Find the shortest substrings occurring only 7 match between a preﬁx of P[i . . . m] and a substring in D in once in θ (n) time. . Find, for each i, the shortest substrings of Si θ (m) time . 8 . ... 5 not occurring elsewhere in D in θ (n) time. . ... 9 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 3 / 25
4. 4. Trie, Radix Tree, Sufﬁx Trie & Sufﬁx Tree trie1 is a dictionary tree. (AKA preﬁx tree) stores a set of words. each node represents a character except that root is empty string. words with common preﬁx share same parent nodes. minimal deterministic ﬁnite automaton that accepts all words. radix tree is a trie with compressed chain of nodes. (AKA patricia trie or radix trie) Each internal node has at least 2 children. sufﬁx trie is a trie which stores all sufﬁx of a given string. sufﬁx tree is a sufﬁx radix tree. that enables linear time construction and fast algorithms of other problems on a string. 1 pronounced as in word retrieval by its inventor, /tri:/ “tree”, butpronounced /traI/ “try” by other authors . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 4 / 25
5. 5. Trie & Radix Tree.Trie of “A to tea ted ten iin inn”. . Radix tree example . .. . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 5 / 25
6. 6. Sufﬁx Trie & Sufﬁx Tree of “banana” ^ b a n \$ 0:6 a n \$ a banana\$ a na \$ 1:0 8:6 6:6 10:6 n a n \$ na \$ na\$ \$ a n \$ a 4:6 9:6 3:2 7:6 n a \$ na\$ \$ a \$ 2:1 5:6 \$ . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 6 / 25
7. 7. Sufﬁx Tree of “mississippi” mississippi\$ 0:11 (1,2) (0,...) (8,9) (11,...) (2,3) i mississi... p \$... s 12:8 1:0 15:10 18:11 4:4 (8,...) (2,5) (11,...) (10,...) (9,...) ppi\$... ssi \$... i\$... pi\$... (3,5) (4,5) 13:8 6:8 17:11 16:10 14:8 si i (8,...) (5,...) ppi\$... ssippi\$... 7:8 2:1 8:8 10:8 (8,...) (5,...) (8,...) (5,...) ppi\$... ssippi\$... ppi\$... ssippi\$... 9:8 3:2 11:8 5:4 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 7 / 25
8. 8. Part Outline.1 What is Sufﬁx Tree.2 History & Naïve Algorithm.3 Optimization of Naïve Algorithm.4 Examples & Analysis . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 8 / 25
9. 9. History of Sufﬁx Tree Algorithms. M. Farach. Optimal sufﬁx tree First linear algorithm was introduced by Weiner construction with large alphabets. In focs, page 1973 as position tree. Awarded by Donald Knuth 137. Published by the IEEE Computer Society, 1997. as “Algorithm of the year 1973”. E. M. McCreight. A space-economical sufﬁx Greatly simpliﬁed by McCreight 1976. tree construction algorithm. J. ACM, 23:262–272, AprilAbove two algorithms are processing string backward. 1976. ISSN 0004-5411. doi: http://doi.acm.org/10. 1145/321941.321946. URL First online construction by Ukkonen 1995, which http: //doi.acm.org/10. is easier to understand. 1145/321941.321946. E. Ukkonen. On-line construction of sufﬁx trees.Above algorithms assume size of alphabet as ﬁxed Algorithmica, 14:249–260, 1995. ISSN 0178-4617.constant . URL http://dx.doi.org/10. Limitation was break by Farach 1997, optimal for 1007/BF01206331. 10.1007/BF01206331. all alphabets. P. Weiner. Linear pattern matching algorithms. In Switching and AutomataFurther study are continued to scale to scenarios when Theory, 1973. SWAT ’08. IEEE Conference Record ofthe whole sufﬁx tree or even input string cannot ﬁt into 14th Annual Symposium on, pages 1 –11, oct. 1973. doi:memory. . . . 10.1109/SWAT.1973.13. . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 9 / 25
10. 10. Backward Construction of Sufﬁx Tree 1 2 3 4 banana\$ banana\$ anana\$ banana\$ anana\$ nana\$ banana\$ ana nana\$ na\$ \$ 7 6 5 banana\$ a na \$ banana\$ a na banana\$ ana na na \$ na\$ \$ na \$ na\$ \$ na\$ \$ na\$ \$ na\$ \$ na\$ \$ . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 10 / 25
11. 11. Construct Sufﬁx Tree by Sorting SufﬁxSufﬁx : Sorted sufﬁx : Tree of sorted sufﬁx :mississippi i | -i - >| -ississippi ippi | | - ppississippi issippi | | - ssi - >| - ppisissippi ississippi | | - ssippiissippi mississippi | - mississippissippi pi | -p - >| - isippi ppi | | - piippi sippi | -s - >| -i - - >| - ppippi sissippi | | - ssippipi ssippi | - si - >| - ppii ssissippi | - ssippiTime complexity will be O(N 2 log N) .Space complexity will be O(N 2 ) . . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 11 / 25
12. 12. Naïve AlgorithmS UFFIX T REE(string)1 for i ← 1 to length(string)2 do U PDATE(treei ) £ Phrase iU PDATE(treei )1 for j ← 1 to i2 do node ← treei .F IND(sufﬁx[j to i − 1])3 E XTEND(node, string[i]) £ Extension jTime complexity will be O(N 3 ) .Space complexity will be O(N 2 ) .The challenge is to make sure treei is updated to treei+1 efﬁciently. . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 12 / 25
13. 13. Sufﬁx Extend Cases of Naïve Algorithm. Case I If path Sj...i ends at leaf, append a char Si+1 to end of edge into leaf. Sj...i = . . . na Sj...i+1 = . . . nan na nan Case II If path Sj...i ends in the middle of an edge , and next char Si+1 is not equal to the next char in the edge, split that edge, create a internal node, add a new edge to a new leaf. Sj...i = . . . na Sj...i+1 = . . . nay n nan na y Case III If path Sj...i ends in the middle of an edge , and next char Si+1 is equal to the next char in the edge, do nothing, extenstion has done. Sj...i = . . . na Sj...i+1 = . . . nan nan nan . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 13 / 25
14. 14. Naïve Online Construction of Sufﬁx Tree 1 2 3 4 b ba a ban an n bana ana na 7 6 5 banana\$ a na \$ banana anana nana banan anan nan na \$ na\$ \$ na\$ \$ . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 14 / 25
15. 15. Properties of Sufﬁx Tree. . Each update will add exactly 1 1 leaf node . 0:6 nr_leaf = N . Sufﬁx tree is full tree. 2 banana\$ a na \$ Each internal node has at least 2 children. 1:0 8:6 6:6 10:6 nr_internal < N nr_node < 2N na \$ na\$ \$ . Worst case Fabonacci word 3 4:6 9:6 3:2 7:6 abaababaabaab . Sufﬁx is either explicit or implicit. 4 na\$ \$ Explicit when it ends at a node. 2:1 5:6 Implicit when it ends in the middle of an edge. . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 15 / 25
16. 16. Part Outline.1 What is Sufﬁx Tree.2 History & Naïve Algorithm.3 Optimization of Naïve Algorithm.4 Examples & Analysis . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 16 / 25
17. 17. Optimization of Naïve Algorithm. . Substrings can be represented as (start, end) pair of their index in 1 orignal string. Reduce space complexity to O(N) if size of alphabet is ﬁxed constant. . Once a leaf, Always a leaf 2 Represent edge that links to a leaf as (start, · · · ). Extend leaf nodes for free. We do not need Extend Case I. 0:6 0:6 banana\$ a na \$ (1,2) (0,...) (6,...) (2,4) a banana\$... \$... na 1:0 8:6 6:6 10:6 8:6 1:0 10:6 6:6 (6,...) (2,4) (6,...) (4,...) na \$ na\$ \$ \$... na \$... na\$... 4:6 9:6 3:2 7:6 9:6 4:6 7:6 3:2 (6,...) (4,...) na\$ \$ \$... na\$... 5:6 2:1 2:1 5:6 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 17 / 25
18. 18. Active Point During a phrase, if we meet Extend Case III, that is if we found S[i + 1] already exists in sufﬁx[j . . . i] then S[i + 1] will exists in ∀sufﬁx[k . . . i], k ∈ j . . . i. Thus Case III is a sign that means update of this phrase is ﬁnished. During phrase i if we stopped at sufﬁx[k . . . i] by Case III, then in next phrase we can start from sufﬁx[k . . . i + 1] because all sufﬁx start with 1 . . . k − 1 will end at Case I. We called this point(current internal node, current position k in string) as Active Point.a ab aba abaa abaab abaaba abaabab abaababa abaababaa abaababaab abaababaaba abaababaabaa abaababaaba ab0:0 0:1 0:2 0:3 0:4 0:5 0:6 0:7 0:8 0:9 0:10 0:11 0:12 (0,...) (0,...) (1,...) (0,...) (1,...) (0,1) (1,...) (0,1) (1,...) (0,1) (1,...) (0,1) (1,3) (0,1) (1,3) (0,1) (1,3) (0,1) (1,3) (0,1) (1,3) (0,1) (1,3) (0,1) (1,3) a... ab... b... aba... ba... a baa... a baab... a baaba... a ba a ba a ba a ba a ba a ba a ba1:0 1:0 2:1 1:0 2:1 3:3 2:1 3:3 2:1 3:3 2:1 3:3 7:6 3:3 7:6 3:3 7:6 3:3 7:6 3:3 7:6 3:3 7:6 3:3 7:6 (3,...) (1,...) (3,...) (1,...) (3,...) (1,...) (3,...) (1,3) (3,...) (6,...) (3,...) (1,3) (3,...) (6,...) (3,...) (1,3) (3,...) (6,...) (3,...) (1,3) (3,...) (6,...) (3,...) (1,3) (3,...) (6,...) (3,6) (1,3) (3,6) (6,...) (3,6) (1,3) (3,6) (6,...) a... baa... ab... baab... aba... baaba... abab... ba abab... b... ababa... ba ababa... ba... ababaa... ba ababaa... baa... ababaab... ba ababaab... baab... ababaaba... ba ababaaba... baaba... aba ba aba baabaa... aba ba aba baabaab... 4:3 1:0 4:3 1:0 4:3 1:0 4:3 5:6 2:1 8:6 4:3 5:6 2:1 8:6 4:3 5:6 2:1 8:6 4:3 5:6 2:1 8:6 4:3 5:6 2:1 8:6 13:11 5:6 11:11 8:6 13:11 5:6 11:11 8:6 (3,...) (6,...) (3,...) (6,...) (3,...) (6,...) (3,...) (6,...) (3,...) (6,...) (11,...) (6,...) (3,6) (6,...) (11,...) (6,...) (11,...) (6,...) (3,6) (6,...) (11,...) (6,...) abab... b... ababa... ba... ababaa... baa... ababaab... baab... ababaaba... baaba... a... baabaa... aba baabaa... a... baabaa... ab... baabaab... aba baabaab... ab... baabaab... 1:0 6:6 1:0 6:6 1:0 6:6 1:0 6:6 1:0 6:6 14:11 4:3 9:11 6:6 12:11 2:1 14:11 4:3 9:11 6:6 12:11 2:1 (11,...) (6,...) (11,...) (6,...) a... baabaa... ab... baabaab... 10:11 1:0 10:11 1:0 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 18 / 25
19. 19. Ukk’s Update using Active PointU PDATE(treei ) 1 current_sufﬁx ← active_point 2 next_char ← string[i] 3 while True 4 do if there exists edge start with next_char 5 then break £ Case III 6 else 7 split current edge if implicit 8 create new leaf with new edge labelled next_char 9 if current_sufﬁx is empty10 then break11 else current_sufﬁx ← next shorter sufﬁx12 active_point ← current_sufﬁx . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 19 / 25
20. 20. Sufﬁx Link to ﬁnd next shorter sufﬁx Sufﬁx link Internal node of sufﬁx X α has a link to node α . If α is empty, sufﬁx link points to root. How to create sufﬁx link Link together every internal nodes that are created by splitting in same phrase. banana\$ banana\$ 0:6 0:6 (1,2) (0,...) (6,...) (2,4) (0,...) (1,2) (6,...) (2,4) a banana\$... \$... na banana\$... a \$... na 8:6 1:0 10:6 6:6 1:0 8:6 10:6 6:6 (6,...) (2,4) (6,...) (4,...) (6,...) (2,4) (6,...) (4,...) \$... na \$... na\$... \$... na \$... na\$... 9:6 4:6 7:6 3:2 9:6 4:6 7:6 3:2 (6,...) (4,...) (6,...) (4,...) \$... na\$... \$... na\$... 5:6 2:1 5:6 2:1 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 20 / 25
21. 21. Fast jump using Sufﬁx Link Assume we are at Sufﬁx X αβ , whose parent internal node represent X α . 1. Go back to parent internal node, 2. Jumping follow the node’s sufﬁx link to the node represent Sufﬁx α 3. Go down to Sufﬁx αβ . Even jump down in step 3 because we already know length of β . (Skip/Count trick) Combining all these tricks we can Extend a character in O(1) time. . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 21 / 25
22. 22. Part Outline.1 What is Sufﬁx Tree.2 History & Naïve Algorithm.3 Optimization of Naïve Algorithm.4 Examples & Analysis . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 22 / 25
23. 23. Experiment – mississippi. m 0:0 (0,...) m... 1:0 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 23 / 25
24. 24. Experiment – mississippi. mi 0:1 (1,...) (0,...) i... mi... 2:1 1:0 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 23 / 25
25. 25. Experiment – mississippi. mis 0:2 (1,...) (2,...) (0,...) is... s... mis... 2:1 3:2 1:0 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 23 / 25
26. 26. Experiment – mississippi. miss 0:3 (1,...) (2,...) (0,...) iss... ss... miss... 2:1 3:2 1:0 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 23 / 25
27. 27. Experiment – mississippi. miss i 0:4 (1,...) (0,...) (2,3) issi... missi... s 2:1 1:0 4:4 (4,...) (3,...) i... si... 5:4 3:2 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 23 / 25
28. 28. Experiment – mississippi. miss is 0:5 (1,...) (0,...) (2,3) issis... missis... s 2:1 1:0 4:4 (4,...) (3,...) is... sis... 5:4 3:2 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 23 / 25
29. 29. Experiment – mississippi. miss iss 0:6 (1,...) (0,...) (2,3) ississ... mississ... s 2:1 1:0 4:4 (4,...) (3,...) iss... siss... 5:4 3:2 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 23 / 25
30. 30. Experiment – mississippi. mississi 0:7 (1,...) (0,...) (2,3) ississi... mississi... s 2:1 1:0 4:4 (4,...) (3,...) issi... sissi... 5:4 3:2 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 23 / 25
31. 31. Experiment – mississippi. mississip 0:8 (8,...) (1,2) (0,...) (2,3) p... i mississi... s 14:8 12:8 1:0 4:4 (8,...) (2,5) p... ssi (3,5) (4,5) 13:8 6:8 si i (8,...) (5,...) p... ssip... 7:8 2:1 8:8 10:8 (8,...) (5,...) (8,...) (5,...) p... ssip... p... ssip... 9:8 3:2 11:8 5:4 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 23 / 25
32. 32. Experiment – mississippi. mississip p 0:9 (8,...) (1,2) (0,...) (2,3) pp... i mississi... s 14:8 12:8 1:0 4:4 (8,...) (2,5) pp... ssi (3,5) (4,5) 13:8 6:8 si i (8,...) (5,...) pp... ssipp... 7:8 2:1 8:8 10:8 (8,...) (5,...) (8,...) (5,...) pp... ssipp... pp... ssipp... 9:8 3:2 11:8 5:4 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 23 / 25
33. 33. Experiment – mississippi. mississippi 0:10 (1,2) (0,...) (8,9) (2,3) i mississi... p s 12:8 1:0 15:10 4:4 (2,5) (8,...) (9,...) (10,...) ssi ppi... pi... i... (3,5) (4,5) 6:8 13:8 14:8 16:10 si i (8,...) (5,...) ppi... ssippi... 7:8 2:1 8:8 10:8 (8,...) (5,...) (8,...) (5,...) ppi... ssippi... ppi... ssippi... 9:8 3:2 11:8 5:4 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 23 / 25
34. 34. Experiment – mississippi. mississippi\$ 0:11 (1,2) (0,...) (8,9) (11,...) (2,3) i mississi... p \$... s 12:8 1:0 15:10 18:11 4:4 (8,...) (2,5) (11,...) (10,...) (9,...) ppi\$... ssi \$... i\$... pi\$... (3,5) (4,5) 13:8 6:8 17:11 16:10 14:8 si i (8,...) (5,...) ppi\$... ssippi\$... 7:8 2:1 8:8 10:8 (8,...) (5,...) (8,...) (5,...) ppi\$... ssippi\$... ppi\$... ssippi\$... 9:8 3:2 11:8 5:4 . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 23 / 25
35. 35. Time Complexity Analysis\$ippississim . m i s s i s s i p p i \$Time complexity is 2N = O(N) . . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 24 / 25
36. 36. Experiment – English text. 16 "data" using 1:3 14Construction Time (sec.) 12 10 8 6 4 2 0 0 20000 40000 60000 80000 100000 120000 140000 File size (byte) . . . . . . jc-yang (Osaka-U) SufﬁxTree November 12, 2011 25 / 25