CROSPELL ENGINE - NLP Approach

1. Product of STYX

2. Content • Spell Checking (Porter English, ISRI Arabic, MED) • Language Identification (Multi-language) • Word Prediction • Topic Prediction • Dictionary (Arabic-English) • Sentiment Analysis • The MED – Image Processing – Genome Matching

3. Content • Spell Checking (Porter English, ISRI Arabic, MED) • Language Identification (Multi-language) • Word Prediction • Topic Prediction • Dictionary (Arabic-English) • Sentiment Analysis • The MED – Image Processing – Genome Matching

4. “There are three kinds of people in the world; those who can count and those who can't.” Math Joke!

6. http://www.ecenglish.com/learnenglish/lessons/can-you-read

7. Spell Checking

8. Previous Work Open and Closed Source Systems

9. Ayaspell

10. Ayaspell ATDL

11. Ayaspell ATDL Mishkal & Alkhaleel

12. Ayaspell ATDL Mishkal & Qutrub Alkhaleel

13. How Google Spell-Check?!

14. Spell MED Checking

15. Spell Porter, ISRI, MED Checking

16. CyberSpell

17. CyberSpell “Non-specific”

18. • Dictionary-builder Algorithm CyberSpell • Cyber-fast • Highly Optimized

19. CyberSpell Lazy Init. – Fast Calls

20. CyberSpell Lazy Init. – Fast Calls “Pre-Compute”

21. CyberSpell – Building Dictionary

22. 2 Modes CyberSpell Normal VS CyberFast

23. CyberSpell 1000x FASTER

24. CyberSpell “Early Out”

25. CyberSpell – Dictionary Building Model Dictionary New Word Adder

26. CyberSpell – Dictionary Building Model Dictionary New Word Adder Dictionary

27. CyberSpell – Dictionary Building Model Dictionary New Word Adder Dictionary Word Variations

28. CyberSpell – Dictionary Building Model Dictionary New Word Adder Dictionary Edits

31. CyberSpell – Dictionary Building Dictionar y

32. CyberSpell – Dictionary Building Dictionar y

33. CyberSpell – Dictionary Building Keys <String> Dictionar y

34. CyberSpell – Dictionary Building DictionaryItem DictionaryItem Keys <String> Dictionar y DictionaryItem

35. CyberSpell – Dictionary Building DictionaryItem Keys <String> DictionaryItem [Term] Dictionar y

36. CyberSpell – Dictionary Building DictionaryItem Keys <String> DictionaryItem [Term] Dictionar EditItem y

37. CyberSpell – Dictionary Building New Word = “The” Dictionar y

38. CyberSpell – Dictionary Building New Word = “The” the Dictionar y

39. CyberSpell – Dictionary Building New Word = “The” Edits = “th”, “te”,”he” the Dictionar y

40. CyberSpell – Dictionary Building New Word = “The” Edits = “th”, “te”,”he” the th the, 1 te the, 1 he the, 1 Dictionar y

41. CyberSpell – Dictionary Building the th the, 1 te the, 1 tie, 1 ten, 1 tea, 1 he the, 1 she, 1 her, 1 tie ten tea she Dictionar her y …

42. CyberSpell – Search

43. CyberSpell – Dictionary Search Model Input Word List<Suggestion >

44. CyberSpell – Dictionary Search Model Dictionary Input Word Search Handler List<Suggestion >

45. CyberSpell – Dictionary Search Model Dictionary Input Word Search Dictionary Handler List<Suggestion >

46. CyberSpell – Dictionary Search Model Dictionary Input Word Search Dictionary Handler List<Suggestion >

47. CyberSpell – Dictionary Search Model Dictionary Input Word Search Dictionary Handler Word List<Suggestion Variations >

51. CyberSpell – Dictionary Search Model Dictionary Input Word Search Dictionary Handler Suggestion Word List<Suggestion s Handler Variations >

52. CyberSpell – Dictionary Search Model Dictionary Input Word Search Dictionary Handler Suggestion Word List<Suggestion s Handler Variations >

53. CyberSpell – Dictionary Search Model Dictionary “tade” Search Dictionary Handler Suggestion Word List<Suggestion s Handler Variations >

54. CyberSpell – Dictionary Search Model Dictionary “tade” Search Dictionary Handler Suggestion “ade”, List<Suggestion s Handler “tde”, ..etc >

55. CyberSpell – Dictionary Search Model Dictionary “tade” Search Dictionary Handler CyperFa st Mode Made Take Suggestion “ade”, Trade s Handler “tde”, ..etc Tide Take Bade Fade 7

56. CyberSpell – Dictionary Search Model Dictionary “tade” Search Dictionary Handler CyperFa Normal st Mode Mode Made Made Take Take Suggestion “ade”, Trade Trade s Handler “tde”, ..etc Tide … Take The Bade Have Fade ... 7 128

57. CyberSpell – Heuristics

58. CyberSpell – Keyboard Map

59. CyberSpell – Keyboard Map <?xml version="1.0" encoding="utf-8"?> <KeyboardMap> <KeyboardRow RowNum="0"> <KeyboardRowKeys Layer="0" ShiftEnabled="false"> <![CDATA[` 1 2 3 4 5 6 7 8 9 0 - =]]> </KeyboardRowKeys> <KeyboardRowKeys Layer="1" ShiftEnabled="true"> <![CDATA[~ ! @ # $ % ^ & * ( ) _ +]]> </KeyboardRowKeys> </KeyboardRow> <KeyboardRow RowNum="1"> <KeyboardRowKeys Layer="0" ShiftEnabled="false"> <![CDATA[q w e r t y u i o p [ ]]]> </KeyboardRowKeys> <KeyboardRowKeys Layer="1" ShiftEnabled="true"> <![CDATA[Q W E R T Y U I O P { }]]> </KeyboardRowKeys> </KeyboardRow>

63. CyberSpell – Keyboard Map ` 1 2 3 4 5 6 7 8 9 0 - = <

64. CyberSpell – Keyboard Map ` 1 2 3 4 5 6 7 8 9 0 - = < Tab Q W E R T Y U I O P [ ] Enter

65. CyberSpell – Keyboard Map ` 1 2 3 4 5 6 7 8 9 0 - = < Tab Q W E R T Y U I O P [ ] Enter Caps A S D F G H J K L ; „

66. CyberSpell – Keyboard Map ` 1 2 3 4 5 6 7 8 9 0 - = < Tab Q W E R T Y U I O P [ ] Enter Caps A S D F G H J K L ; „ Shift Z X C V B N M , , . / Shift

67. CyberSpell – Keyboard Map ` 1 2 3 4 5 6 7 8 9 0 - = < Tab Q E R T Y U I O P [ ] Enter Caps A S D F G H J K L ; „ Shift Z X C V B N M , , . / Shift

69. CyberSpell – Keyboard Map ` 1 2 3 4 5 6 7 8 9 0 - = Q E R T Y U I O P [ ] A S D F G H J K L ; „ Z X C V B N M , , . /

80. CyberSpell – Nonlinear Dist. ` 1 2 3 4 5 6 7 8 9 0 - = < Tab Q W E R T Y U I O P [ ] Enter Caps A S D F G H J K L ; „ Shift Z X C V B N M , , . / Shift

86. CyberSpell – Any Keyboard Map ` 1 2 3 4 5 6 7 8 9 0 - = < Tab Enter Caps Shift / Shift

87. CyberSpell – User Preference

88. CyberSpell – User Preference <?xml version="1.0" encoding="utf-8"?> <UserPreferenceDb> <UserPreference id="tade"> <WordPrefered>made</WordPrefered> <Counter>3</Counter> </UserPreference> <UserPreference id="tade"> <WordPrefered>take</WordPrefered> <Counter>4</Counter> </UserPreference> <UserPreference id="jst"> <WordPrefered>just</WordPrefered> <Counter>1</Counter> </UserPreference> </UserPreferenceDb>

92. CyberSpell – Example For a misspelled input word: “tade” CyperFast [154]Made [87]Take [52]Trade [13]Tide [5]Tale [2]Bade [1]Fade 7

93. CyberSpell – Example For a misspelled input word: “tade” CyperFast NormalMod e [154]Made [154]Made [87]Take [87]Take [52]Trade [52]Trade [13]Tide … [5]Tale The [2]Bade Have [1]Fade ... 7 128

94. CyberSpell – Example For a misspelled input word: “tade” CyperFast NormalMod UserPref e [154]Made [154]Made [5]Take [87]Take [87]Take [4]Made [52]Trade [52]Trade [1]Trade [13]Tide … [13]Tide [5]Tale The [5]Tale [2]Bade Have [2]Bade [1]Fade ... [1]Fade 7 128 3

95. CyberSpell – Example For a misspelled input word: “tade” CyperFast NormalMod UserPref e [154]Made [154]Made [5]Take [87]Take [87]Take [4]Made [52]Trade [52]Trade [1]Trade [13]Tide … [13]Tide [5]Tale The [5]Tale [2]Bade Have [2]Bade [1]Fade ... [1]Fade 7 128 3

96. CyberSpell – Example For a misspelled input word: “tade” CyperFast NormalMod UserPref KeyMap e [154]Made [154]Made [5]Take [52]Trade [87]Take [87]Take [4]Made [52]Trade [52]Trade [1]Trade [13]Tide … [13]Tide [5]Tale The [5]Tale [2]Bade Have [2]Bade [1]Fade ... [1]Fade 7 128 3 1

97. CyberSpell – Example For a misspelled input word: “tade” CyperFast NormalMod UserPref KeyMap Scallar e [154]Made [154]Made [5]Take [52]Trade [487]Take [87]Take [87]Take [4]Made [454]Make [52]Trade [52]Trade [1]Trade [152]Trade [13]Tide … [13]Tide [13]Tide [5]Tale The [5]Tale [5]Tale [2]Bade Have [2]Bade [2]Bade [1]Fade ... [1]Fade [1]Fade 7 128 3 1 3

98. CyberSpell – Example For a misspelled input word: “tade” CyperFast NormalMod UserPref KeyMap Scallar e [154]Made [154]Made [5]Take [52]Trade [487]Take [87]Take [87]Take [4]Made [454]Make [52]Trade [52]Trade [1]Trade [152]Trade [13]Tide … [13]Tide [13]Tide [5]Tale The [5]Tale [5]Tale [2]Bade Have [2]Bade [2]Bade [1]Fade ... [1]Fade [1]Fade 7 128 3 1 3

99. CyberSpell – Example For a misspelled input word “tade” CyperFast ` 1 2 3 4 5 6 7 8 9 0 - = < [154]Made [87]Take Tab Q W E R T Y U I O P [ ] Enter [52]Trade [13]Tide Caps A S D F G H J K L ; „ [5]Tale Shift Z X C V B N M , , . / Shift [2]Bade [1]Fade 7

102. CyberSpell – Example For a misspelled input word “tade”, the Key map hueristic will give the following suggestions: ` 1 2 3 4 5 6 7 8 9 0 - = < KeyMap Tab Q W E R T Y U I O P [ ] Enter [52]Trade Caps A S D F G H J K L ; „ 1 Shift Z X C V B N M , , . / Shift

103. Language Identification

104. • Text Size (Long, Short) Language • Approach • Uni-gram Identification • Bi-gram • N-gram

105. Language • Single Language Identification • Multiple Languages

106. Language Identification Language X Corpus

107. Language Identification Train Language X LM Corpus

108. Language Identification Model Language 1 Train Language X Language 2 LM Corpus . . Language N

109. Language Identification Model Language 1 Train Language X Language 2 LM Corpus . . Language N Text to Identify

110. Language Identification Model Language 1 Train Language X Language 2 LM Corpus . . Language N Text to Identification Identify Handler

111. Language Identification Model Language 1 Train Language X Language 2 LM Corpus . . Language N Text to Identification Identified Identify Handler Language

112. Word Prediction

113. • Approach Word – Uni-gram – Bi-gram Prediction – Tri-gram

114. Word Prediction Language X Corpus

115. Word Prediction Train Language X LM Corpus

116. Word Prediction Model Train Language X LM Corpus Dictionary

117. Word Prediction Model Train Language X LM Corpus Dictionary Input Code

118. Word Prediction Model Train Language X LM Corpus Dictionary Input Word Predictor Code

119. Word Prediction Model Train Language X LM Corpus Dictionary Input List Word Predictor <PredictedWord Code >

120. Topic Prediction

121. • Classified Corpus Topic • Approach – Uni-gram Prediction • Long Text

122. Topic Prediction • Classified Corpus – Culture – Economy – International News – Local News – Religion – Sports

123. Topic Prediction Model Train & Language X Classif Corpus y Dictionary

124. Topic Prediction Model Train & Language X Classif Corpus y Dictionary Input Topic Predictor Text

125. Topic Prediction Model Train & Language X Classif Corpus y Dictionary Input List Topic Predictor <PredictedTopic> Text

126. The Dictionary

127. The • Arabic-English Dictionary • Big Huge Thesaurus Dictionary

128. The Dictionary Model Input Word

129. The Dictionary Model Input Arab.-Eng. Word Dictionary

130. The Dictionary Model Finding Input Arab.-Eng. Word Word Dictionary Meanings

131. The Dictionary Model Finding Input Arab.-Eng. Word Word Dictionary Meanings Big Thesaurus

132. The Dictionary Model Finding Input Arab.-Eng. Word Word Dictionary Meanings Finding Big Word Synonym Thesaurus s

133. The Dictionary Model Finding Input Arab.-Eng. Word Word Meaning Word Dictionary Meanings Finding Word Big Word Synonym Synonyms Thesaurus s

134. The Dictionary

136. • Sentence orientation • Data set – For each word • Positivity value • Negativity value

138. Sentiment Analysis

139. Sentiment Analysis Input (sub) Sentence

140. Sentiment Analysis Model Input (sub) Sentence

141. Sentiment Analysis Model Input (sub) Total Sentence Sentiment

142. Sentiment Analysis Model Cracking Input (sub) sentence Total to sub Sentiment Sentence words

143. Sentiment Analysis Model Cracking Word 1 Input (sub) sentence Total to sub Sentiment Sentence words Word 2 …. Word n

144. Sentiment Analysis Model Cracking eval Word 1 w1 Input (sub) sentence Total to sub Sentiment Sentence words Word 2 …. Word n

145. Sentiment Analysis Model Cracking eval Word 1 w1 Input (sub) sentence Total to sub Sentiment Sentence words Word 2 w2 …. Word n

146. Sentiment Analysis Model Cracking eval Word 1 w1 Input (sub) sentence Total to sub Sentiment Sentence words Word 2 w2 …. …. Word n wn

147. Sentiment Analysis Model Cracking eval Word 1 w1 Input (sub) sentence Total to sub Sentiment Sentence words Word 2 w2 …. …. Word n wn

148. The

149. The MED

150. The MED Images / Genomes

151. The MED Image Processing

152. Image Processing

157. Image Processing - MED

158. Image Processing - MED D = 30 D = 100 D = 70 D = 20 D = 98 D=0

161. Image Processing - MED D = 30, 40, 24 D = 100, 60, 80 D = 70, 39, 100 D = 20, 16, 37 D = 98, 80, 120 D = 0, 0, 0

162. Image Processing – Regular MED Source Image Target Image

163. Image Processing – Regular MED Model – Phase 1 Source Image Target Image

164. Image Processing – Regular MED Model – Phase 1 R Comp Source Image G Comp B Comp Target Image

165. Image Processing – Regular MED Model – Phase 1 Stream R Comp Source R Thresholdin Stream Image G Comp g G Stream B Comp B Target Image

166. Image Processing – Regular MED Model – Phase 1 Stream R Comp Source R Thresholdin Stream Image G Comp g G Stream B Comp B Stream R Comp Target R Thresholdin Stream Image G Comp g G Stream B Comp B

170. Image Processing – Regular MED Model – Phase 2 Final Distnace

171. Image Processing – Regular MED Model – Phase 2 MED Final Distance Distnace

172. Image Processing – Regular MED Model – Phase 2 MED Final Distance Distnace For Each Stream Pair CalcDist(Stream1, Stream2) Stream: CompR, CompG, CompB

173. Image Processing – Regular MED Model – Phase 2 MED Distances Final Distance Handler Distnace For Each Stream Pair CalcDist(Stream1, Stream2) Stream: CompR, CompG, CompB

174. Image Processing - MED D = 30 D = 100 D = 70 Very Slow D = 20 D = 98 D=0

175. Image Processing - MED D = 30 Memory Dump D = 100 D = 70 D = 20 D = 98 D=0

176. Image Processing - MED D = 30 D = 100 Run out of memory in D = 70 big images D = 20 D = 98 D=0

177. Optimization

178. “Early Out”

179. Image Processing Optimization Distances File – Zoom-Out Shot

180. Image Processing Optimization Distances File – Zoom-Out Shot (Image Very Difference to Target)

183. Image Processing Optimization Distances File – Zoom-Out Shot (Image Close to Target)

184. Image Processing Ordering

188. Image Processing Ordering • Latest set seems to struggle

189. Image Processing Ordering • Latest set seems to struggle

190. Optimization Again!

193. Image Processing Optimization

196. Image Processing Optimization • Latest set seems to struggle

197. Image Processing Late Binding Optimization

200. Image Processing - Proc. Time 18.6 sec Regular

201. Image Processing - Proc. Time 18.6 sec 21.4 Regular sec Stalker With Polym.

202. Image Processing - Proc. Time 18.6 sec 21.4 13.6 Regular sec Stalker sec Stalker With Polym. Without Polym.

203. Image Processing - Proc. Time 18.6 sec

204. Image Processing - Proc. Time 18.6 sec 13.6 Without Polym. sec

205. Image Processing - Proc. Time 18.6 sec 13.6 Without Polym. sec 5.9 sec

206. Image Processing - Proc. Time Model Performance VS Time 25 20 15 10 5 0 1 2 3 4 5 1. MED Regular 4. MED Stalker with BT Enhanced 2. MED Stalker with BT (With Polym.) 5. MED Stalker without BT Enhanced 3. MED Stalker without BT (No Polym.)

211. Image Processing - Proc. Time Algorithm Processing Time for a set of 250 pictures, each 26 x 26 MEDRegular 18.6 MEDStalkerWithBT 21.4 MEDStalkerWithoutBT 13.6 MEDStalkerWithBTEnhanced 6.1 MEDStalkerWithoutBTEnhanced 5.9

212. Image Processing - Proc. Time Algorithm Processing Time for a set of 250 pictures, each 26 x 26 MEDRegular 18.6 MEDStalkerWithBT 21.4 MEDStalkerWithoutBT 13.6 MEDStalkerWithBTEnhanced 6.1 MEDStalkerWithoutBTEnhanced 5.9

213. The MED Genomes

214. G e n o m e s

215. G e n o m e s GCGAGTTCATCTATCACGACCGCGGTCG CTATCACCTGACCTCCAGGCCGATGCCCCTTCCGGC

222. G e n o m e s Detection of Sub-Matches

223. G e n o m e s Detection of Sub-Matches Overlapping

224. G e n o m e s Needleman-Wunsch

225. Needleman-Wunsch Distances Array

231. G e n o m e s Smith-Waterman

232. Smith-Waterman Distances Array

236. G e n o m e s and Images

237. G e n o m e s For Images Distances Array

238. G e n o m e s For Images Image1 Image2 Distances Array

239. Genomes and Images Difference Patch

240. Genomes and Images Matching Patch

241. Future Perspectives

CROSPELL ENGINE - NLP Approach

Recommended

Recommended

More Related Content

More from Mohammad Shaker

More from Mohammad Shaker (20)

CROSPELL ENGINE - NLP Approach