Open Source Natural Language Processing - Francis Bond

Open Source Natural Language Processing Francis Bond <www3.ntu.edu.sg/home/fcbond/> Division of Linguistics and Multilingual Studies Nanyang Technological University <bond@ieee.org> 2009-08-21 (GeekCamp)

Self Introduction ¢ BA in Japanese and Mathematics ¢ BEng in Power and Control ¢ PhD in “Machine Translation” ¢ 1991-2006 NTT (Nippon Telegraph and Telephone) Japanese - English/Malay Machine Translation Japanese corpus, grammar and ontology (Hinoki) ¢ 2006-2009 NICT (National Inst. for Info. and Comm. Technology) Japanese - English, Chinese Machine Translation Japanese WordNet (Released in March 2009) 2009-08-21 (GeekCamp) 1

Overview ¢ What is NLP (and Why do it)? ¢ Machine Translation Examples ¢ Why Open Source? ¢ Wrap Up ¢ State of the Art 2009-08-21 (GeekCamp) 2

The basic problem We get words People saw her duck. We want meaning 2009-08-21 (GeekCamp) 3

People saw her duck1 http://www.animaltalk.us/for/Animals/ fw-cute-picture-of-your-daughter-with-duck/ 2009-08-21 (GeekCamp) 4

People saw her duck2 http://www.nataliedee.com/012109/ ducking-incoming-balls.jpg 2009-08-21 (GeekCamp) 5

People saw her duck3 OpenClipArtLibrary 2009-08-21 (GeekCamp) 6

Syntax (1) (2) (3) S S S NP VP NP VP NP VP N N N V NP V VP V NP N N N V:see DET N V NP V V:saw DET N N N N saw her N V:see N V saw her N People People People N saw her V N duck. duck. duck. 2009-08-21 (GeekCamp) 7

Structural Semantics Who did what to whom, how, where, when and why? (1) see(people, ducki: past) poss(ducki, pron:[3rd, sg, fem]: past) (2) see(people, duckj ) duckj (pron:[3rd, sg, fem]) (3) saw(people, ducki) poss(ducki, pron:[3rd, sg, fem]) 2009-08-21 (GeekCamp) 8

Lexical Semantics What are people? What’s a duck? What does sawing entail? (4) people ⊂ entity (5) see ⊂ perceive (6) saw ⊂ cut (7) ducki ⊂ bird (8) duckj ⊂ move 2009-08-21 (GeekCamp) 9

Pragmatics The study of meaning in context. ¢ Which people? ¢ What duck? ¢ Why did you say that? ¢ What does it imply? 2009-08-21 (GeekCamp) 10

The problem restated ¢ How can we model and resolve ambiguity? ¢ Two main approaches Deduce implicit models ∗ bag of words, n-gram chunks, . . . Deﬁne explicit models ∗ Grammars, lexicons and thesauri ¢ Then build a statistical language model (machine learning) 2009-08-21 (GeekCamp) 11

Not just algorithms ¢ The data is as important as the algorithm ¢ Two areas of development Open (?) Content ∗ The Web!, Text Corpora, WordNet, Wikipedia, dictionaries, . . . Open Software ∗ NLTK (python), Gate, DELPH-IN, . . . ¢ Copyright issues are always with us (;_;) 2009-08-21 (GeekCamp) 12

Some Examples ¢ Speech Recognition ¢ Text-to-speech ¢ Segmentation: split strings into words ¢ Part-of-Speech (nouns or verbs) ¢ Named Entity Recognition ¢ Syntactic Parsing: syntactic trees and dependencies ¢ Word Sense Disambiguation: lexical semantics ¢ Semantic Parsing: structural semantics 2009-08-21 (GeekCamp) 13

Two Examples of Open Source MT ¢ MOSES (http://www.statmt.org/moses/) Open Source Statistical MT tool kit Just add bilingual corpus! ¢ LOGON (www.delph-in.net/) Open Source Knowledge-based MT tool kit Just add transfer rules! 2009-08-21 (GeekCamp) 14

Statistical Machine Translation? Basic Idea (Brown et al 1990) ˆ E = argmax P (E|J ) E Japanese Translation Model English Language Model J P (J |E) E P (E) Decoder ˆ J argmaxE P (E)P (J |E) E 2009-08-21 (GeekCamp) 15

Translation Model (IBM Model 4) P (J, A|E) Fertility Model could you recommend another hotel n(φi|Ei) NULL Generation Model could could recommend another another hotel m−φ0 φ0 p0 2φ0 pφ0 m− 1 Lexicon Model could could recommend NULL another another hotel NU t(Jj |EAj ) Distortion Model ていただけます紹介しを他のホテルか d1(j − k|A(Ei)B(Jj )) d1>(j − j ′|B(Jj )) 他のホテルを紹介していただけますか Now with chunks (another hotel ↔ 他のホテル)! 2009-08-21 (GeekCamp) 16

Knowledge-based MT Source Source Semantic Target Target Text Analysis MRS S MRS T Generation Text (JACY) Transfer (ERG) Stochastic Model(s) ¢ From text to meaning and back again Grammars for Japanese and English Stochastic models to choose interpretations Brittle but powerful 2009-08-21 (GeekCamp) 17

Some Examples Source 私はいやいやその仕事をした。 Ref I did the work against my will. Moses I did the work against his will. JaEn I did that work unwillingly. Source バイオリンの音色はとても美しい。 Ref The sound of the violin is very sweet. Moses The violin 音色 is very beautiful . JaEn Really, the violin timbers are beautiful. Source メイドはテーブルにナイフとフォークを並べた。 Ref The maid arranged the knives and forks on the table. Moses The maid on the table arranged the knives and forks. JaEn The maid set up the fork with the knife in the table. 2009-08-21 (GeekCamp) 18

Source その銀行はここから遠いですか。 Ref Is there bank far from here? Moses The bank is a long way from here? JaEn Is that bank distant from here? Source シェークスピアに匹敵する劇作家はいない。 Ref No dramatist can compare with Shakespeare. Moses Shakespeare is quite equal to a dramatist. (no no) JaEn A playwright, that matches in Shie-kusupia, doesn’t live. Source 彼はなぜそんなことをしたのか。 Ref Why did he do that? Moses Why did he did such a thing? JaEn Why did he do that business? 2009-08-21 (GeekCamp) 19

Why Open? ¢ NLP needs serious resources They cannot be built and maintained by a single group Open source is a very practical way of achieving ﬂexible multi-group collaboration ¢ NLP needs standards and historically the successful ones have been created bottom-up. ¢ Seeing one’s work used by other groups is very rewarding. ¢ People are generally enthusiastic about contributing to widely used work. Not just the warm inner glow 20

¢ Making resources open source removes diﬃculties in distributing work or in continuing work at another institution. ¢ Researchers are evaluated by the impact that their work has: Open Source work generally has more impact. ¢ Research should be open in principle: . . . the principle of openness in research - the principle of freedom of access by all interested persons to the underlying data, to the processes, and to the ﬁnal results of research - is one of overriding importance. Openness in Research (Stanford, Research Policy Handbook 2.6) Not just the warm inner glow 21

NLP by regexp Bilingual Dictionaries from mainly monolingual text! ¢ Fully Bracketed Examples 「収穫逓減の法則(the law of diminishing return)」 ¢ Partly Bracketed Examples 図１に，明瞭性 (Clarity)・新奇性 (Novelty) ¢ Over a million pairs from the Japanese Web corpus Not yet released (copyright again) It’s fun 22

The ultimate goal ¢ NLP is fairly wide in scope ¢ We want to know everything about everything and how it ﬁts together The best source of knowledge we have is still text Replace human bandwidth with machine bandwidth Process, reﬁne, reprocess ¢ Need both technical and social approaches Linguistic Analysis Machine Learning User Generated Content Mad Scientists of the World Unite 23

Closing ¢ There are many great open source NLP tools the bleeding edge is mainly open source If you want to know more Or even better want to play with them Or best of all develop them ⇒ Say hello: (especially PhD candidates) bond@ieee.org And now, the end is near 24

Another Example of the Problem (9) Everyone gets a little of Cucumber’s ♥. ¢ Lexical gaps: Cucumber (name) ¢ Lexical gaps: ♥ (noun – we have it as verb: I ♥ NY) ¢ How to model ambiguity Cucumber is deliberately ambiguous here ∗ research show rude jokes are funnier ∗ can we model this? Topical Example 25

Solutions ¢ Morphological analysis should guess the POS Based on two to three words of previous context and a large learned lexicon and model This allows us to parse Actually there are issues with ♥ (words are [a-z -]+) ¢ Recognizing “Cucumber” as software Cucumber is a tool that can execute . . . ¢ Linking ♥ to love: ♥n → ♥v (v2n derivational rule) ¢ Scaling is the problem Feel free to use these slides or extracts from them for any purpose at all, Francis Bond 2009-08-22. 26

Open Source Natural Language Processing - Francis Bond

by jasonong on Aug 24, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 11

Statistics

Open Source Natural Language Processing - Francis Bond — Presentation Transcript