Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

IMPACT Final Conference - Jesse de Does

3,695 views

Published on

Evaluation of lexicon supported OCR and Information retrieval with Jesse de Does from the INL

Published in: Education, Technology
  • My personal experience with research paper writing services was highly positive. I sent a request to ⇒ www.WritePaper.info ⇐ and found a writer within a few minutes. Because I had to move house and I literally didn’t have any time to sit on a computer for many hours every evening. Thankfully, the writer I chose followed my instructions to the letter. I know we can all write essays ourselves. For those in the same situation I was in, I recommend ⇒ www.WritePaper.info ⇐.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Dating for everyone is here: ♥♥♥ http://bit.ly/2Qu6Caa ♥♥♥
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Follow the link, new dating source: ❶❶❶ http://bit.ly/2Qu6Caa ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THAT BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full doc Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download doc Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book that can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer that is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story That Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money That the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths that Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

IMPACT Final Conference - Jesse de Does

  1. 1. IMPACT Lexica in OCR and IR Evaluation for Bulgarian, Czech, Dutch, English, French, German, Polish, Slovenian, Spanish Jesse de Does
  2. 2. Contents <ul><li>OCR evaluation </li></ul><ul><ul><li>Use of lexica in OCR </li></ul></ul><ul><ul><li>Evaluation Method </li></ul></ul><ul><ul><li>(non-final) Results </li></ul></ul><ul><li>IR evaluation </li></ul><ul><ul><li>Use of lexica in IR </li></ul></ul><ul><ul><li>Evaluation Method </li></ul></ul><ul><ul><li>(Very preliminary) results </li></ul></ul>
  3. 3. Use of lexica in OCR <ul><li>! This is not about postcorrection, but about what happens during OCR </li></ul><ul><li>Using “Finereader Engine External Dictionary Interface” Functionality: </li></ul><ul><li>Any procedure that prunes a set of candidates and assigns weights can be implemented in this way </li></ul><ul><li>Such a procedure need not be limited to the use static of word lists </li></ul><ul><li>Permits dynamic implementations (spelling variation rules, morphology, …) </li></ul>date footertext
  4. 4. Finereader SDK external dictionaries <ul><li>SDK users have to implement a COM interface to prune a set of “Fuzzy Words” </li></ul>eerde cc cc eerste cc f c o o External dictionary prunes this to the linguistically possible ones (In this case: { eerste, eerde}) Fuzzy Word: set of character recognition candidates for each position in a word
  5. 5. Finereader SDK external dictionaries eerde cc cc eerste cc f c o o <ul><li>Of cause a lot of things may go wrong in this simple scenario </li></ul><ul><li>Lexicon may be too small (you will never have all spelling variations, compounds, …) </li></ul><ul><li>Lexicon may include typical OCR errors (eu, cn, ….) </li></ul><ul><li>! The Fuzzy word may be too restricted (or of course too comprehensive) </li></ul>{eerste, eerde} x ____
  6. 6. OCR Evaluation <ul><li>Measure evaluation of Finereader SDK 10 with default included dictionary  Finereader SDK 10 with both default dictionary AND use of historical lexicon </li></ul><ul><li>Main performance indicator: word recall: after alignment, how many of the words in the ground truth have a (case-insensitive) match in the OCR. Errors on punctuation not penalized. </li></ul><ul><li>Specific evaluation tool (only word accuracy) </li></ul><ul><ul><li>Workaround for region segmentation problems </li></ul></ul><ul><ul><li>Display specific information about dictionary coverage, information about performance on dictionary words, false friends …. </li></ul></ul>
  7. 8. Dictionary “cleaning” <ul><li>Dictionary hallucinations: </li></ul><ul><li>Many in-dictionary errors (“false friends”) </li></ul><ul><li>Many errors on short words </li></ul><ul><li>Dictionary cleaning procedures: </li></ul><ul><li>Remove false friends (words related by frequent OCR substitution to much more frequent words) </li></ul><ul><li>Remove infrequent short words (even if correct) </li></ul>
  8. 9. Evaluation procedure (data) <ul><li>IMPACT demonstrator sets (size between ~1200-8000 pages) </li></ul><ul><li>Split: </li></ul><ul><ul><li>Development </li></ul></ul><ul><ul><li>Evaluation </li></ul></ul><ul><ul><li>Demonstration </li></ul></ul><ul><li>OCR evaluation sets: random choice of about 200 pages from evaluation portion </li></ul><ul><ul><li>Manageable size (one experiment takes between 30 min and 1:30) </li></ul></ul>
  9. 10. Bulgarian <ul><li>Bylgarska iliustracia, 1880 - </li></ul><ul><li>Jenski glas, 1889 - </li></ul><ul><li>Sborniche za spomen na 25-godishninata ot smyrtta na Levski, 1898 - </li></ul><ul><li>Spisanie Dennica, 1890 - </li></ul><ul><li>Ugozapadna Bulgaria, 1893 - </li></ul><ul><li>Zelokupna Bulgaria, 1880 - </li></ul>
  10. 12. Freq. OCR->GT 114 д->л 218 ш->н і 242 и->п 247 г->т 256 ъ->ь 270 ь->ъ 356 и->н 378 п->н 441 е->с 579 н->п 732 н->и 825 ж -> ѫ 968 п->и Freq. OCR->GT 165 е-> ѣ 185 г->т 200 е->с 205 и->п 217 ъ->ь 220 ш->н і 249 и->н 283 ж-> ѫ 330 ь->ъ 354 п->н 463 н->п 599 н->и 733 п->и
  11. 14. 1. Czech <ul><li>Co jest konstituce?, čili, Krátký, prostonárodní wýklad hlawnějších zásad konstitucí ewropejských, 1848 </li></ul><ul><li>Ferina Lišák z Kuliferdy a na Klukově, čili, Kratičká historye zlopověstných kousků starého Reinecke, 1848 </li></ul><ul><li>Homerowa Iliada, 1802 </li></ul><ul><li>Na den narození neimocněišího, a neijasněišího cysare rímského, téz dědičného rakauského a krále ceského, Frantiska II., w Praze 12. den mesyce Unora, léta 1805, 1805 </li></ul><ul><li>Plody sborů učenců řeči českoslowanské prešporského, 1836 </li></ul><ul><li>Rozprawy o gmenách, počátkách i starožitnostech národu Slawského a geho kmeni /, 1830 </li></ul><ul><li>Sokol, 1872 </li></ul><ul><li>Základowé pitwy (Anatomie), čili, Soustawnj rozbor a popis těla lidského a gednotliwých geho částek, 1840 </li></ul>
  12. 16. 2.Dutch <ul><li>18th and 19th century books, newspapers, parliamentary papers </li></ul><ul><li>…… .. </li></ul><ul><li>Provinciale Overijsselsche en Zwolsche courant : staats-, handels-, nieuws- en advertentieblad, 1852-1852 </li></ul><ul><li>Rechtsgeleerd advis in de zaak van den gewezen stadhouder, en over deszelfs schryven aan de gouverneurs van de Oost- en West-Indische bezittingen van den staat [...]. Ingelevert [...] op den 7 january 1796. / By B. Voorda et al, 1796-1796 </li></ul><ul><li>Verhaal van het levensgevaar, waar in zig drie Rotterdamsche burgers [...] bevonden hebben, te Utrecht, 1784-1784 </li></ul><ul><li>Vrijmoedige aanmerkingen, over de uitsluiting van allen die door publieke armkassen bedeeld worden, als stemgerechtigden [...] bij eene oproeping van het Nederlandsche volk tot eene Nationaale Conventie, 1795-1795 </li></ul>
  13. 19. English <ul><li>Standard Finereader language: OldEnglish </li></ul><ul><li>15th-19th century material </li></ul><ul><li>2 sets: </li></ul><ul><ul><li>One general set, 15th-19th century </li></ul></ul><ul><ul><li>One 17th century-specific set </li></ul></ul>
  14. 20. General set with various choices of dictionary – no improvement!
  15. 21. More distinct improvement on 17th century set with special dictionary compiled from OED quotations dated 1580-1720
  16. 22. French <ul><li>Standard Finereader language: OldFrench </li></ul><ul><li>17th century books </li></ul><ul><li>Conduite du jugement naturel où tous les bons esprits de l'un et l'autre sexe pourront facilement puiser la pureté de la science, par M. Jacques Forton, sieur de S. Ange,..., 1653 </li></ul><ul><li>Dissertation de la philosophie en général, 1668 </li></ul><ul><li>La Dialectique du sieur de Launay, contenant l'art de raisonner juste sur toute sorte de matières..., 1673 </li></ul><ul><li>Lettre de M. Gadroys à M. de La Grange Trianon,... pour servir de réponse à celle que M. de Castelet a écrite contre les raisons de M. Descartes touchant le flux et le reflux de la mer. - Seconde lettre de M. Gadroys... [au même, sur le même sujet.], 1677 </li></ul><ul><li>Traitez de métaphysique démontrée selon la méthode des géomètres. [Par le sieur de La Coudraye.], 1693 </li></ul>
  17. 24. German <ul><li>Standard Finereader language: OldGerman </li></ul><ul><li>Das Buch des heyligen Römischen Reichs unnderhalltunge, 1501 </li></ul><ul><li>Die Poesie ihr Wesen und ihre Formen mit Grundzügen der vergleichenden Literaturgeschichte, 1884 </li></ul><ul><li>Echo Deß Hochzeitlichen Te Deum Laudamus, 1722 </li></ul><ul><li>Ergebnisse der Erhebungen über die Beschäftigung gewerblicher Arbeiter an Sonn- und Festtagen, Bd.:1, Gruppe I bis VII der Gewerbestatistik, Berlin, 1887, 1887 </li></ul><ul><li>Quedlinburgisches Kreis-Tags-Memorial, 1673 </li></ul><ul><li>Von der Regierung der Kirche und den unterschiedlichen Würden der Geistlichkeit *(full title in comments), 1779 </li></ul><ul><li>Warhaffter und grundlicher Bericht uß was Ursachen Martinus du Voysin (zu Basel verburgerter Krämer) inn der Statt Surseew im Aargöw, ..., den 13. Tag Octobris deß 1608. Jars erstlich enthauptet, und volgends verbrennt worden, 1609 </li></ul>
  18. 26. Polish <ul><li>Adwersaria, albo terminata sprawy wojennej, która się toczyła w wołoskiej ziemi z tureckim cesarzem, 1621 </li></ul><ul><li>Chorągiew Sarmacka w Wołoszech, to jest pospolite ruszenie i szczęśliwy powrót Polaków z Wołoch w roku 1621, 1621 </li></ul><ul><li>Diariusz wiadomości od wyjazdu króla z Wilna do Smoleńska, 1610 </li></ul><ul><li>Discurs o cenie pieniedzy teraznieyszey y o niektorych skutkach iey…, 1632 </li></ul><ul><li>Nowe Ateny, albo Akademia wszelkiey scyencyi pełna, na różne tytuły iak na classes podzielona, mądrym dla memoryału, idiotom dla nauki, politykom dla praktyki, melancholikom dla rozrywki erygowana ... . Część 3 albo Supplement., 1746 </li></ul><ul><li>Pasja żołnierzy obojga narodów w stolicy moskiewskiej krótko opisana, 1613 </li></ul><ul><li>Powodzenia niebezpiecznego ale szczęśliwego wojska j. k. m. w Multanach opisanie, 1601 </li></ul><ul><li>Relacja chwalebnej ekspedycji Jana Kazimierza, króla polskiego i szwedzkiego, 1650 </li></ul><ul><li>Wyprawa i wyjazd sułtana Amurata, cesarza tureckiego, na wojnę do Korony Polskiej, 1634 </li></ul><ul><li>Wyprawa i wyjazd sułtana Amurata, cesarza tureckiego, na wojnę do Korony Polskiej_BW, 1634 </li></ul><ul><li>Żałosne opisanie upadku króla hiszpańskiego na morzu i na lądzie, 1589 </li></ul>
  19. 28. Slovene <ul><li>Genovefa, 1841 </li></ul><ul><li>Gosp. Krištofa Šmida korarja avgustanskiga, zgodBe S. Pisma za mlade ljud..., 1850 </li></ul><ul><li>Kmetijske in rokodelske novice, 1844 </li></ul><ul><li>Kratkozhasne uganke, 1788 </li></ul><ul><li>Kuharske Bukve, 1799 </li></ul><ul><li>Marianske Kempensar, ali Dvoje bukuvze, 1769 </li></ul><ul><li>Novice kmetijskih, rokodelnih in narodskih reči, 1851 </li></ul><ul><li>Sgodbe svetiga pisma za mlade ljudi, 1830 </li></ul><ul><li>Ta male katechismus, 1768 </li></ul><ul><li>Vezhna pratika od gospodarstva, 1789 </li></ul><ul><li>Zerkviza na skali, 1855 </li></ul>
  20. 30. Spanish <ul><li>Carta athenagorica, 1690 </li></ul><ul><li>Commentarios reales, 1609 </li></ul><ul><li>El Parnasso español, 1648 </li></ul><ul><li>Obras de Garcilasso de la Vega con las anotaciones por el Mtro. Francisco Sánchez Brocense, 1612 </li></ul><ul><li>Obras de Lope de Vega, 1604 </li></ul><ul><li>Vida de Lazarillo de Tormes, 1652 </li></ul>
  21. 31. Results
  22. 32. Summary
  23. 33. Evaluation of “IR” <ul><li>Main question: </li></ul><ul><li>Are we able to retrieve historical variants of words? </li></ul><ul><li>Practical evaluation criterion: </li></ul><ul><li>Measure accuracy of modern lemma assignment </li></ul><ul><li>( If we can do this, good retrieval is possible) </li></ul><ul><li>More complete evaluation to follow soon – all partners are finishing the work </li></ul>
  24. 34. Evaluation method <ul><li>Each language partner annotates ~10.000 tokens of Ground Truth with modern lemma and/or equivalent word form </li></ul><ul><li>We measure performance of: </li></ul><ul><ul><li>Lemmatization with a modern lexicon </li></ul></ul><ul><ul><li>Lemmatization with a modern lexicon and spelling variation patterns </li></ul></ul><ul><ul><li>Lemmatization with a historical lexicon, a modern lexicon and spelling variation patterns </li></ul></ul><ul><ul><li>No context information is used </li></ul></ul>
  25. 35. English <ul><li>Using OED IR lexicon and very restricted set of spelling variation patterns </li></ul><ul><li>Considered tokens: 9409. </li></ul><ul><li>8994 had a correct lemma (recall 0,956 ) </li></ul><ul><li>Total correct suggestions : 8994 </li></ul><ul><li>Average rank of correct lemma : 1,086280 </li></ul><ul><li>total possible lemmata : 23859 </li></ul><ul><li>None match at all : 265 </li></ul><ul><li>Matched With Patterns : 1330 </li></ul><ul><li>Exact Match : 7814 </li></ul>
  26. 36. Spanish <ul><li>Using Apertium modern Spanish Lexicon, IMPACT historical spanish IR lexicon and </li></ul><ul><li>9298 token considered </li></ul><ul><li>With only modern lexicon and patterns </li></ul><ul><li>7473 with at least one correct lemma and 1825 without (recall 0,80 ) Average rank of correct lemma: 1,1, Total suggestions 9699 No match at all: 991 </li></ul><ul><li>Modern Exact: 7471; Modern With Patterns: 836 </li></ul><ul><li>With historical lexicon, modern lexicon and patterns: </li></ul><ul><li>8864 with at least one correct lemma and 434 without (recall 0,926 ) Average rank of correct lemma: 1,16, </li></ul><ul><li>Total suggestions 12417 </li></ul><ul><li>ModernWithPatterns: 186 </li></ul><ul><li>No match at all: 542 </li></ul><ul><li>Historical Lexicon Exact match: 8265 </li></ul><ul><li>ModernExact: 305 </li></ul>

×