SWT Final Project Presentation

What happened? Martin Majlis

Outline  Introduction  Architecture  Back-end  Downloading  Extraction  Front-end  Web application  iGoogle Gadget 28/01/10 SWT - Final Project 2

Introduction  Answer on questions:  what happened on 3 January  what happened on 3 January 1865  what happened on January 1825  what happened from January until July 1985  what happened during the 16th century  what started on January 1930  what ended in 1990 28/01/10 SWT - Final Project 3

Architecture  Back-end  Downloading  Structure Converting  Parsing  Front-end  Web application  iGoogle Gadget 28/01/10 SWT - Final Project 4

Build process  Fully automatized  Target for each phase  Less error-prone  GNU Make 28/01/10 SWT - Final Project 5

Data Source  Czech Wikipedia  Documented format  Dumps regularly generated  Cleaner than general texts 28/01/10 SWT - Final Project 6

Downloading / Conversion  Downloading  Script from DBPedia  Added traffic shaping  Data Conversion  Recognizing pages/categories  Building category “hierarchy” 28/01/10 SWT - Final Project 7

Categories  Confusing Structure  Netherlands - 229  Physics, Planets, Illusions, Psychology, Literature, Organ, Neuroscience, etc.  Maximal deep 5  Median: 31  Mean: 33.87 28/01/10 SWT - Final Project 8

Date Extraction - Tools  Standard way:  GNU Flex / GNU Bison  Ragel  Problem with UTF-8 support  Unicode – almost 100.000 characters  Big transition tables (100.000 vs 127) 28/01/10 SWT - Final Project 10

Date Extraction - Mixed  Lexical Analysis  Regular Expressions  Filling Table  Syntactic Analysis  Theoretically CFG  Practically again regular expressions 28/01/10 SWT - Final Project 11

Date Extraction - Example  Lexical Analysis  “From 23 January 1956 until 2 February 1960”  “From {{DATE_1}} until {{DATE_2}}”  Syntactic Analysis  Interval = “From” DATE “to” DATE  Interval = “Between” DATE “and” DATE 28/01/10 SWT - Final Project 12

Date Representation  Dates from 10.000 BC to 2500 AC th  Not exact: 13 century, June 1689  Zero  2 January - 5days = 28 December  2 January 1AC -5days = 28 December 1BC  Simple tuples  (“I”, 23, 1, 1956, 20, 2, 2, 1960, 20) 28/01/10 SWT - Final Project 13

Web application  PHP5 + MySQL  Nette Framework + Dibi  http://css.majlis.cz/  GT: http://jdem.cz/dspw9  HTML, JSON, XML output 28/01/10 SWT - Final Project 14

iGoogle Gadget  iGoogle = Google personalized homepage  URL: http://jdem.cz/dspx7  Using JSON  Tricky development 28/01/10 SWT - Final Project 15

Future Work  Improve performance  20th century events – 28s – 406.980 (one OR)  20th century events – 0.0007s – 392.573 (no OR)  Improve parser architecture 28/01/10 SWT - Final Project 16

Questions? 28/01/10 SWT - Final Project 17

Thank You! 28/01/10 SWT - Final Project 18

SWT Final Project Presentation

by Martin Majlis on Feb 17, 2010

Accessibility

Tags

Upload Details

Usage Rights

1 Embed 56

Statistics

SWT Final Project Presentation — Presentation Transcript