Your SlideShare is downloading. ×
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
I-Semantics 2012 Keynote
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

I-Semantics 2012 Keynote

5,329

Published on

This talk was given as a keynote at the I-Semantics 2012 conference, Graz, Austria

This talk was given as a keynote at the I-Semantics 2012 conference, Graz, Austria

Published in: Technology, News & Politics
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,329
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
74
Comments
0
Likes
11
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Dial E for EventsLora AroyoMonday, September 10, 12
  • 2. Observation events are important events are omni-present in events carry different points of view th in ew ou orl r p d, e er .g. so na new ne l li s, tw ve sc or s, ien kin e.g ce g .s oc etc ial . Flickr: elkabong iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 3. Position ts are p art of s ue ne sof even greeme nt & vag human disa vent se mantics The the e Flickr: elkabong iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 4. Objects vs. Events events perdure = their parts exist at different time points objects endure = they have all their parts at all points in time objects are wholly present at any point in time, events unfold over time Flickr: vanilllaph iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 5. Events are important create context for objects, e.g. people, locations, organizations, etc. Lora Aroyo @laroyo iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 6. Events are important create meaning for objects, e.g. artifacts, pictures, videos. iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 7. Events are important link concepts, objects, and stories. iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 8. Events in the World events anchor the information we consume daily Flickr: craftydogma iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 9. Events @ Google News iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 10. Events @ Google News iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 11. Julian Assange’s Extradition Row iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 12. Julian Assange’s Extradition Row iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 13. Julian Assange’s Extradition Row iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 14. Julian Assange’s Extradition Row iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 15. Lance Armstrong’s Doping Fight iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 16. Lance Armstrong’s Doping Fight iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 17. The Arab Spring iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 18. The Arab Spring iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 19. Events @ Social Web iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 20. Events @ Social Web iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 21. Events @ Social Web iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 22. Events @ Social Web iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 23. Events @ Social Web iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 24. Events are Vague Humans have no clear notion of what events are iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 25. “event is a significant "happening" or gathering of people. I would define a "happening" as an event if the group of people gathered were united in one common goal.” We Asked the Crowd What an EVENT is Flickr: massimo vitali iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 26. Event is a happening, which can be scheduled or unscheduled. An earthquake or fire happens (unscheduled). A wedding or birthday party (scheduled). It is an occasion that is unusual and tends to be memorable. We Asked the Crowd What an EVENT is iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 27. “An event would be any occurrence where physical action has taken place. It may be a single, momentary instance (I sneezed), or it may span a period of time (the festival ran for four hours). An event may also be made up of a number of smaller events, such as a day at school is an event, but each individual class is also an event itself. Basically an event must have a physical action over any delimited time span.” We Asked the Crowd What an EVENT is iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 28. “Event can refer to many things such as: An observable occurrence, phenomenon or an extraordinary occurrence.” “an event is an incident thats very important or monumental” “A planned public or social get together or occasion.” “An event is something occurring at a specific time and/or date to celebrate or recognize a particular occurrence.” “a location where something like a function is held. you could tell if something is an event if there people gathering for a purpose.” We Asked the Crowd What an EVENT is iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 29. What do Experts think an EVENT is? “an event is the exemplification of a property by a substance at a given time” Jaegwon Kim, 1966 “events are changes that physical objects undergo” Lawrence Lombard, 1981 “events are properties of spatiotemporal regions”, David Lewis, 1986under30ceo.com iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 30. Event-centric Projects how events can be detected & extracted from natural language text how those extracted events are represented for use on the semantic web how to identify the same events in different sources how to capture different perspectives iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 31. Activists prominent on the new web through different channels by nature multi- perspective, biased & emotional iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 32. Activists prominent on the new web through different channels by nature multi- perspective, biased & emotional iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 33. “All protest events Greenpeace participated in.” Mapping Online blogs, news, activists websites Networks of Activism iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 34. • build visualizations • create appropriate analytics“All protest events • answer questions of end users & social in.” Greenpeace participated scientists Mapping Online blogs, news, activists websites Networks of Activism iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 35. Extracting Historical Events • objects (digitized • What happened artworks and before/after? artifacts) • Who does what, • events (concrete when, and where? particulars) • All bomb attacks in • entities (actors, the 1950s locations, periods) • In what events did • narratives Indonesia participate? (organization of events) • ‘Grand narratives’ iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 36. iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 37. • generate meaningful event sequences • capture the different perspectives • serve both end users & history researchers iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 38. Timelines from Text “al-Qaeda activities in Syria” 4 right, 2 wrong, 3 missing events two have no explicit times & are in the wrong order One involved al-Qaeda but took place in Jordan on the Syrian border does a fuzzy task require fuzzier metrics? iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 39. Timelines from Text “al-Qaeda activities in Syria” 4 right, 2 wrong, 3 missing events two have no explicit times & are in the wrong order One involved al-Qaeda but took place in Jordan on the Syrian border does a fuzzy task require fuzzier metrics? iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 40. Machine Reading build event timelines from text in 2 example domains • NFL: news articles on football; • Ontology: 4 classes, 20 relations • Intel: news articles on terrorist events; • Ontology: 20 Classes, 50 relations iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 41. Why is event semantics hard? iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 42. According to NLP tradition Gather Extract 1 2 your source material events and properties Analyze Visualize 3 4 statistics, timelines, etc. find links between events iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 43. but for events we stumble Gather Extract 1 2 your source material events and properties Analyze Visualize 3 4 statistics, timelines, etc. find links between events iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 44. but for events we stumble Gather Extract experts typically: 1 2 define a problem annotate ground truth your source material events and properties train Analyze Visualize 3 evaluate 4 statistics, timelines, etc. find links between events iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 45. Closed World Dictatorship 1. domain experts define the meaning 2. using limited vocabulary 3. aim for agreement to fix the problem of high disagreement for events experts enforce more tyranny - stricter rules comparatively little annotated data for training & evaluation of event detection systems iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 46. But the World is Open 1. events have multiple dimensions 2. each dimension has levels of granularity 3. people have different views on both all this leads to very complex semantics iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 47. and our goal is ... 1. not to enforce agreement 2. to capture different view points 3. to teach machines to reason in the disagreement space Flickr: elkabong iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 48. Position Artificially restricting humans d Machines oes not h will learn elp mach from dive ines to le arn. rsity Flickr: elkabong iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 49. Professional Dictatorship of the Closed World • Museum, libraries, archives & researchers have been dominating the views. • Controlled vocabularies & annotation schemes were leading. • Professionals enforced agreement among themselves. • End-users needs & tasks are not considered. iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 50. there is a tiny overlap between end-user terminology & professional annotations the latter are typically coarse-grained & refer to entire object / topic Flickr: ganzelka iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 51. only 1,900 tags (32,200 in total) match in vocabularies 257 in people (83 validated) 1,661 in geo (666 validated) 9,796 validated, but no match in professional vocabulary 8% professional vocab 23 % lexical vocab 63% meaningful Google matches iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 52. Amateur democracy of the Open World • Once the Web opened the world of information, professional dictatorship clashed with end-users democracy. • What professionals consider interesting, relevant or important does not match what users think of it. • Amateurs cannot find what they were searching for. (c) banksy iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 53. people are interested in different annotation categories than the professionals Flickr: ganzelka iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 54. Tag sample: 1,343 verified tags of 5 random video fragments Abstract General Specific Total Video aspects that are Who 10 166 177 31% described by those tags: 5 12 What 73 563 12 57% non-visual (0) Where 0 68 8 7% perceptual (11), e.g. color When 4 31 31 6 5% conceptual (1,332) Total 7% 74% 9% Object tags (1,313) Scene tags (30) 195 tags (adverbs & adjectives) couldn’t be classified iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 55. Harnessing the Crowd • we include the user’s opinion as first class citizens. • this brings the need to combine all these (different) opinions into a system of opinions that makes sense. • new solutions are needed, e.g. crowdsourcing of perspectives on events that exploit disagreementFlickr: AmyJanelle iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 56. What do People Disagree on? are sub-events always mere parts? are “mentions” meaningful for events? are events coreferential across documents? (e.g. perspectives, observations) iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 57. the bombing targeted a housing development in Baghdad, killing 3 and injuring 13 indistinguishable by people, confusable: is bombing part of killing, or killing part of bombing? What about targeting? “merelogically extensional” (i.e arbitrary): container bursting into fragments as a result of explosion some events don’t exist: an action by military forces prevented the bombing. iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 58. Disagreement Framework • ontology: disagreements on the basic status of events themselves as referents of linguistic utterances, e.g. are people events or do events exist at all. • granularity: disagreements that result from issues of granularity, e.g. the location being a country, region, or city, the time being a day, week, month, etc. • interpretation: disagreements that result from (non- granular) ambiguity, differences in perspective, or error in interpreting an expression, e.g. classifying a person as a terrorist/hero, ”October Revolution” took place in September. iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 59. Disagreement Framework • ontology: disagreements on the basic status of events themselves as referents of linguistic utterances, e.g. are people events or do events exist at all. • granularity: disagreements that result from issues of granularity, e.g. the location being a country, region, or city, the time being a day, week, month, etc. • interpretation: disagreements that result from (non- granular) ambiguity, differences in perspective, or error in interpreting an expression, e.g. classifying a person as a terrorist/hero, ”October Revolution” took place in September. iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 60. Granularity Disagreement spatial, temporal, participants compositional, classificational iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 61. iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 62. iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 63. Event Participants Disagreement Israeli Prime minister 10% 50% Government Benjamin Netanyahu Israeli Cabinet 15% his Cabinet 15% 35% Benjamin {TOLD} Netanyahu Benjamin Israeli Prime Netanyahu’s 5% 15% minister Cabinet Cabinet 45% iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 64. Temporal Disagreement Prime minister 50% Benjamin 50% Sunday Netanyahu March 1, 1998 25% 35% Benjamin {TOLD} March 1998 15% Netanyahu Spring 1998 5% Israeli Prime 15% minister iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 65. Spatial Disagreement Southern 35%30% Israel Lebanon {WILLING TO WITHDRAW} Lebanon 45%65% Israels Northern Frontier Middle East 10% iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 66. Approach Principles 1. tolerate, capture & exploit disagreement 2. understand the range of disagreements by creating a space of possibilities with frequencies & similarities 3. score the machine output based on where it falls in this space 4. adaptable to new annotation tasks Flickr: auroille iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 67. it  seems  to  refer  to  an   Top  Israeli  officials  SENT  strong   does not inference  or   new  SIGNALS  Sunday  that  Israel   refer to communicated  feeling   wants  to  withdraw  from  southern   an event more  than  specific   Lebanon,  ... event. a  group  of  people  did   refers to something  specific  at  a   an event specific  point  in  6me. the  actors  in  ques6on   (top  Israeli  officials)   refers to performed  an  ac6on   an event during  a  specified  6me   (Sunday). it  refers  to  what  the   israelis  did  on  sunday,   a  specific  6me. iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 68. it  is  not  a  par6cular   That  1978  resolu6on  calls  for   movement  that  has  or  is   Israels  uncondi6onal   does not going  on  but  a  request  that   WITHDRAWAL  from  the  self-­‐ refer to the  country  of  Israel   declared  security  zone  it   an event remove  their  forces  from   occupies  in  south  Lebanon,  ... the  zone  they  occupy. does not refer to an event the  sentence  is  speaking  of   a  demand  for  a  withdrawal   that  had  not  yet  occurred. refers to an event Because  it  is  describing  a   historical  issue  concerning   the  resolu6on  of  1978 iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 69. The Dark Side of Crowdsourcing Disagreement • disagreement is beautiful, except when it results from spamming • crowdsourcing has to account for people that want to get paid for not doing any work • spammers generate disagreement for the wrong reasons • most spam detection requires gold standard iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 70. Spam or not? • cut & paste from text • identical to other explanations • much shorter time than the average • low trust value of the worker • shorter than 5-6 words iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 71. Spam or not? + low worker trust Prime  Minister  Benjamin   does not Because  being  told   Netanyahu  TOLD  his  Cabinet  on   refer to something  doesnt   Sunday  that  Israel  was  willing  to  ... an event seem  like  an  event. Top  Israeli  officials  sent  strong  new   + low worker trust signals  Sunday  that  Israel  wants  to   refers to Because  the  WAR  is   withdraw  from  southern  Lebanon,   an event being  described  as  a   where  a  costly  WAR  of  aTri6on   costly  event. has  been  claiming  soldiers  lives. + low worker trust Top  Israeli  officials  sent  strong  new   Because  Israel  WANTS   signals  Sunday  that  Israel  WANTS   refers to TO  WITHDRAW  from   TO  WITHDRAW  from  southern   an event Lebanon. Lebanon,  ... Top  Israeli  officials  sent  strong  new   + short time refers to Because  WANTS  TO   signals  Sunday  that  Israel  WANTS   an event WITHDRAW  is  an  ac6on. TO  WITHDRAW    ... iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 72. Motivation-Verification Method • 2-stage method: • disagreement collection + motivation • spam filtering = motivation judgement • Additionally: • sample the motivation stage to manually extract gold standard for stage 2 iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 73. Extraction of Putative Events • a new way of measuring ground truth input: Manual selection of putative events Gold Questions input: • output of A Phase I: Phase I: a new set of semantic features for learning A. Collect event input: annotations + output of A C. Filtering spam motivations event annotations in event extraction input: input: input: list of events list of events list of events Phase III: Phase IV: Phase II: A. Collect event A. Collect event A. Collect event types modalities + + motivations role fillers + motivations motivations input: input: input: output of A output of A output of A input: input: input: output of A Manual output of A Manual output of A Manual selection of selection of selection of Gold Questions Gold Questions Gold Questions Phase II: Phase IV: Phase III: B. Filtering spam B. Filtering spam B. Filtering spam event types event modalities event role fillers iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 74. Position Artificially restricting humans d Machines oes not h will learn elp mach from dive ines to le arn. rsity Flickr: elkabong iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 75. Position ts are p art of s ue ne sof even greeme nt & vag human disa vent se mantics The the e Flickr: elkabong iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 76. finally ... n ny l ty ra ifu ut t he be a n d is e t en re em isag d Flickr: elkabong iSemantics2012 Lora Aroyo @laroyoMonday, September 10, 12
  • 77. Acknowledgements Roxane Segers Iina Hellsten Chiel van den Akker Geertje Jacobs Frank de Bakker Marteen Brinkerink Bibiana Armenta Piek Vossen Thomas Johan Chris Welty Michiel Guus Schreiber Ploeger Oomen Hildebrand Riste Gligorov Marieke Lourens van Susan Jacco van Geert-Jan Lotte Belice van Erp der Meij Legêne Ossenbruggen Houben BaltussenMonday, September 10, 12
  • 78. Questions? @laroyo http://lora-aroyo.orgMonday, September 10, 12

×