0
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 1
Events in Multimedia -
Theory, Model, Appli...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 2
Motivation
• Events are a natural abstracti...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 3
Theory
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 4
Brothers at enmity with each other …
Event
...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 5
What is an event?
• Perduring entities that...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 6
Brothers at enmity with each other …
• Some...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 7
Event Object
Brothers at enmity with each o...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 8
Model
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 9
Events need to be modeled …
• … and are use...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 10
Emergency
Control Center
Forward
Liaison
O...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 11
Requirements to a Common Event
Model*
• Pa...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 12
Survey on Event Models & Systems
…
Partici...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 13
Survey on Event Models & Systems
…
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 14
Ontology Patterns of Event-Model-F
• Event...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 15
6 Patterns of the Event-Model-F
• (1) Part...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 16
Example: Descriptions ‘n’ Situations
• Dom...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 17
(1) Participation Pattern
• Participation ...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 18
(3) Causality Pattern
• Event (cause) impl...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 19
Use Case: Emergency Response
Domain ontolo...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 20
Application
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 21
• Which bakery close by is open on Sunday?...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 22
Different Sources of Social Events …
• Ope...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 23
Event Model for Social Media!
• Derive a s...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 24
• Exploration of POIs and
events in real-t...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 25
Mediator-based Architecture
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 26
Update Event 003Add Event 003
Incremental ...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 27
Provider …
Entity Resolution in Detail
Pro...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 28
Entity Resolution vs. Data Delivery
• Perc...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 29
Faceted Exploration of Events & Objects
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 30
Exploring Time as Natural as Space
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 31
• 18 subjects, average age of 25.5 (SD=3.0...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 32
Evaluation in the Field (2)
• Session last...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 33
Conclusion
• Events are an important conce...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 34
But there is a long interest …
• … in this...
Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 35
Try it out …in Barcelona!
Upcoming SlideShare
Loading in...5
×

Events in Multimedia - Theory, Model, Application

247

Published on

Talk by Ansgar Scherp.

Title: Events in Multimedia - Theory, Model, Application

Event: Workshop on Event-based Media Integration and Processing, ACM Multimedia, 2013

Published in: Science, Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
247
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Events in Multimedia - Theory, Model, Application"

  1. 1. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 1 Events in Multimedia - Theory, Model, Applicat ion Workshop on Event-based Media Integration and Processing, ACM Multimedia, 2013 Juniorprof. Dr. habil. Ansgar Scherp mail@ansgarscherp.net
  2. 2. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 2 Motivation • Events are a natural abstraction of human experience • Events are everywhere! • Lifelogs • Experience sharing • Emergency response • Cultural heritage • News • News • Sports • Surveillance • …
  3. 3. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 3 Theory
  4. 4. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 4 Brothers at enmity with each other … Event Object Object Event • Earlier: object-based and entity-based systems • Now: applications that consider events at least as important as objects vs.
  5. 5. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 5 What is an event? • Perduring entities that unfold over time • Occurrences in which humans participate • Subject to discussions and interpretations by humans • Enduring entities that unfold over space Object Event
  6. 6. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 6 Brothers at enmity with each other … • Some philosophers consider objects as 4D • Extend across time just as they do in space Casati R, Varzi A (2006) Events. Stanford encyclopedia of philosophy. http://plato.stanford.edu/entries/events Object Event
  7. 7. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 7 Event Object Brothers at enmity with each other … • Events and objects as first class entities • Events and objects require each other! • For example, in DOLCE ‘is participant in’ ‘has participant’ • … not necessarily!
  8. 8. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 8 Model
  9. 9. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 9 Events need to be modeled … • … and are useful in a variety of domains • Lifelogs • Multimedia-based experience sharing • Emergency response • Cultural heritage • News • Sports • Surveillance • …
  10. 10. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 10 Emergency Control Center Forward Liaison Officer Documentary support Calls to report about a power outage Creates incident with audio recording Request to report about a flooded cellar Reports by taking photos etc. Emergency Response Coordination Emergency Hotline Fire Department Police Department Coordinate and keep up to date Report and update about the incident Coordinate and keep up to date Report and update about the incident Citizen • Several emergency response entities are involved • Using different event-based systems • Common understanding of multimedia information needed to efficiently communicate between ERs Snapped pole image from: http://www.dailymail.co.uk/ Emergency Response Scenario
  11. 11. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 11 Requirements to a Common Event Model* • Participative aspect • Temporal aspect • Spatial aspect • Structural aspect • Mereology (composition) • Causality • Correlation • Interpretation • Experiential aspect (documentation) * Analysis of 21 models and systems [SM13, SSF+12, SSF+09]
  12. 12. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 12 Survey on Event Models & Systems … Participa- tion
  13. 13. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 13 Survey on Event Models & Systems …
  14. 14. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 14 Ontology Patterns of Event-Model-F • Event-Model-F* defines six ontology patterns • Ontology design pattern similar to SE • Build on top of DOLCE+DnS Ultralight ontology • Cf. theory on events and objects • Provides Description and Situation pattern • Specified in Web Ontology Language (OWL) • Formalized in Description Logics * Homage to event model E by Westermann and Jain
  15. 15. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 15 6 Patterns of the Event-Model-F • (1) Participation pattern • (2) Mereology pattern (composition) • (3) Causality pattern • (4) Correlation pattern • (5) Documentation pattern • (6) Interpretation pattern • All based on Descriptions and Situations (DnS) • Contextualization of events and objects w.r.t. specific situations
  16. 16. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 16 Example: Descriptions ‘n’ Situations • DomesticPowerOutage- Description defines roles • AffectedObjectRole • AffectedPersonRole • AffectedPersonRole • … • DomesticPowerOutage- Situation defines objects house-1 : Building paul-1 : NaturalPerson sandy-1 : NaturalPerson … • In a different situation, NaturalPerson paul-1 may play a FireFighterRole • People may have differing opinions about the cause of the power outage • … Image source: Wikipedia Classify
  17. 17. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 17 (1) Participation Pattern • Participation of living and non-living objects in events • Reuse of domain knowledge Roles the entities play Real world entities EventParticipationDescription defines exactly 1 DescribedEvent defines min 1 Participant defines some LocationParameter defines some TimeParameter defines only (DescribedEvent or Participant or LocationParameter or TimeParameter) isSatisfiedBy exactly 1 EventParticipationSituation Example: Firemen and home owner are involved in an incident of a house fire.
  18. 18. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 18 (3) Causality Pattern • Event (cause) implies other event (effect) • Causal relationship holds under some justification • Causes and effects are events, and only events Description EventCausalityDescription EventCausalitySituation Situation Cause Effect Justification EventRole Concept Event Description classifies isRoleOf defines isEventIncludedIn satisfies isObjectIncludedIn Role EventCausalityDescription defines exactly 1 Cause defines exactly 1 Effect defines exactly 1 Justification defines only (Cause or Effect or Justification) isSatisfiedBy exactly 1 EventCausalitySituation Example: The event of a snapped power pole causes a power outage.
  19. 19. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 19 Use Case: Emergency Response Domain ontology by City of Sheffield, UK Events and media metadata Event details [MTAP2012] Web 2.0 content and metadata
  20. 20. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 20 Application
  21. 21. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 21 • Which bakery close by is open on Sunday? • What could I do tonight? • Which sights are in the area? Are they still open? • Existing applications – Mostly focus on location, i.e., points-of-interests – No support for integrated search for events Sun: 7am-11am Problem: Events in Social Media
  22. 22. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 22 Different Sources of Social Events … • Opening hours of a shop like a bakery • Concert of favorite rock star in my town • Café serving English tea in my neighborhood • Happy hour at a bar • Special sales at the shopping mall • … …
  23. 23. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 23 Event Model for Social Media! • Derive a specific model of social media events • Use notions of events and objects defined in Event-Model-F and specialize it for social media • Focus on what is needed in the domain –(1) Participation pattern –(5) Documentation pattern • Future work: (2) Mereology Pattern
  24. 24. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 24 • Exploration of POIs and events in real-time • Multithreaded integration of social data sources: KlickTel, DBpedia, GeoNames, Eventful, etc. Pad Phone mobEx: Mobile Social Media Explorer
  25. 25. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 25 Mediator-based Architecture
  26. 26. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 26 Update Event 003Add Event 003 Incremental Data Integration on Server mobEx Server Client Café Vienna Phone.: ??? Opening hours: Mo – Fr. 10 – 2 Uhr Provider 1 Provider 2 Café Vienna Phone.: 01234 Opening hours: ??? Café Vienna Mo – Fr. 10 – 2 o‘clock Café Vienna Tel.: 01234 Mo – Fr. 10 – 2 o‘clock Entity Resolution Time
  27. 27. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 27 Provider … Entity Resolution in Detail Provider 1 Provider 2 …. Weighted comparison of attributes:  Location information  Time information  Similarity score computed from single weights  Above threshold  integration Café Vienna Tel.: 01234 Opening hours: Mo – Fr. 10 – 2 Uhr S1, 15 68161 Mannheim Calculated GPS Data www.cafevienna.de/ Description: XYZ High Weight Low Weight Medium Weight High Weight Low Weight High Weight Heuristic filter (blocking):  GPS coordinates > 500m distance  Except: same address  Different entity types are not compared (Persons, Places, Events) High Weight
  28. 28. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 28 Entity Resolution vs. Data Delivery • Percentage of resources the client receives • Percentage of resolved resolution at that time
  29. 29. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 29 Faceted Exploration of Events & Objects
  30. 30. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 30 Exploring Time as Natural as Space
  31. 31. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 31 • 18 subjects, average age of 25.5 (SD=3.03) and six female • Tracked detailed activities over three weeks Evaluation in the Field . • More than 4000 single events in 234 sessions • Session is defined as • Active usage of application until it is closed • Or: after 30 seconds of inactivity
  32. 32. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 32 Evaluation in the Field (2) • Session lasted on average 2:42 minutes. • 57% of the participants used the application daily, 72% every second day or more. • Users spent more time on the map screen than on the screen showing the facets (ca. x1.5) • Time slider not used more often in the course of the three weeks
  33. 33. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 33 Conclusion • Events are an important concept in multimedia • Tremendous research conducted in the past • But, we have a lack of –Common theory of events in multimedia –Integrated tool chain to deal with events –From detection over representation to use Acknowledgements: • R. Jain, C. Saathoff, T. Franz, S. Staab, D. Schmeiß • B. Opitz, T. Sztyler, B. Pfister, M. Jess, C. Bikar, F. Knip
  34. 34. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 34 But there is a long interest … • … in this topic in the multimedia community • 1st ACM Int. Workshop on Events in Multimedia, ACM Multimedia 2009, Beijing, China • 2nd ACM Int. Workshop on Events in Multimedia, ACM Multimedia 2010, Firenze, Italy • 3rd ACM Int. Workshop on Events in Multimedia, ACM Multimedia 2011, Scottsdale, Arizona, USA • "Multimedia Activity and Event Understanding" Area, ACM Multimedia 2012, Nara, Japan • Workshop on Event-based Media Integration and Processing, ACM Multimedia 2013, Barcelona
  35. 35. Events in Multimedia Ansgar Scherp – ansgar@informatik.uni-mannheim.de Slide 35 Try it out …in Barcelona!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×