Your SlideShare is downloading. ×
Event Web
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Event Web

2,388
views

Published on

Representing and creating events and lining them in EventWeb may result in powerful systems to represent dynamic situations. This is one of the next evolutionary stages of the Web.

Representing and creating events and lining them in EventWeb may result in powerful systems to represent dynamic situations. This is one of the next evolutionary stages of the Web.

Published in: Technology, Spiritual

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,388
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
30
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. EventWeb Ramesh Jain and Several Collaborators Contact: jain@ics.uci.edu ICIG Xian Sept 21, 2009 © Ramesh Jain 1
  • 2. Today • Data, Experience, and Knowledge • Semantic Gap • Bridging the Semantic Gap • EventWeb • Applications: – EMME: Experiential Media Management Environment – E2E: Environment to Environment Connection ICIG Xian Sept 21, 2009 © Ramesh Jain 2
  • 3. Name 3 most popular companies that emerged in the last 3 years. • YouTube • Facebook • Twitter ICIG Xian Sept 21, 2009 © Ramesh Jain 3
  • 4. Twitter’s Growth This is a growth pattern I have never seen in terms of speed - not in the nearly 25 years I've been watching this industry. --- John Battelle ICIG Xian Sept 21, 2009 © Ramesh Jain 4
  • 5. Name 3 most popular Internet concepts in the last 3 years. • Social Networks • Micro-blogging (Ambient Awareness) • Tags News: None of these come from GYM or Academia. ICIG Xian Sept 21, 2009 © Ramesh Jain 5
  • 6. Interesting!!! • YouTube • Social Networks • Facebook • Microblogging • Twitter • Tags What does this tell us? ICIG Xian Sept 21, 2009 © Ramesh Jain 6
  • 7. Messages for Research Community People want: • New media: Text based media is not enough. • Experiences: People want to experience and share experiences – with minimal latency. • Socialize: Family and friends remain a strong influence in all facets of life. – Family and friends are closer to each other today than ever!!! ICIG Xian Sept 21, 2009 © Ramesh Jain 7
  • 8. Events Life = + Experiences ICIG Xian Sept 21, 2009 © Ramesh Jain 8
  • 9. Recording experiences • Visual • Aural • Tactile • (not worry about smell and taste for the next few years) • Text • Log of activities ICIG Xian Sept 21, 2009 © Ramesh Jain 9
  • 10. ICIG Xian Sept 21, 2009 © Ramesh Jain 10
  • 11. ICIG Xian Sept 21, 2009 © Ramesh Jain 11
  • 12. The Challenge Connecting ICIG Xian Sept 21, 2009 © Ramesh Jain 12
  • 13. Transformations Lists, Arrays, Documents, Images … Alphanumeric Characters Bits and Bytes ICIG Xian Sept 21, 2009 © Ramesh Jain 13
  • 14. Semantic Gap The semantic gap is the lack of coincidence between the information that one can extract from the visual data and the interpretation that the same data have for a user in a given situation. A linguistic description is almost always contextual, whereas an image may live by itself. Content-Based Image Retrieval at the End of the Early Years Found in: IEEE Transactions on Pattern Analysis and Machine Intelligence Arnold Smeulders , et. al., December 2000 ICIG Xian Sept 21, 2009 © Ramesh Jain 14
  • 15. Semantic Gap ICIG Xian Sept 21, 2009 © Ramesh Jain 15
  • 16. History: Year 1991 – We had Internet. – Lots of computers were connected to each other. – Computers had files on them. – We had GOPHER and other FTP mechanisms. ICIG Xian Sept 21, 2009 © Ramesh Jain 16
  • 17. Tim Berners-Lee thought: • Suppose all the information stored on computers everywhere were linked. • Suppose I could program my computer to create a space in which anything could be linked to anything. Others – including Bush -- had that idea earlier but the technology was not ready. ICIG Xian Sept 21, 2009 © Ramesh Jain 17
  • 18. That resulted in the Web • DocumentWeb – Each node is a ‘Page’ or a document. – Pages are linked through explicit referential links ICIG Xian Sept 21, 2009 © Ramesh Jain 18
  • 19. Today we have • WWW that has revolutionized our life in many ways. • Sensor networks – Billions of sensor getting connected • Ambitious projects – Planetary Skin by Cisco and NASA – Smart Planet by IBM ICIG Xian Sept 21, 2009 © Ramesh Jain 19
  • 20. Can we take the Web beyond the Cyberspace? • Can things in real world be connected to other things? • Does this even make sense? ICIG Xian Sept 21, 2009 © Ramesh Jain 20
  • 21. Connectors • My 5 Senses are connectors between ‘me’ and the world. • We use our sensors (vision, audio, …) to experience the world. • Sensors could be the interface between the Cyberspace and the Real World. • Sensors are placed for ‘detecting events’. – How do you decide what sensors to put at any place? – Would you put a sensor if nothing interesting ever happens at a place? ICIG Xian Sept 21, 2009 © Ramesh Jain 21
  • 22. Modeling the World • Data (Semantic Web) • Objects (Search Companies, …) • Relationships and Events ICIG Xian Sept 21, 2009 © Ramesh Jain 22
  • 23. Evolution of ‘Thought’ • Objects -- popular in the West. • Relationships and Events – popular in the East. • Objects and Events – seems to be the new trend. • The Web has reemphasized the importance of every object and event being connected to others -- East Meets West. ICIG Xian Sept 21, 2009 © Ramesh Jain 23
  • 24. Continuing the Evolution of the Web • Consider a Web in which each node – Is an event – Has informational as well as experiential data – Is connected to other nodes using • Referential links • Structural links • Relational links • Causal links – Explicit links can be created by anybody • This EventWeb is connected to other Webs. ICIG Xian Sept 21, 2009 © Ramesh Jain 24
  • 25. Different types and Granularities • Conferences – Days • Sessions – Talks » Purpose of the talk • Wedding • An Earthquake • The Big Bang • 9/11 • 26/11 • Formation of Google • Attending WWW • Me – My Birth, – Being here, and – Dying in 100 years. ICIG Xian Sept 21, 2009 © Ramesh Jain 25
  • 26. Event Representation ICIG Xian Sept 21, 2009 © Ramesh Jain 26
  • 27. Transformations Text, Images, Audio, Video, Tactile… Alphanumeric, Pixel, Characters Bits and Bytes ICIG Xian Sept 21, 2009 © Ramesh Jain 27
  • 28. Events 1- dimensional Space Time ICIG Xian Sept 21, 2009 © Ramesh Jain 28
  • 29. EventWeb 1- dimensional Space Time ICIG Xian Sept 21, 2009 © Ramesh Jain 29
  • 30. Multimedia Storytelling • Collect information about events – Select relevant events – For each event, select appropriate information – In right media • Stories are sequence of coherent events. – Stories/Novels – Drama – Movies Present right event information using right media in right order. ICIG Xian Sept 21, 2009 © Ramesh Jain 30
  • 31. Multimedia Storytelling 1- dimensional Space Text Photo Video Time ICIG Xian Sept 21, 2009 © Ramesh Jain 31
  • 32. Stages in EventWeb • Cyberspace: Every event is ‘created’ in cyberspace and exists there. • Physical and Cyberspace: Events in real space are also linked. We have Sentient EventWeb linked with the Cyber Web. ICIG Xian Sept 21, 2009 © Ramesh Jain 32
  • 33. Experiential Media Management Environment • Event-based • Should be able to deal with ‘multimedia’ – Photos – Audio – Video – Text – Information and data – … • Searching based on events and media. • Storytelling First Photos then other media. ICIG Xian Sept 21, 2009 © Ramesh Jain 33
  • 34. Content Contenxt Context • Contenxt = Content + Context • Context is as powerful, possibly more, as content in understanding audio-visual information ICIG Xian Sept 21, 2009 © Ramesh Jain 34
  • 35. Modern Cameras • Are more than ‘Camera Obscura’: They capture an event. • Many sensors capture scene context and store it along with intensity values. • EXIF data is all metadata related to the Event. Exposure Time Aperture Diameter Flash Metering Mode ISO Ratings Focal Length Time Location (soon) Face ICIG Xian Sept 21, 2009 © Ramesh Jain 35
  • 36. Photos can be tagged using only EXIF We will also use content features and LSCOM Concepts – will soon start using them. Information from calendars and other sources will be introduced soon. ICIG Xian Sept 21, 2009 © Ramesh Jain 36
  • 37. EMME Event Cycle Event Atomic Presentation/ Event Entry Navigation EXIF Features Event Base Tags/ Context Event Grouping, Linking, 2009 ICIG Xian Sept 21, Assimilation © Ramesh Jain 37
  • 38. Using Context/Models to Build the EventWeb • Folder structure • Calendar • Social Network • EXIF Data • Event Ontology • Personal annotations • Photostream Segmentation • Event Detection from photos ICIG Xian Sept 21, 2009 © Ramesh Jain 38
  • 39. Photo Stream Segmentation Definition: given a photo stream P = { pi } Segment Event 1 Event 2 Event 3 ICIG Xian Sept 21, 2009 © Ramesh Jain 39
  • 40. EMME Event Cycle Event Atomic Presentation/ Event Entry Navigation EXIF Features Tags/ Event Base Context User Photo Annot- Event stream ations Ontology Segment. Minimize Manual Work Event Grouping, Linking, 2009 ICIG Xian Sept 21, Assimilation © Ramesh Jain 40
  • 41. EMME Event Cycle Event Atomic Presentation/ Event Entry Navigation Story Telling EXIF Explore Features Search Tags/ Event Base Context User Photo Annot- stream ations Event Segment. Ontology Event Grouping, Linking, 2009 ICIG Xian Sept 21, Assimilation © Ramesh Jain 41
  • 42. Using EMME • Searching for photo – ISWC2008 • Creating Albums: – Professional – Family – Tourism • Telling stories – What did I do in Karlsruhe • Scenario: In December 2008, I have 20,000 pictures taken in 2008. How do I (semi-automatically) select 25 to send to – My mother – The uncle that I hate Version 0.1 – My personal friend is ready – My professional friend – … ICIG Xian Sept 21, 2009 © Ramesh Jain 42
  • 43. Personal-Photo-EventWeb ICIG Xian Sept 21, 2009 © Ramesh Jain 43
  • 44. Singapore – Outdoor -- People ICIG Xian Sept 21, 2009 © Ramesh Jain 44
  • 45. People-No Face - Outdoor ICIG Xian Sept 21, 2009 © Ramesh Jain 45
  • 46. Singapore-Outdoor-Night ICIG Xian Sept 21, 2009 © Ramesh Jain 46
  • 47. Indoor no faces ICIG Xian Sept 21, 2009 © Ramesh Jain 47
  • 48. Indoor People ICIG Xian Sept 21, 2009 © Ramesh Jain 48
  • 49. Indoor Portraits ICIG Xian Sept 21, 2009 © Ramesh Jain 49
  • 50. E-mail can also be parsed. ICIG Xian Sept 21, 2009 © Ramesh Jain 50
  • 51. Photos Summarization ICIG Xian Sept 21, 2009 © Ramesh Jain 51
  • 52. Original Data Set ICIG Xian Sept 21, 2009 © Ramesh Jain 52
  • 53. Photo-Summarization using content ICIG Xian Sept 21, 2009 © Ramesh Jain 53
  • 54. Photo-Summarization using Faces ICIG Xian Sept 21, 2009 © Ramesh Jain 54
  • 55. Event Mash-up Interface Event Mash-up Interface Composite Events Event Digestor Event Digestor Clustering Annotation Event Ontology Atomic Events Event Ingestor Event Ingestor EXIF Extractor Text Parser Calendar Parser ICIG Xian Sept 21, 2009 © Ramesh Jain 55
  • 56. ICIG Xian Sept 21, 2009 © Ramesh Jain 56
  • 57. ICIG Xian Sept 21, 2009 © Ramesh Jain 57
  • 58. Sentient EventWeb • Use sensors to create events. – This conference • Infrastructure to create and link all such events. – Index – Search – Present • Capture events and put them on the Web. Imagine each sensor tweeting. ICIG Xian Sept 21, 2009 © Ramesh Jain 58
  • 59. Sentient EventWeb • Use sensors to create events. – This Seminar • Infrastructure to create and link all such events. – Index – Search – Present • Archive events and put them on the Web. ICIG Xian Sept 21, 2009 © Ramesh Jain 59
  • 60. First Version: E2E Communication Modes: • Person to Person • Device to Device – Telephone – Video Conferencing • Environment to Environment (E2E) ICIG Xian Sept 21, 2009 © Ramesh Jain 60
  • 61. Environment • Fixed objects • Active Objects • Events and Activities • Sensor to capture Objects and Events – Cameras, microphones, pressure, RFID, … • Displays and Actuators – Projectors, Displays, Speakers, … ICIG Xian Sept 21, 2009 © Ramesh Jain 61
  • 62. Key Idea • Put enough sensors and devices to make mediation disappear. • System detects events and objects and sends appropriate data streams and displays/renders incoming streams on appropriate devices • Bring back Person to Person communication ICIG Xian Sept 21, 2009 © Ramesh Jain 62
  • 63. Vision • First build Communication Nodes • Next build Web Nodes • Build Sentient EventWeb ICIG Xian Sept 21, 2009 © Ramesh Jain 63
  • 64. E2E communication: Vision Environment 1 Environment 2 Device to Device Sentient Sentient Information communication Information System Web System ICIG Xian Sept 21, 2009 © Ramesh Jain 64
  • 65. Environment: Node Architecture EventBas Sensors e Situation Physical Environment Environment Network/ MMDB based Environment Model Server Transmiss controller ion Actuators / Presentation Actuator / Devices Presentation Model ICIG Xian Sept 21, 2009 © Ramesh Jain 65
  • 66. Event detection and organization ICIG Xian Sept 21, 2009 © Ramesh Jain 66
  • 67. Joint Env. 2 Env. 1 SM JSM JSM 1 1 JSM 2 Env. 3 Env. 5 Env. 4 Shared Visualization Spaces for Environment to Environment Communication , in Workshop on Media, Arts, Science Xian Sept 21, 2009 (MAST 09), 2009. ICIG and Technology © Ramesh Jain 67
  • 68. Telemedicine Application ICIG Xian Sept 21, 2009 © Ramesh Jain 68
  • 69. Conclusion • New models for Events and Objects are required for bridging the semantic gap. • Semantics emerges due to relationships captured in Web. • EventWeb facilitates emerging applications dealing with multimedia data. ICIG Xian Sept 21, 2009 © Ramesh Jain 69
  • 70. Thanks for your time and attention. For questions: jain@ics.uci.edu ICIG Xian Sept 21, 2009 © Ramesh Jain 70