Your SlideShare is downloading. ×
0
Events
      and
         Experiences

             Ramesh Jain

   Department of Computer Science
    University of Calif...
Data Data Everywhere but Not a relevant byte of

         Information/Knowledge/Wisdom

                       to help.


...
Inventing the Future
          by
Learning from the Past




     NSF III Workshop April 23, 2010   © Ramesh Jain
What is most Important in inventing
           the future?
     Imagination is more important than
      knowledge. For k...
The only source of knowledge
        is experience.




         NSF III Workshop April 23, 2010   © Ramesh Jain
Mechanics: Statics and Dynamics

          Issac Newton
                                                           1687
  ...
Newton’s Laws
  Law 1: Every object in a state of uniform
   motion tends to remain in that state of
   motion unless an ...
The role of these laws




     NSF III Workshop April 23, 2010   © Ramesh Jain
What is the most important
 invention in the last 1000
          years?
  Newton: Mechanics
  Einstein: Theory of relati...
Life is a set of experiences.




         NSF III Workshop April 23, 2010   © Ramesh Jain
Events
Life = +
      Experiences


      NSF III Workshop April 23, 2010   © Ramesh Jain
The Story of Human Civilization
  is the Story of Experiential
Communication among Humans.


      Sharing Experiences
   ...
Share experiences: Languages




         NSF III Workshop April 23, 2010   © Ramesh Jain
Across Time: Writing




        NSF III Workshop April 23, 2010   © Ramesh Jain
Making it portable: Paper




        NSF III Workshop April 23, 2010   © Ramesh Jain
Democratization: Gutenberg’s Movable Press




               NSF III Workshop April 23, 2010   © Ramesh Jain
Telegraph: Instantaneous
        Remote




       NSF III Workshop April 23, 2010   © Ramesh Jain
Telephone: Remote Speech




       NSF III Workshop April 23, 2010   © Ramesh Jain
Photography: Recording Visual
         Experience




         NSF III Workshop April 23, 2010   © Ramesh Jain
Audio Recording: Electrical Signals
      enter recording space




            NSF III Workshop April 23, 2010   © Ramesh...
Radio: Live Broadcast




     NSF III Workshop April 23, 2010   © Ramesh Jain
Television: Live Broadcast combines
              two Media




            NSF III Workshop April 23, 2010   © Ramesh Jain
Digital Processing




   NSF III Workshop April 23, 2010   © Ramesh Jain
Internet: The Game Changer




Medium is the message. – McLuhan was right THEN.
Medium is just that – Medium.       NOW
  ...
Communicating Experiences:
                   Spatial and Temporal
Inventions                          Application Impact
...
Why is Gutenberg’s Moveable Print
          so influential?

   Brought knowledge to masses: creation,
    dissemination,...
History: Gopher to Google



    We had Internet.
    Lots of computers were connected to each other.
    Computers had...
Tim Berners-Lee thought:

   Suppose all the information stored on
    computers everywhere were linked.
   Suppose I co...
That gave us WWW, Google, …

  Search
  Maps
  Instantaneous access to immense amount of
   knowledge in documents




...
Continuing On
The 4 most popular companies that emerged in
              the last 4 years.

   Flickr
   YouTube
   Fac...
And the 4 most popular Internet
 concepts in the last 4 years.

  Social Networks
  Micro-blogging (Ambient Awareness)
...
Interesting!!!

    Flickr                        Social Networks
    YouTube                       Microblogging
   ...
Messages are clear

  New media: Text based media is not enough.
  Medium is NOT the message: medium is just
   the medi...
Crossing the Structure Chasm: Semantic Gap
Transforma)ons





        Lists, Arrays, Documents, Images …


              ...
Computer Science: Modeling the
            World


  Data
  Objects
  Events


     Both Objects and Events are essenti...
Events

  Take place in the real world.
  Captured using different sensory
   mechanism.
         Each sensor captures ...
Event Representation

          EXPERIENTIAL




     NSF III Workshop April 23, 2010   © Ramesh Jain
Events: Different types and
       Granularities
         Conferences
              Days
                   Sessions
  ...
Events Events Everywhere, …
              1- dimensional Space
Time




                NSF III Workshop April 23, 2010   ...
Butterfly effect:

Does the flap of a butterfly’s
wings in Brazil set off a
hurricane in Florida?




           NSF III W...
Continuing the Evolution of the
                  Web
    Consider a Web in which each node
         Is an event
      ...
EventWeb
       1- dimensional Space
Time




         NSF III Workshop April 23, 2010   © Ramesh Jain
Stages in EventWeb
  Cyberspace:
             Every event is ‘created’ in
 cyberspace and exists there.
    Calendars,  ...
Sensors are the Bridge

    Our 5 Senses are connectors to the world.
    We use our sensors (vision, audio, …) to
     ...
Creating a Sentient EventWeb
    Imagine Sensors Tweeting – in addition to
     human sensors.
         All video camera...
Social Pixels

    Traditional Pixels
         Photons aggregating at locations
    Social Pixels
         User intere...
Spatio-temporal aggregation using social pixels

      Intuitive visualization
          Dashboards
      Effective rep...
Architecture


             Situation
             Detection
             Engine




NSF III Workshop April 23, 2010   © R...
Micro-blogs: Twitter

    Each Tweet:
         Used to be: What are you doing
         Now it is: What’s happening
  R...
iPhone
Expression of interest in iPhone release.




                NSF III Workshop April 23, 2010   © Ramesh Jain
Operators




NSF III Workshop April 23, 2010   © Ramesh Jain
Aggregation of operators




      NSF III Workshop April 23, 2010   © Ramesh Jain
Swine flu social image and operation
        on it (segmentation)




            NSF III Workshop April 23, 2010   © Rame...
Situation awareness: Swine Flu




         NSF III Workshop April 23, 2010   © Ramesh Jain
Recommendation System: Swine Flu




           NSF III Workshop April 23, 2010   © Ramesh Jain
From Micro-events to
               Environments and Situations


                                EventBas
  Sensors      ...
Connecting Environments
                             Env. 2


  Env. 1



                    JSM
                      1
...
Extending Environments




TCS mKrishi: If experts can not go to a farm;
      lets bring the farm to experts.
           ...
Sensors in the field




    NSF III Workshop April 23, 2010   © Ramesh Jain
Query No. 276
               In text:
               I think because of salt in water, the leaves are getting
            ...
Expert Console


            Info specific   From public
            to crop         website




                         ...
Building an ‘AVIARY’

  Humans Tweeting
  Sensors Tweeting
  Environments detecting Situations and
   reporting
  Aggr...
NSF III Workshop April 23, 2010   © Ramesh Jain
NSF III Workshop April 23, 2010   © Ramesh Jain
NSF III Workshop April 23, 2010   © Ramesh Jain
NSF III Workshop April 23, 2010   © Ramesh Jain
Goal: Person-context-sensitive
         just-in-time Information

    Data from various sensors, including human
     sen...
Thanks for your time and attention.




   For questions: jain@ics.uci.edu
            NSF III Workshop April 23, 2010   ©...
Upcoming SlideShare
Loading in...5
×

Nsf iii events and experiences 100422

1,025

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,025
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Nsf iii events and experiences 100422"

  1. 1. Events and Experiences Ramesh Jain Department of Computer Science University of California, Irvine jain@ics.uci.edu NSF III Workshop April 23, 2010 © Ramesh Jain
  2. 2. Data Data Everywhere but Not a relevant byte of Information/Knowledge/Wisdom to help. Too many silos with too much data = CONFUSION NSF III Workshop April 23, 2010 © Ramesh Jain
  3. 3. Inventing the Future by Learning from the Past NSF III Workshop April 23, 2010 © Ramesh Jain
  4. 4. What is most Important in inventing the future?   Imagination is more important than knowledge. For knowledge is limited to all we now know and understand, while imagination embraces the entire world, and all there ever will be to know and understand. Albert Einstein NSF III Workshop April 23, 2010 © Ramesh Jain
  5. 5. The only source of knowledge is experience. NSF III Workshop April 23, 2010 © Ramesh Jain
  6. 6. Mechanics: Statics and Dynamics Issac Newton 1687 1643- 1727 Statics is the branch of mechanics concerned with the analysis of loads (force, torque/moment) on physical systems in static equilibrium, that is, in a state where the relative positions of subsystems do not vary over time, or where components and structures are at a constant velocity. The study of moving bodies is known as dynamics, and in fact the entire field of statics is a special case of dynamics. From Wikipedia NSF III Workshop April 23, 2010 © Ramesh Jain
  7. 7. Newton’s Laws   Law 1: Every object in a state of uniform motion tends to remain in that state of motion unless an external force is applied to it.   Law 2: The relationship between an object's mass m, its acceleration a, and the applied force F is F = ma.   Law 3: For every action there is an equal and opposite reaction. http://teachertech.rice.edu/Participants/louviere/Newton/ NSF III Workshop April 23, 2010 © Ramesh Jain
  8. 8. The role of these laws NSF III Workshop April 23, 2010 © Ramesh Jain
  9. 9. What is the most important invention in the last 1000 years?   Newton: Mechanics   Einstein: Theory of relativity.   Babbage: Digital Computer.   Tesla: AC   Gutenberg: Moveable print   Edison: Electric Bulb   . . . NSF III Workshop April 23, 2010 © Ramesh Jain
  10. 10. Life is a set of experiences. NSF III Workshop April 23, 2010 © Ramesh Jain
  11. 11. Events Life = + Experiences NSF III Workshop April 23, 2010 © Ramesh Jain
  12. 12. The Story of Human Civilization is the Story of Experiential Communication among Humans. Sharing Experiences Recording Experiences NSF III Workshop April 23, 2010 © Ramesh Jain
  13. 13. Share experiences: Languages NSF III Workshop April 23, 2010 © Ramesh Jain
  14. 14. Across Time: Writing NSF III Workshop April 23, 2010 © Ramesh Jain
  15. 15. Making it portable: Paper NSF III Workshop April 23, 2010 © Ramesh Jain
  16. 16. Democratization: Gutenberg’s Movable Press NSF III Workshop April 23, 2010 © Ramesh Jain
  17. 17. Telegraph: Instantaneous Remote NSF III Workshop April 23, 2010 © Ramesh Jain
  18. 18. Telephone: Remote Speech NSF III Workshop April 23, 2010 © Ramesh Jain
  19. 19. Photography: Recording Visual Experience NSF III Workshop April 23, 2010 © Ramesh Jain
  20. 20. Audio Recording: Electrical Signals enter recording space NSF III Workshop April 23, 2010 © Ramesh Jain
  21. 21. Radio: Live Broadcast NSF III Workshop April 23, 2010 © Ramesh Jain
  22. 22. Television: Live Broadcast combines two Media NSF III Workshop April 23, 2010 © Ramesh Jain
  23. 23. Digital Processing NSF III Workshop April 23, 2010 © Ramesh Jain
  24. 24. Internet: The Game Changer Medium is the message. – McLuhan was right THEN. Medium is just that – Medium. NOW NSF III Workshop April 23, 2010 © Ramesh Jain
  25. 25. Communicating Experiences: Spatial and Temporal Inventions Application Impact Languages Communicate symbolic experiences Written Languages Record Symbolic experiences (time) Paper Portability (space) Most influential Print Invention in history distribution (time and space) Mass Telegraph Remote narrow communication (space) Telephone Remote analog communication (space) Radio Analog broadcasting of sound (space) Television Combining two senses – media (space) Recording media Photos, audio, video (time) Digital processing Machine enhancement and processing Future of Internet Communication Interactive Multimedia communication NSF III Workshop April 23, 2010 © Ramesh Jain
  26. 26. Why is Gutenberg’s Moveable Print so influential?   Brought knowledge to masses: creation, dissemination, and access .   Resulted in increased awareness of scientific approaches.   Reduced misinformation and influence of religions. NSF III Workshop April 23, 2010 © Ramesh Jain
  27. 27. History: Gopher to Google   We had Internet.   Lots of computers were connected to each other.   Computers had files on them.   We had GOPHER and other FTP mechanisms. NSF III Workshop April 23, 2010 © Ramesh Jain
  28. 28. Tim Berners-Lee thought:   Suppose all the information stored on computers everywhere were linked.   Suppose I could program my computer to create a space in which anything could be linked to anything. Others – including Bush -- had that idea earlier but the technology was not ready. NSF III Workshop April 23, 2010 © Ramesh Jain
  29. 29. That gave us WWW, Google, …   Search   Maps   Instantaneous access to immense amount of knowledge in documents NSF III Workshop April 23, 2010 © Ramesh Jain
  30. 30. Continuing On The 4 most popular companies that emerged in the last 4 years.   Flickr   YouTube   Facebook   Twitter NSF III Workshop April 23, 2010 © Ramesh Jain
  31. 31. And the 4 most popular Internet concepts in the last 4 years.   Social Networks   Micro-blogging (Ambient Awareness)   Tags   Real Time Search Surprisingly, they did not come from Academia or GYM. Why??? NSF III Workshop April 23, 2010 © Ramesh Jain
  32. 32. Interesting!!!   Flickr   Social Networks   YouTube   Microblogging   Facebook   Tags   Twitter   Real Time Search What does this tell us? NSF III Workshop April 23, 2010 © Ramesh Jain
  33. 33. Messages are clear   New media: Text based media is not enough.   Medium is NOT the message: medium is just the medium.   Experiences: People want to experience and share experiences – with minimal latency.   Socialize: Family and friends remain a strong influence in all facets of life.   Family and friends are closer to each other today than ever!!! NSF III Workshop April 23, 2010 © Ramesh Jain
  34. 34. Crossing the Structure Chasm: Semantic Gap Transforma)ons
 Lists, Arrays, Documents, Images … Alphanumeric Characters Bits and Bytes NSF III Workshop April 23, 2010 © Ramesh Jain
  35. 35. Computer Science: Modeling the World   Data   Objects   Events Both Objects and Events are essential to model the real world. Remember mechanics? NSF III Workshop April 23, 2010 © Ramesh Jain
  36. 36. Events   Take place in the real world.   Captured using different sensory mechanism.   Each sensor captures only a limited aspect of the event.   Are used to understand a Situation. NSF III Workshop April 23, 2010 © Ramesh Jain
  37. 37. Event Representation EXPERIENTIAL NSF III Workshop April 23, 2010 © Ramesh Jain
  38. 38. Events: Different types and Granularities   Conferences   Days   Sessions   Talks   Purpose of the talk   Clicking for the next slide-event   Wedding   An Earthquake   The Big Bang   9/11   Formation of Google   Attending WWW   Me   My Birth,   Being here, and   Dying in 100 years. NSF III Workshop April 23, 2010 © Ramesh Jain
  39. 39. Events Events Everywhere, … 1- dimensional Space Time NSF III Workshop April 23, 2010 © Ramesh Jain
  40. 40. Butterfly effect: Does the flap of a butterfly’s wings in Brazil set off a hurricane in Florida? NSF III Workshop April 23, 2010 © Ramesh Jain
  41. 41. Continuing the Evolution of the Web   Consider a Web in which each node   Is an event   Has informational as well as experiential data   Is connected to other nodes using   Referential links   Structural links   Relational links   Causal links   Explicit links can be created by anybody   This EventWeb is connected to other Webs. NSF III Workshop April 23, 2010 © Ramesh Jain
  42. 42. EventWeb 1- dimensional Space Time NSF III Workshop April 23, 2010 © Ramesh Jain
  43. 43. Stages in EventWeb   Cyberspace: Every event is ‘created’ in cyberspace and exists there.   Calendars, Status updates, Twitter,   Physical and Cyberspace: Events in real space are also linked. Sentient EventWeb linked with the Web.   Foursquare, Geomapping, my camera, … NSF III Workshop April 23, 2010 © Ramesh Jain
  44. 44. Sensors are the Bridge   Our 5 Senses are connectors to the world.   We use our sensors (vision, audio, …) to experience the world.   Sensors are placed for ‘detecting events’.   Would you put a sensor if nothing interesting ever happens at a place?   Sensors are the bridge connecting the Cyberspace and the Real World. NSF III Workshop April 23, 2010 © Ramesh Jain
  45. 45. Creating a Sentient EventWeb   Imagine Sensors Tweeting – in addition to human sensors.   All video cameras   Your refrigerator   Each Phone   Many more …   Infrastructure to create and link all such events.   Filter – in Real Time   Detect Situations   Create proper Situation Indexes   Create alerts for relevant recipients. NSF III Workshop April 23, 2010 © Ramesh Jain
  46. 46. Social Pixels   Traditional Pixels   Photons aggregating at locations   Social Pixels   User interest at geo-locations   Create social Image, social Video…   Image Processing operators Situation Detection operators NSF III Workshop April 23, 2010 © Ramesh Jain
  47. 47. Spatio-temporal aggregation using social pixels   Intuitive visualization   Dashboards   Effective representation for spatio-temporal data   Acts as common format for representing any geo-spatial data e.g. weather, demographics, laws, interest levels …   Applicability of media processing inspired tools e.g. Convolution   Image processing inspired, but semantics are geo- spatial NSF III Workshop April 23, 2010 © Ramesh Jain
  48. 48. Architecture Situation Detection Engine NSF III Workshop April 23, 2010 © Ramesh Jain
  49. 49. Micro-blogs: Twitter   Each Tweet:   Used to be: What are you doing   Now it is: What’s happening   Really an event reported by human sensors   Can associate experiential data along with information.   Time and location associated with each Tweet I consider Twitter as a phenomena – have no investment in the company. Also, this concept can be further extended to more interesting cases. NSF III Workshop April 23, 2010 © Ramesh Jain
  50. 50. iPhone Expression of interest in iPhone release. NSF III Workshop April 23, 2010 © Ramesh Jain
  51. 51. Operators NSF III Workshop April 23, 2010 © Ramesh Jain
  52. 52. Aggregation of operators NSF III Workshop April 23, 2010 © Ramesh Jain
  53. 53. Swine flu social image and operation on it (segmentation) NSF III Workshop April 23, 2010 © Ramesh Jain
  54. 54. Situation awareness: Swine Flu NSF III Workshop April 23, 2010 © Ramesh Jain
  55. 55. Recommendation System: Swine Flu NSF III Workshop April 23, 2010 © Ramesh Jain
  56. 56. From Micro-events to Environments and Situations EventBas Sensors e Situation Physical Environment Environment Network/ MMDB Detection and Environment Model Server Transmis Control sion Actuators / Presentation Actuator / Devices Presentation Model NSF III Workshop April 23, 2010 © Ramesh Jain
  57. 57. Connecting Environments Env. 2 Env. 1 JSM 1 JSM 2 Env. 3 Env. 5 Env. 4 NSF III Workshop April 23, 2010 © Ramesh Jain
  58. 58. Extending Environments TCS mKrishi: If experts can not go to a farm; lets bring the farm to experts. NSF III Workshop April 23, 2010 © Ramesh Jain
  59. 59. Sensors in the field NSF III Workshop April 23, 2010 © Ramesh Jain
  60. 60. Query No. 276 In text: I think because of salt in water, the leaves are getting damages. I am sending pictures of leaves. Please advise. Expert’s Advise Meaning: Please use compost in the soil in large quantity for next two years Irvine TLG October 30, 2009 © Ramesh Jain 60
  61. 61. Expert Console Info specific From public to crop website Sensor data – if applicable Voice query Ready references for expert Picture NSF III Workshop April 23, 2010 © Ramesh Jain
  62. 62. Building an ‘AVIARY’   Humans Tweeting   Sensors Tweeting   Environments detecting Situations and reporting   Aggregating and analyzing   Indexing   Accessing the Web   For every day use. NSF III Workshop April 23, 2010 © Ramesh Jain
  63. 63. NSF III Workshop April 23, 2010 © Ramesh Jain
  64. 64. NSF III Workshop April 23, 2010 © Ramesh Jain
  65. 65. NSF III Workshop April 23, 2010 © Ramesh Jain
  66. 66. NSF III Workshop April 23, 2010 © Ramesh Jain
  67. 67. Goal: Person-context-sensitive just-in-time Information   Data from various sensors, including human sensors, to understand situations.   Sensor Networks (including Billions of cameras)   Social Networks   Building (evolving) personal profile from social graph and status updates for filtering immense amount of data.   Crowd-source knowledge to improve social and environmental conditions NSF III Workshop April 23, 2010 © Ramesh Jain
  68. 68. Thanks for your time and attention. For questions: jain@ics.uci.edu NSF III Workshop April 23, 2010 © Ramesh Jain
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×