1. Ramesh Jain
with
Several Collaborators
8/17/2012 1

2.  Scarcity: inadequate supply, Insufficiency of
amount or supply
 Abundance: an extremely plentiful or
oversufficient quantity or supply
Proprietary and Confidential, Not For
8/17/2012 Distribution 2

3. Scarcity
Proprietary and Confidential, Not For
8/17/2012 Distribution 3

4. Abundance
Proprietary and Confidential, Not For
8/17/2012 Distribution 4

5. Proprietary and Confidential, Not For
8/17/2012 Distribution 5

6. Proprietary and Confidential, Not For
8/17/2012 Distribution 6

7. Proprietary and Confidential, Not For
8/17/2012 Distribution 7

8. We are immersed in Networks of
 People
 Things
 Events
It is now possible to be Pansophical.
8/17/2012 8

9. Past is EXPERIENCE
Present is EXPERIMENT
Future is EXPECTATION
Use your Experiences
In your Experiments
To achieve your Expectations
8/17/2012 9

10. Astrology
To
Astronomical
Volumes of
Data
8/17/2012 10

11. Proprietary and Confidential, Not For
8/17/2012 Distribution 11

12. Have been reporting events as micro-blogs
Sensors and Internet of Things are creating
and reporting even more events than
humans are.
8/17/2012 12

13.  Objects -- popular in the West.
 Relationships and Events – popular in the
East.
 Objects and Events – seems to be the new
trend.
 The Web has re-emphasized the importance
of every object and event being connected to
others -- East Meets West.

14.  Data
 Objects
 Relationships and Events

15. Recognize
Objects
Situations
Knowledge
Observe
Big Data
Act
Planning
8/17/2012
Control 15

16.  Take place in the real world.
 Captured using different sensory mechanism.
 Each sensor captures only a limited aspect of the
event.
 Can be used to bridge the semantic gap.

17.  Conferences
 Days
 Sessions
 Talks
 Purpose of the talk
 Wedding
 An Earthquake
 The Big Bang
 9/11
 Formation of Google
 Media Lab Trip
 Me
 My Birth,
 Being here, and
 Dying in 100 years.

18. People
Things
Places
Time
Experiences
Events
E by Westerman and Jain
E* by Gupta and Jain

19. Connecting
People

20. Reporting events as micro-blogs
Massive collection of events.
Facebook reports 20 Billion updates –
3 Billion Photos –
each month.

21. Time

22. Does the flap of a butterfly’s wings in Brazil set off a tornado
in Texas?

23. Atomic and Composite Events
Time

26. Current
Social
Networks
Important
Unsatisfied
Needs
8/17/2012 26

27. The World as seen through
Mobile Phones
Most attention by
Top 1.5 Technologists – so
Billion far.
Middle of the Pyramid
Middle 3.5 Billion
(MOP):
Ready, BUT …
Bottom 2 Billion
Not Ready

28.  Resources
 Physical: food, water, goods, …
 Informational: Wikipedia, Doctors, …
 Transportation
 Employment
 Spiritual
 Timeliness
 Efficiency

29. Connecting Information
People
Aggregation Situation Alerts
and
Composition
And
Detection
Queries
Resources
8/17/2012 29

30. Proprietary and Confidential, Not For
8/17/2012 Distribution 30

31. Dynamic Event Situation
Static Object Scene
Atomic Composite

32.  Situation: An actionable abstraction of
observed spatio-temporal characteristics
 Allow users to define their own spatio-
temporal features and create the situation
detection filters.
8/17/2012 32

33. Level 0: Raw data streams
e.g. tweets, cameras, traffic, weather, …
…
Level 1: Unified
representation Properties
(STT Data)
STT
Stream
Level 2:
Aggregation Properties Emage
(Emage)
Level 3:
Symbolic rep. Properties Situation
(Situations)

34. (a) Pollen levels (Source: Visual) (b) Census data (Source: text file) (c) Reports on ‘Hurricanes’ (source: Twitter stream)
d) Cloud cover (Source: Satellite imagery) (e) Predicted hurricane path (source: KML) (f) Open shelters coverage(Source: KML)
Representation for different data sources into a
common spatio-temporal format.

35. S. No Operator Input Output
1 Selection  Temporal Temporal
E-mage Set E-mage Set
2 Arithmetic & K*Temporal E-mage Temporal E-mage Set
Logical Set
3 Aggregation α Temporal E-mage set Temporal E-mage Set
4 Grouping  Temporal E-mage Set Temporal E-mage Set
5 Characterization :
•Spatial  •Temporal E-mage Set •Temporal Pixel Set
•Temporal  •Temporal Pixel Set •Temporal Pixel Set
6 Pattern Matching 
•Spatial  •Temporal E-mage Set •Temporal Pixel Set
•Temporal  •Temporal Pixel Set •Temporal Pixel Set
35
8/17/2012 35

36. Experimentation is Front End GUI
essential to deal with New
Data
New
Query
E-mage
Stream
Alert
Request
evolving unstructured Source
Back End Controller
sensory data. E-mage Stream
Personalized
Registered Stream Query Processor
Alert Unit
Inspired by
Queries
E-mage Stream
Photoshop.
User Info
Registered
Data Data Ingestor Raw Data
Storage
Sources
API Calls Raw Spatial
Data Stream
Data Cloud
8/17/2012 36

37.  Business decision making: Demand-supply
analysis, opening a new store, offer,…
 Medical : Epidemic monitoring, Asthma,
pollution effect mitigation
 Disaster relief: (hurricane, flood, fire) directing
people to appropriate resources.
 Traffic: Suggesting best routes
 Election
8/17/2012 37

38. Proprietary and Confidential, Not For
8/17/2012 Distribution 38

39. Proprietary and Confidential, Not For
8/17/2012 Distribution 39

40. Retail Store
Locations
Net Catchment
area
Proprietary and Confidential, Not For
8/17/2012 Distribution 40

41. Proprietary and Confidential, Not For
8/17/2012 Distribution 41

42. Planetary scale 1) Macro
sensing situation
Social sensors
Device sensors
+
Macro sensors
2)
Personalized
Personal situation
context
Personal life streams
+
Profile/
Preferences e.g. High Flu risk 3)
Recommend
Actions
Available resources
+
Resource data

43. into ‘high’ and ‘low ’activity zones.
Proprietary and Confidential, Not For
8/17/2012 Distribution 43

44. Macro situation
Alert Level=High
Date=12/09/10
Micro event Situational
Control Action
e.g. “Arrgggh, I controller
“Please visit
have a sore
nearest CDC
throat” •Goal
center at 4th St
(Loc=New York, •Macro Situation
immediately”
Date=12/09/10) •Rules
Level 1 personal threat + Level 3 Macro threat -> Immediate
8/17/2012
action 44

45. 8/17/2012 45

46. Flood Shelter
Classify (Flood level - Shelter)
Twitter
Flood Level
Shelter
8/17/2012 46

47. 8/17/2012 47

48. Proprietary and Confidential, Not For
8/17/2012 Distribution 48

49. 1. Alert me when major Allergy outbreak
happens in my location !
2. How healthy is today for me ?
3. What is the best location for me to undertake
outdoor activities?

50. ϵ {low,
Allergy Threat Level mid, high}

US,
24 hrs,
1 X 1 lat long
Air quality Pollen count Tweet reports
Emage Emage Emage
(air quality index) (pollen level) (number of reports)
Δ Δ Δ
Air quality S-t-t (#reports)
US, Pollen count
24 hrs, Δ
1X1lat long
US,
24 hrs, US,
1X1 lat long 24 hours,
Weather.com Twitter
2X2
Pollen.com
Twitter.com

51. ϵ {low,
Personal asthma threat mid, high}
 Thresholds
Low:{0, 0.3],
Mid: {0.3, 0.7],
High: {0.7,1}
Heart rate Sneezing severity Asthma threat level
∏  ∏
∏
Sensor stream EventShop
Twitter
Cardio device
Twitter.com Pollen.com, AQI.com,
Twitter

53. Proprietary and Confidential, Not For
8/17/2012 Distribution 53

54.  Framework tested using applications:
 Store location
 Political campaign
 Flu monitoring
 EventShop system:
 Operators implemented:
 Selection , Arithmetic & Logical, Aggregation , Grouping,
Characterization (spatial + temporal), Pattern Matching
(spatial + temporal)
 Applications tested:
 Thai flood relief
 Hurricane alerts
 Safe locations for Asthmatic patients
8/17/2012 54

55.  Scalability
 Data discovery
 Application discovery
 Conceptual modeling of situations
 Richer operation set
 User experience
Proprietary and Confidential, Not For
8/17/2012 Distribution 55

56.  Make EventShop Robust
 Develop system to deal with BIG DATA
 Experiment with many applications
Proprietary and Confidential, Not For
8/17/2012 Distribution 56

57. Proprietary and Confidential, Not For
8/17/2012 Distribution 57