SITUATION RECOGNITION:AN EVOLVING PROBLEM FOR HETEROGENEOUS       DYNAMIC BIG MULTIMEDIA DATAVivek K. Singh1,2, Mingyan Ga...
2Sandy in New York: Situation today                              Weather                              forecastImagine     ...
An Interesting Problem           When we were data poor           – we searched for words           in documents.         ...
4Data, Information, Knowledge, Wisdom     Data is Essential.     But, we are really interested in products:          Infor...
5BIG DATA                   Variety                               Volume Big Data offers Big Opportunities.
6The   Grand         ChallengeSense making from multimodal  massive geo-social data-         streams.
Social Networks     Connecting       People
Fundamental Problem Connecting People to Resourceseffectively, efficiently, and promptly         in given situations.
11/1/12                                       9Social Life Networks          Connecting            Information            ...
10Concept Recognition: Last Century       Location   Scenes                  Environ          Trajectories                ...
Visual Concept Recognition: First research papers                      Trajectory Situation     Object Scene           Eve...
12Concept Recognition: This Century                                                Heterogeneous Media       Location   En...
13Situations: DefinitionAn actionable abstraction ofobserved spatio-temporalcharacteristics.
11/1/12   14
Overall framework                          15A) Situation                   B) Situation        C) Visualization, Modeling...
16   Challenge: Unifying Multimodal Big Data   •  Spatio-temporal-thematic (STT) real-time streams   •  E-mage as a unifyi...
17                       Situation ModelingGet_components (v){                                   USA,                     ...
18    Situation recognition: Workflow             Level	  0:	  Raw	  data	  streams	  	     e.g.	  tweets,	  cameras,	  tr...
11/1/12                                        19Billions of data sources.Selecting and combining appropriate sources to d...
11/1/12                                                                                                  20               ...
11/1/12   21
22     Building Blocks: Operators                                                             Supporting                  ...
Personalized Alerts
24Situation recognition and control                      Aggregation, Operations                                          ...
Allergy System: Beyond Twitter
26          EVENTSHOP:Recognizing situations from web streams
27Thailand Flood Mitigation
28In New York:https://twitter.com/researchrerere  Help today.
29Connecting resources: Problems andResearch Community•  Big Data is BIG in challenges and opportunities   particularly fo...
30Thanks.We Need Collaborators: EventShop.
Upcoming SlideShare
Loading in …5
×

Situation recognition acm mm 121029

1,622 views

Published on

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,622
On SlideShare
0
From Embeds
0
Number of Embeds
71
Actions
Shares
0
Downloads
20
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Situation recognition acm mm 121029

  1. 1. SITUATION RECOGNITION:AN EVOLVING PROBLEM FOR HETEROGENEOUS DYNAMIC BIG MULTIMEDIA DATAVivek K. Singh1,2, Mingyan Gao1, Ramesh Jain11 University of California, Irvine2 MIT Media LabPresenter: jain@ics.uci.edu
  2. 2. 2Sandy in New York: Situation today Weather forecastImagine What doThis I do?
  3. 3. An Interesting Problem When we were data poor – we searched for words in documents. Now that we are data rich – should we still search for words?Time has come for us to stop thinking data poor;really start thinking and behaving data rich.
  4. 4. 4Data, Information, Knowledge, Wisdom Data is Essential. But, we are really interested in products: Information, Knowledge, and Wisdom.
  5. 5. 5BIG DATA Variety Volume Big Data offers Big Opportunities.
  6. 6. 6The Grand ChallengeSense making from multimodal massive geo-social data- streams.
  7. 7. Social Networks Connecting People
  8. 8. Fundamental Problem Connecting People to Resourceseffectively, efficiently, and promptly in given situations.
  9. 9. 11/1/12 9Social Life Networks Connecting Information People Aggregation Situation Alerts and Detection CompositionAnd Queries Resources
  10. 10. 10Concept Recognition: Last Century Location Scenes Environ Trajectories Situations Single Media aware ments Location Visual Real world Visual Objects Objects Activities Events unaware Static Dynamic SPACE TIME Data = Text or Images or Video
  11. 11. Visual Concept Recognition: First research papers Trajectory Situation Object Scene Event 1960 1970 1980 1990 2000 2010 •  1963: Object Recognition [Lawrence + Roberts] •  1967: Scene Analysis [Guzman] •  1984: Trajectory detection [Ed Chang+ Kurz] •  1986: Event Recognition [Haynes + Jain] •  1988: Situation Recognition [Dickmanns]
  12. 12. 12Concept Recognition: This Century Heterogeneous Media Location Environ Situations aware ments Location Real world Objects Activities unaware Static Dynamic SPACE TIME Data is just Data. Medium and sources do not matter.
  13. 13. 13Situations: DefinitionAn actionable abstraction ofobserved spatio-temporalcharacteristics.
  14. 14. 11/1/12 14
  15. 15. Overall framework 15A) Situation B) Situation C) Visualization, Modeling Recognition Personalization, and Alerts i) Visualization C1 … ⊕v2 v3 Personal context ii) Personalization Personal v5 v6 ized STT situation Stream Available resources Emage iii) Alerts Situation
  16. 16. 16 Challenge: Unifying Multimodal Big Data •  Spatio-temporal-thematic (STT) real-time streams •  E-mage as a unifying representation(a) Pollen levels (Source: Visual) (b) Census data (Source: text file) (c) Reports on ‘Hurricanes’ (source: Twitter stream)  d) Cloud cover (Source: Satellite imagery) (e) Predicted hurricane path (source: KML) (f) Open shelters coverage(Source: KML)  
  17. 17. 17 Situation ModelingGet_components (v){ USA, 5 mins, v ϵ { Low, Mid,1)  Identify output state space 0.01x 0.01 f1 High}2)  Identify S-T bounds3)  Define component features: v2 v3 v4 v=f(v1, …, vk) @ f2 ∏ •  If (type = imprecise) •  identify learning data source, method Emage v5 v6 Emage @4)  ForEach (feature vi) { Δ ∏ Δ If (atomic) •  Identify Data source. D1 Emage Emage D2 •  Type, URL, ST bounds Δ Δ •  Identify highest Rep. level reqd. D2 D3 •  Identify operations Else Get_components(vi) }}
  18. 18. 18 Situation recognition: Workflow Level  0:  Raw  data  streams     e.g.  tweets,  cameras,  traffic,  weather,  …   … Level  1:  Unified   representa3on   Proper3es (STT  Data) STT Stream Level  2:   Aggrega3on   Proper3es Emage (Emage)  Operations Level  3:   Symbolic  rep.   Proper3es Situation (Situa3ons)
  19. 19. 11/1/12 19Billions of data sources.Selecting and combining appropriate sources to detectsituations.Interactions with different types of Users Decision Makers Individuals Want to use: Contact jain@ics.uci.edu
  20. 20. 11/1/12 20 Front  End  GUI New New E-­‐mage Alert Data Query Stream Request Source Back  End  Controller E-­‐mage  Stream Personalized   Registered Stream  Query  Processor Queries Alert  Unit E-­‐mage  Stream User  Info Registered Data Data  Ingestor Raw  Data Storage Sources API  Calls Raw  Spatial   Data  Stream Data  Cloud
  21. 21. 11/1/12 21
  22. 22. 22 Building Blocks: Operators Supporting Operator Type Data parameter(s) Output1) Data into right representation Transform … Spatio-temporal window Filter + Mask Aggregate +2) Analyze data to Classification derive features Classification method Characterization Property Growth Rate required = 125% Pattern Matching + Pattern 72% {Features}3) Use features to Learn f Learning method fevaluate situations {Situation}
  23. 23. Personalized Alerts
  24. 24. 24Situation recognition and control Aggregation, Operations Alert level = High Date: 3rd Jun, 2011 STT data Situation Detection User-Feedback Tweet: 1) Classification ‘Please visit Dr. Cureit at ‘Urrgh… sinus’ 2) Control action 4th St immediately’ Loc: NYC,Date: 3rd Jun, 2011 Theme: Allergy
  25. 25. Allergy System: Beyond Twitter
  26. 26. 26 EVENTSHOP:Recognizing situations from web streams
  27. 27. 27Thailand Flood Mitigation
  28. 28. 28In New York:https://twitter.com/researchrerere Help today.
  29. 29. 29Connecting resources: Problems andResearch Community•  Big Data is BIG in challenges and opportunities particularly for Multimedia research community.•  Situation recognition is the challenge for NOW.•  IF PUBLICATIONS motivate you, THEN this is a an opportunity to grab.•  IF you want to make an IMPACT, THEN this is an opportunity for you.
  30. 30. 30Thanks.We Need Collaborators: EventShop.

×