0
12/5/12	     1	  
•    Social	  systems	  rely	  on	  primi0ve	  technology.	  •    Big	  Data	  has	  opened	  Big	  Opportuni0es.	  •    S...
Send:	   •      Comments.	   •      Sugges1ons.	   •      Collabora1on	  opportuni1es.	   	   •      jain@ics.uci.edu	   •...
Intelligent:	  	  displaying	  or	  characterized	  by	  quickness	  of	  understanding,	  sound	  thought,	  or	  good	  ...
•  Introduc1on	  •    Social	  Systems	  •    Intelligent	  Social	  Systems	  •    Designing	  Intelligent	  Social	  Sys...
An	  Interes0ng	  Situa0on	  	                       When	  we	  were	  data	  poor	  –	                       we	  search...
Variety	                                                     Volume	  Big	  Data	  offers	  Big	  Opportuni4es.	  But,	  …....
Most	  aOen0on	  by	            Top	  1.5	               Technologists	  –	  so	  far.	            Billion	  Middle	  	  4...
Data	  is	  Essen0al.	  	  	  But,	  we	  are	  really	  interested	  in	  its	  products:	  	             	  Informa0on,	...
Recognize	                                                 Objects	                                                 Situa0...
Past is EXPERIENCE              Present is EXPERIMENT              Future is EXPECTATION                  Use your Experie...
 	  	  Astrology	                             To	                                   Astronomical	                         ...
We	  are	  immersed	  in	  Networks	  of	    •  People	    •  Things	    •  Events	  It	  is	  now	  possible	  to	  be	  ...
Our	  mobile	  wireless	  infrastructure	  can	  be	  “reality	  mined”	  to	  understand	  the	  paOerns	  of	  human	  b...
•  Objects	  -­‐-­‐	  popular	  in	  the	  West.	  •  Rela0onships	  and	  Events	  –	  popular	  in	  the	  East.	  •  Ob...
•  Data	  	  •  Objects	  	  •  Rela0onships	  and	  Events	  
•  Take	  place	  in	  the	  real	  world.	  •  Captured	  using	  different	  sensory	  mechanism.	      –  Each	  sensor	...
Events:	  Types	  and	  Granulari1es	           •  Conferences	                 –  Days	                      •  Sessions	...
People	  Things	  Places	  Time	  Experiences	  Events	        E	  	  by	  Westerman	  and	  	  Jain	                 	   ...
Sense	  making	  from	  mul1modal	  massive	  geo-­‐social	  data-­‐streams.	  	                                          ...
 •  Introduc1on	  • Social	  Systems	  •    Intelligent	  Social	  Systems	  •    Designing	  Intelligent	  Systems	  •   ...
Poli0cs	                     Religion	                Educa0on	  Health	                     Economics	  
Connec4ng	  People	  to	  Resources	  	  effec4vely,	  efficiently,	  and	  promptly	  	         in	  given	  situa4ons.	  
•  Minimize	  hunger	  in	  the	  world.	  •  Maximize	  female	  educa1on	  in	  India.	  •  Minimize	  ‘deaths’	  in	  t...
•  System:	     –  	  	  A	  set	  of	  diverse	  parts	  forming	  a	  whole.	     –  Parts	  are	  put	  together	  with...
•  A	  social	  system	  is	  composed	  of	  persons	  or	     groups	  who	  share	  a	  common	  objec1ve.	  •  An	  in...
•    Persons	  •    Families	  •    Organiza1ons	  •    Communi1es:	  City,	  State,	  Country	  •    Socie1es	  •    Cult...
•  Top	  Down:	     –  The	  social	  system	  determines	  its	  parts.	     –  People’s	  behavior	  determined	  by	  s...
•  Each	  social	  en1ty	  is	  a	  holon.	  •  Holon:	  Each	  en1ty	  is	  simultaneously	  a	  part	  and	  a	     whol...
•  The	  primary	  ‘currency’	  	  of	  a	  social	  system	  is	     informa1on.	  •  System	  behavior	  can	  be	  unde...
 •  Introduc1on	  •  Social	  Systems	  •  Intelligent	  Social	  Systems	  •    Designing	  Intelligent	  Systems	  •    ...
•  Systems	  that	  perceive,	  reason,	  learn,	  and	  act	     intelligently.	  •  Adaptability	  to	  varying	  enviro...
•  Social	  systems	  that	  perceive,	  reason,	  learn,	     and	  act	  intelligently.	  •  What	  does	  ‘perceive’,	 ...
 •  Introduc1on	  •  Social	  Systems	  •  Intelligent	  Social	  Systems	  • Designing	  Intelligent	  Social	    Systems...
•  Desired	  state	  (Goal)	  •  System	  model	  and	  Control	  Signal	     (Ac0ons)	  •  Current	  State	  (for	  Feedb...
                       bserve d	  State                                                                               	   ...
Social	  Networks	  	  Connecting  People
Needs	  and	  Resources	                              Not	  even	                             FaceBook!	  
Current	                	  Social	                Networks	                Important	                Unsa1sfied	  	        ...
•  Resources	  	      –  Physical:	  food,	  water,	  goods,	  …	      –  Informa:onal:	  Wikipedia,	  Doctors,	  …	      ...
Connecting            Information                   	         	                          People              Aggregation S...
 •    Introduc1on	  •    Social	  Systems	  •    Intelligent	  Social	  Systems	  •    Designing	  Intelligent	  Systems	 ...
Connec4ng	  People	  to	  Resources	  	  effec4vely,	  efficiently,	  and	  promptly	  	         in	  given	  situa4ons.	  
•  rela1ve	  posi1on	  or	  combina1on	  of	     circumstances	  at	  a	  certain	  moment.	  •  The	  combina1on	  of	  c...
•  Situa1on	  awareness,	  or	  SA,	  is	  the	  percep1on	     of	  environmental	  elements	  within	  a	  volume	  of	 ...
•  Example	  1:	  	      –  A	  person	  shou1ng.	      –  1000	  people	  shou1ng.	           •  In	  a	  contained	  bui...
Facebook	  and	  TwiBer	               (now	  GOOGLE	  +)	  Have	  been	  repor0ng	  events	  as	  micro-­‐blogs	         ...
Time	  
Does	  the	  flap	  of	  a	  buEerfly’s	  wings	  in	  Brazil	  set	  off	  a	       tornado	  in	  Texas?	  	  	  	  
Have	  been	  repor0ng	  events	  as	  micro-­‐blogs	  Sensors	  and	  Internet	  of	  Things	  are	  crea1ng	  and	    re...
FROM TWEETS TO REVOLUTIONS
Atomic	  and	  Composite	  Events	          Time	  
•  Given	  a	  plethora	  of	  event	  data.	  How	  can	  we:	      –  Disambiguate	  relevant	  and	  irrelevant	  event...
1.  Inherent	  support	  for	  event-­‐based	  (temporal)	      reasoning	  2.  The	  ability	  of	  the	  controller	  to...
An	  ac4onable	  abstrac4on	  of	  observed	  spa4o-­‐temporal	  characteris0cs.	                                         ...
 •    Introduc1on	  •    Social	  Systems	  •    Intelligent	  Social	  Systems	  •    Designing	  Intelligent	  Systems	 ...
Data	  Type	                     1950	                        2000	                                Time	  Line	  
Situa0on	  	                                                                                      2010	                   ...
Loca1on	      Scenes	                    Environm                 Trajectories	                                           ...
Trajectory	   Situa1on	        Object	   Scene	                  Event	  1960	                1970	     1980	             ...
Heterogeneous	  Media	                        Loca1on	     Environm                                                       ...
 •    Introduc1on	  •    Social	  Systems	  •    Real	  Time	  Social	  Systems	  •    Designing	  Real	  Time	  Systems	 ...
A)	  Situa0on	  Modeling	                   B)	  Situa0on	  Recogni0on	     C)	  Visualiza0on,	  Personaliza0on,          ...
Proprietary	  and	  Confiden1al,	  Not	  For	  12/5/12	                                                           71	      ...
Aggrega1on,	  	   Opera1ons	                                                                                   Alert	  lev...
 •    Introduc1on	  •    Social	  Systems	  •    Intelligent	  Social	  Systems	  •    Designing	  Intelligent	  Systems	 ...
•  E-­‐mage	  	  	  	  	  	      –  Visualiza1on	      –  Intui1ve	  query	  and	  mental	  model	      –  Common	  spa1o	...
•  Spa1o-­‐temporal	  element	      –  STTPoint	  =	  {s-­‐t-­‐coord,	  theme,	  value,	  pointer}	  •  E-­‐mage	      –  ...
Proprietary	  and	  Confiden1al,	  Not	  For	  12/5/12	                                                           78	      ...
Proprietary	  and	  Confiden1al,	  Not	  For	  12/5/12	                                                           79	      ...
Retail	  Store	                                                                      Loca0ons	                            ...
•  Humans	  as	  sensors	    •  Space	  +	  Time	  as	  fundamental	  axes	  	    •  Real	  0me	  situa0on	  evalua0on	  (...
•  Help	  domain	  experts	  externalize	  their	  internal	        models	  of	  situa1ons	  of	  interest	  e.g.	  epide...
 Knowledge	  or	  data	  driven	  building	  blocks	                        Growth	  rate	  	                        (Flu	...
Get_components (v){                                                                                v                      ...
3)	  Instan1ate	  2)	  Revise	  1) Model	  	                                                                              ...
Level	  0:	  Raw	  data	  streams	  	      e.g.	  tweets,	  cameras,	  traffic,	  weather,	  …	                             ...
Suppor1ng	     Operator	  Type	     Data	                                           parameter(s)	                         ...
Suppor1ng	                                        Operator	  Type	     Data	                                              ...
S.	  No   Operator                        Input                                                Output1          Filter	  ∏...
•  Select	  E-­‐mages	  of	  US	  for	  theme	  ‘Obama’.	       –  ∏spa1al(region=[24,-­‐125],[24,-­‐65])	     (TEStheme=O...
Personalized	  situa0on:	  An	  ac4onable	  integra4on	  of	  a	  users	  personal	  context	  with	  surrounding	  spa4ot...
Personalized	  Situa1on	  Recogni1on:	              Operators	                                                            ...
•  IF   𝑢𝑖   𝑖𝑠𝑖𝑛   𝑧𝑗     𝑇𝐻𝐸𝑁   𝑐𝑜𝑛𝑛𝑒𝑐𝑡  ( 𝑢𝑖,   𝑛𝑒𝑎𝑟𝑒𝑠𝑡(𝑢𝑖,      𝑟𝑘))  	                1)  𝑖 𝑠𝑖𝑛(𝑢𝑖,   𝑧𝑗)  → 𝑚𝑎𝑡𝑐ℎ( 𝑢...
Billions	  of	  data	  sources.	  	  Selec0ng	  and	  combining	  appropriate	  sources	  to	  detect	  situa0ons.	  	  In...
12/5/12	     95	  
Front	  End	  GUI               New         New                          E-­‐mage                             Alert       ...
11/28/2012	     97	  
Emage                                                                                                                     ...
Macro	  situa0on	                                                                               Alert	  Level=High	       ...
•  What	  personal	  informa1on	  can	  be	  shared?	  •  How	  should	  it	  be	  shared	  to	  benefit	  the	  user?	  • ...
Macro	  situa1on	  model	                                         Asthma	  Threat	                                        ...
Personal	                                                                                      c	  ϵ	  {Low,	  mid,	      ...
Personal	                                                        threat	  level	           c	  ϵ	  {Low,	  mid,	          ...
Classify (Flood level - Shelter                                      Flood         Shelter)                Twitter        ...
12/5/12	     108	  
Proprietary	  and	  Confiden1al,	  Not	  For	  12/5/12	                                                           109	     ...
Outline	  •    Introduc1on	  •    Social	  Systems	  •    Real	  Time	  Social	  Systems	  •    Designing	  Real	  Time	  ...
•  Social	  observa1ons	  are	  now	  possible	  with	  liBle	     latency.	  •  Now	  possible	  to	  design	  social	  s...
Useful	  Links	  •  Demo:	          –  hBp://auge.ics.uci.edu/eventshop	  •  Data	  Defini1on	  Language	  Schema	         ...
Thanks	  for	  your	  1me	  and	  aBen1on.	       For	  ques1ons:	  jain@ics.uci.edu	  
Designing intelligent social systems 121205
Designing intelligent social systems 121205
Designing intelligent social systems 121205
Designing intelligent social systems 121205
Designing intelligent social systems 121205
Designing intelligent social systems 121205
Designing intelligent social systems 121205
Designing intelligent social systems 121205
Designing intelligent social systems 121205
Designing intelligent social systems 121205
Designing intelligent social systems 121205
Designing intelligent social systems 121205
Upcoming SlideShare
Loading in...5
×

Designing intelligent social systems 121205

1,333

Published on

With emerging technologies and big data, it is now possible to design intelligent social systems. In this presentation, ideas related to designing such systems are presented

Published in: Education
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,333
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
33
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Designing intelligent social systems 121205"

  1. 1. 12/5/12   1  
  2. 2. •  Social  systems  rely  on  primi0ve  technology.  •  Big  Data  has  opened  Big  Opportuni0es.  •  Situa0on  recogni0on  is  a  key  technology.  •  EventShop  may  be  useful  in  designing   Intelligent  Social  Systems.  
  3. 3. Send:   •  Comments.   •  Sugges1ons.   •  Collabora1on  opportuni1es.     •  jain@ics.uci.edu   •  Gmail,  FB,  TwiBer:  jain49  
  4. 4. Intelligent:    displaying  or  characterized  by  quickness  of  understanding,  sound  thought,  or  good  judgment.    Social  Systems:  Social  systems  are  the  paBerns  of  behavior  of  a  group  of  people  possessing  similar  characteris1cs  due  to  their  existence  in  same  society.      
  5. 5. •  Introduc1on  •  Social  Systems  •  Intelligent  Social  Systems  •  Designing  Intelligent  Social  Systems  •  Situa1on  Recogni1on  •  Concept  recogni1ons  •  Personalized  Situa1ons  •  EventShop  
  6. 6. An  Interes0ng  Situa0on     When  we  were  data  poor  –   we  searched  for  words  in   documents.     Now  that  we  are  data  rich  –   should  we  s0ll  search  for   words?    Time  has  come  for  us  to  stop  thinking  data  poor;  really  start  thinking  and  behaving  data  rich.  
  7. 7. Variety   Volume  Big  Data  offers  Big  Opportuni4es.  But,  ….  ?????   7  
  8. 8. Most  aOen0on  by   Top  1.5   Technologists  –  so  far.   Billion  Middle    4  Billion     Middle  of  the   Pyramid  (MOP):     Ready.   BoOom  2  Billion   Not  Ready  
  9. 9. Data  is  Essen0al.      But,  we  are  really  interested  in  its  products:      Informa0on,      Knowledge,  and      Wisdom.   9    
  10. 10. Recognize   Objects   Situa0ons   Knowledge   Observe   Big  Data   Act   Planning  12/5/12   Control   10  
  11. 11. Past is EXPERIENCE Present is EXPERIMENT Future is EXPECTATION Use your Experiences In your Experiments To achieve your Expectations12/5/12   11  
  12. 12.      Astrology   To   Astronomical   Volumes  of   Data    12/5/12   12  
  13. 13. We  are  immersed  in  Networks  of   •  People   •  Things   •  Events  It  is  now  possible  to  be  Pansophical.     12/5/12   13  
  14. 14. Our  mobile  wireless  infrastructure  can  be  “reality  mined”  to  understand  the  paOerns  of  human  behavior,  monitor  our  environments,  and  plan  social  development.      -­‐-­‐-­‐-­‐  Pentland  in    “Society’s  Nervous  System:  Building  Effec0ve  Government,  Energy,  and  Public  Health  Systems”   Proprietary  and  Confiden1al,  Not  For  12/5/12   14   Distribu1on  
  15. 15. •  Objects  -­‐-­‐  popular  in  the  West.  •  Rela0onships  and  Events  –  popular  in  the  East.  •  Objects  and  Events  –  seems  to  be  the  new  trend.  •  The  Web  has  re-­‐emphasized  the  importance  of   every  object  and  event  being  connected  to  others     -­‐-­‐  East  Meets  West.   Geography  of  Thought  by  Richard  NisbeB  
  16. 16. •  Data    •  Objects    •  Rela0onships  and  Events  
  17. 17. •  Take  place  in  the  real  world.  •  Captured  using  different  sensory  mechanism.   –  Each  sensor  captures  only  a  limited  aspect  of  the   event.  •  Can  be  used  to  bridge  the  seman1c  gap.  
  18. 18. Events:  Types  and  Granulari1es   •  Conferences   –  Days   •  Sessions   –  Talks   »  Purpose  of  the  talk   •  Wedding   •  An  Earthquake   •  The  Big  Bang   •  World  Wide  Web   •  Yahoo:  Winter  School  2012   •  Me   –  My  Birth,     –  Being  here,  and     –  Dying  in  100  years.  
  19. 19. People  Things  Places  Time  Experiences  Events   E    by  Westerman  and    Jain     E*  by  Gupta  and  Jain  
  20. 20. Sense  making  from  mul1modal  massive  geo-­‐social  data-­‐streams.     20  
  21. 21.  •  Introduc1on  • Social  Systems  •  Intelligent  Social  Systems  •  Designing  Intelligent  Systems  •  Situa1on  Recogni1on  •  Concept  recogni1ons  •  Personalized  Situa1ons  •  EventShop  
  22. 22. Poli0cs   Religion   Educa0on  Health   Economics  
  23. 23. Connec4ng  People  to  Resources    effec4vely,  efficiently,  and  promptly     in  given  situa4ons.  
  24. 24. •  Minimize  hunger  in  the  world.  •  Maximize  female  educa1on  in  India.  •  Minimize  ‘deaths’  in  the  coming  hurricane  in   Florida.  •  Minimize  work-­‐hours  lost  in  traffic  during   week  days  in  Bangalore.  
  25. 25. •  System:   –     A  set  of  diverse  parts  forming  a  whole.   –  Parts  are  put  together  with  a  common  objec1ve/ purpose.  •  Each  part  could  be  considered  a  system.  •  Each  part  plays  a  role  towards  the  system   objec1ve.  •  Designing  the  informa1on  flow  among  parts  is   essen1al  to  make  a  system  work  apprpriately.    
  26. 26. •  A  social  system  is  composed  of  persons  or   groups  who  share  a  common  objec1ve.  •  An  individual  objec1ve  is  usually  a  part  of  the   group’s  objec1ve.    
  27. 27. •  Persons  •  Families  •  Organiza1ons  •  Communi1es:  City,  State,  Country  •  Socie1es  •  Cultures  
  28. 28. •  Top  Down:   –  The  social  system  determines  its  parts.   –  People’s  behavior  determined  by  society.  •  BoBom  Up:   –  The  Society  is  the  sum  of  its  indivduals   –  Individual  ac1ons  determine  the  character  of  the   society.  
  29. 29. •  Each  social  en1ty  is  a  holon.  •  Holon:  Each  en1ty  is  simultaneously  a  part  and  a   whole.  •  A  social  component  is  made  up  of  parts  and  at  the   same  1me  maybe  part  of  some  larger  whole.  •  Any  system  is  by  defini1on  both  part  and  whole.  
  30. 30. •  The  primary  ‘currency’    of  a  social  system  is   informa1on.  •  System  behavior  can  be  understood  as  the   movement  of  informa1on:   –  Within  a  system   –  Between  the  system  and  its  environment  •  Informa1on  is  used  to  sense  as  well  as  to  control  or   act.  
  31. 31.  •  Introduc1on  •  Social  Systems  •  Intelligent  Social  Systems  •  Designing  Intelligent  Systems  •  Situa1on  Recogni1on  •  Concept  recogni1ons  •  Personalized  Situa1ons  •  EventShop  
  32. 32. •  Systems  that  perceive,  reason,  learn,  and  act   intelligently.  •  Adaptability  to  varying  environmental   situa1ons  is  a  key  element  of  intelligent   systems  
  33. 33. •  Social  systems  that  perceive,  reason,  learn,   and  act  intelligently.  •  What  does  ‘perceive’,  ‘reason’,  ‘learn’,  and   ‘act’  mean  in  the  context  of  social  systems?  
  34. 34.  •  Introduc1on  •  Social  Systems  •  Intelligent  Social  Systems  • Designing  Intelligent  Social   Systems  •  Situa1on  Recogni1on  •  Concept  recogni1ons  •  Personalized  Situa1ons  •  EventShop  
  35. 35. •  Desired  state  (Goal)  •  System  model  and  Control  Signal   (Ac0ons)  •  Current  State  (for  Feedback)  
  36. 36.   bserve d  State   O Fe edbackObserva0ons   Control   Signals   Events   Real  World  
  37. 37. Social  Networks    Connecting People
  38. 38. Needs  and  Resources   Not  even   FaceBook!  
  39. 39. Current    Social   Networks   Important   Unsa1sfied     Needs  12/5/12   46  
  40. 40. •  Resources     –  Physical:  food,  water,  goods,  …   –  Informa:onal:  Wikipedia,  Doctors,  …   –  Transporta:on   –  Employment   –  Spiritual  •  Timeliness  •  Efficiency  
  41. 41. Connecting Information     People Aggregation Situation Alerts and Detection CompositionAnd Queries Resources12/5/12   48  
  42. 42.  •  Introduc1on  •  Social  Systems  •  Intelligent  Social  Systems  •  Designing  Intelligent  Systems  • Situa0on  Recogni0on  •  Concept  recogni1ons  •  Personalized  Situa1ons  •  EventShop  
  43. 43. Connec4ng  People  to  Resources    effec4vely,  efficiently,  and  promptly     in  given  situa4ons.  
  44. 44. •  rela1ve  posi1on  or  combina1on  of   circumstances  at  a  certain  moment.  •  The  combina1on  of  circumstances  at  a  given   moment;  a  state  of  affairs.  
  45. 45. •  Situa1on  awareness,  or  SA,  is  the  percep1on   of  environmental  elements  within  a  volume  of   1me  and  space,  the  comprehension  of  their   meaning,  and  the  projec1on  of  their  status  in   the  near  future.  •  What  is  happening  around  you  to  understand   how  informa1on,  events,  and  your  own   ac1ons  will  impact  your  goals  and  objec1ves,   both  now  and  in  the  near  future.  
  46. 46. •  Example  1:     –  A  person  shou1ng.   –  1000  people  shou1ng.   •  In  a  contained  building   •  In  main  parts  of  a  city  •  Example  2:   –  One  person  complaining  about  flu.   –  Many  people  from  different  areas  of  a  country   complaining  about  flu.  
  47. 47. Facebook  and  TwiBer   (now  GOOGLE  +)  Have  been  repor0ng  events  as  micro-­‐blogs   Massive  collec1on  of  events.  
  48. 48. Time  
  49. 49. Does  the  flap  of  a  buEerfly’s  wings  in  Brazil  set  off  a   tornado  in  Texas?        
  50. 50. Have  been  repor0ng  events  as  micro-­‐blogs  Sensors  and  Internet  of  Things  are  crea1ng  and   repor1ng    even  more  events  than  humans   are.   12/5/12   57  
  51. 51. FROM TWEETS TO REVOLUTIONS
  52. 52. Atomic  and  Composite  Events   Time  
  53. 53. •  Given  a  plethora  of  event  data.  How  can  we:   –  Disambiguate  relevant  and  irrelevant  events?   –  Combine  events  into  meaningful  representa1ons  ?   –  Allow  inference  and  cascading  effects?   –  Support  different  interpreta1ons  based  on   applica1on  domain?   –  Support  Control  &  decision  making?    
  54. 54. 1.  Inherent  support  for  event-­‐based  (temporal)   reasoning  2.  The  ability  of  the  controller  to  reason  based   on  symbols  (rather  than  just  signals)  3.  Explicit  inclusion  of  domain  seman1cs  (to   support  mul1ple  applica1ons)  
  55. 55. An  ac4onable  abstrac4on  of  observed  spa4o-­‐temporal  characteris0cs.   62  
  56. 56.  •  Introduc1on  •  Social  Systems  •  Intelligent  Social  Systems  •  Designing  Intelligent  Systems  •  Situa1on  Recogni1on  • Concept  recogni0ons  •  Personalized  Situa1ons  •  EventShop  
  57. 57. Data  Type   1950   2000   Time  Line  
  58. 58. Situa0on     2010   Events    Data  Type   1986   Objects     1963   Speech   1962   Character   1959   1950   2000   Time  Line  
  59. 59. Loca1on   Scenes   Environm Trajectories   Situa1ons   aware   ents   Single  Media   Loca1on   Visual   Real  world   Visual   Objects   Objects   Ac1vi1es   Events   unaware   Sta1c   Dynamic   SPACE   TIME  Data  =  Text  or  Images  or  Video   66  
  60. 60. Trajectory   Situa1on   Object   Scene   Event  1960   1970   1980   1990   2000   2010  •  1963:  Object  Recogni1on  [Lawrence  +  Roberts]  •  1967:  Scene  Analysis  [Guzman]  •  1984:  Trajectory  detec1on  [Ed  Chang+  Kurz]  •  1986:  Event  Recogni1on  [Haynes  +  Jain]  •  1988:  Situa1on  Recogni1on  [Dickmanns]  
  61. 61. Heterogeneous  Media   Loca1on   Environm Situa1ons   aware   ents   Loca1on   Real  world   Objects   Ac1vi1es   unaware   Sta1c   Dynamic   SPACE   TIME  Data  is  just  Data.  Meta-­‐data  is  also  data.    Caste  system  does  not  exist  here.  Medium  and  sources  do  not  maOer.   68  
  62. 62.  •  Introduc1on  •  Social  Systems  •  Real  Time  Social  Systems  •  Designing  Real  Time  Systems  •  Situa1on  Recogni1on  •  Concept  recogni1ons  • Personalized  Situa0ons  •  EventShop  
  63. 63. A)  Situa0on  Modeling   B)  Situa0on  Recogni0on   C)  Visualiza0on,  Personaliza0on, and  Alerts   i)  Visualiza1on   C1   …   ⊕ v2   v3   Personal   context   ii)  Personaliza1on   Personali v5   v6   zed   STT  Stream   situa1on   Available   resources   Emage   iii)  Alerts   Situa1on   70        
  64. 64. Proprietary  and  Confiden1al,  Not  For  12/5/12   71   Distribu1on  
  65. 65. Aggrega1on,     Opera1ons   Alert  level     =  High   Date:  3rd  Jun,  2011   STT  data   Situa1on  Detec1on       User-­‐Feedback   1)      Classifica1on     Tweet:     ‘Please  visit  Dr.  Cureit  at   ‘Urrgh…  sinus’     2)      Control  ac1on   4th  St  immediately’       Loc:  NYC,  Date:  3rd  Jun,  2011   Theme:  Allergy   73  
  66. 66.  •  Introduc1on  •  Social  Systems  •  Intelligent  Social  Systems  •  Designing  Intelligent  Systems  •  Situa1on  Recogni1on  •  Concept  recogni1ons  •  Personalized  Situa1ons  • EventShop  
  67. 67. •  E-­‐mage             –  Visualiza1on   –  Intui1ve  query  and  mental  model   –  Common  spa1o  temporal  data  representa1on   –  Data  analysis  using  media  processing  operators     (e.g.  segmenta1on,  background  subtrac1on,   convolu1on)   76        
  68. 68. •  Spa1o-­‐temporal  element   –  STTPoint  =  {s-­‐t-­‐coord,  theme,  value,  pointer}  •  E-­‐mage   –  g  =  (x,  {(tm,  v(x))}|xϵ X  =  R2  ,  tm ϵ  θ,  and  v(x)  ϵ  V  =  N)  •  Temporal  E-­‐mage  Stream   –  TES=((ti,  gi),  ...,  (tk,  gk))  •  Temporal  Pixel  Stream   –  TPS  =  ((ti,  pi),  ...,  (tk,  pk))   77  
  69. 69. Proprietary  and  Confiden1al,  Not  For  12/5/12   78   Distribu1on  
  70. 70. Proprietary  and  Confiden1al,  Not  For  12/5/12   79   Distribu1on  
  71. 71. Retail  Store   Loca0ons   Net  Catchment   area   Proprietary  and  Confiden1al,  Not  For  12/5/12   80   Distribu1on  
  72. 72. •  Humans  as  sensors   •  Space  +  Time  as  fundamental  axes     •  Real  0me  situa0on  evalua0on  (E-­‐mage  Streams)  (a) Pollen levels (Source: Visual) (b) Census data (Source: text file) (c) Reports on ‘Hurricanes’ (source: Twitter stream)  d) Cloud cover (Source: Satellite imagery) (e) Predicted hurricane path (source: KML) (f) Open shelters coverage(Source: KML)   81  
  73. 73. •  Help  domain  experts  externalize  their  internal   models  of  situa1ons  of  interest  e.g.  epidemic.   •  Building  blocks:     –  Operators     –  Operands     •  Wizard:     –  A  prescrip1ve  approach  for  modeling  situa1ons  using  the   operators  and  operands    Singh,  Gao,  Jain:  Situa:on  recogni:on:  An  evolving  problem  for  heterogeneous  dynamic  big   82   mul:media  data,  ACM  Mul0media  ‘12.  
  74. 74.  Knowledge  or  data  driven  building  blocks   Growth  rate     (Flu  reports)   Feature   TwiBer-­‐Flu   Data  source   -­‐Emage  (#Reports)   Representa1on  level   Thresholds   (0,  50)   Meta-­‐data   83  
  75. 75. Get_components (v){ v ϵ  {  Low,   Mid,  High}   f1   ⊕  1)  Identify output state space <USA,    5  mins,   0.01x  0.01>    2)  Identify S-T bounds v2   v3  3)  Define component f2   v4   @ ⊕ ∏ features: v=f(v1, …, vk) Emage   v5   v6   Emage   •  If (type = imprecise) ∏ @ Δ Δ –  identify learning data source, method    4)  ForEach (feature vi) { D1   Emage   Emage   D2        If (atomic) Δ Δ •  Identify Data source. D2   D3   •  Type, URL, ST bounds •  Identify highest Rep. level reqd. •  Identify operations Else Get_components(vi) 84  
  76. 76. 3)  Instan1ate  2)  Revise  1) Model     Epidemic   ϵ  {Low,  mid,  high},   Outbreaks     <USA,  5  mins,  0.01x   Classifica1on:   0.01>   Thresh  (30,70)   Growing  Unusual   ac1vity   Mul1ply   Unusual   Ac1vity?   Growth  Rate   Subtract   Historical   Current  ac1vity   Emage     ac1vity  level   level   (#reports  ILI)   Subtract   Normalize   Emage  (Historical   [0,100]   avg)   Emage  Emage     TwiBer-­‐Flu     Emage   (#reports  ILI)   (#reports  ILI)   (popula1on)   TwiBer.com   <USA,  5  mins,     TwiBer-­‐Avg   0.01x  0.01>   DB,     TwiBer.com   TwiBer-­‐Flu   CSV-­‐   Census.gov,     TwiBer.com   TwiBer-­‐Flu   <USA,  5  mins,     <USA,  5  mins,     Popula1on   <USA,  5  mins,     <USA,  5  mins,     0.01x  0.01>   0.01x  0.01>   0.01x  0.01>   85         0.01x  0.01>  
  77. 77. Level  0:  Raw  data  streams     e.g.  tweets,  cameras,  traffic,  weather,  …   …   Level  1:  Unified   representa1on   Proper1es (STT  Data)   STT  Stream   Level  2:   Aggrega1on   Proper1es Emage   (Emage)    Opera1ons   Level  3:   Symbolic  rep.   Proper1es Situa1on   (Situa1ons)   86  
  78. 78. Suppor1ng   Operator  Type   Data   parameter(s)   Output   Filter   + Mask   Aggregate   + Classifica1on   Classifica1on   method  Characteriza1on   Property     Growth  Rate   required   =  125%  PaBern  Matching   + 72%   PaBern   87        
  79. 79. Suppor1ng   Operator  Type   Data   parameter(s)   Output   1)  Data  into  right   representa1on   Transform   …   Spa1o-­‐temporal   window   Filter   + Mask   Aggregate   + 2)  Analyze  data  to   Classifica1on   derive  features     Classifica1on   method   Characteriza1on   Property     Growth  Rate   required   =  125%   PaBern  Matching   + PaBern 72%     {Features}  3)  Use  features  to   Learn   f   Learning     f  evaluate  situa1ons   {Situa1on}   method   88        
  80. 80. S.  No Operator Input Output1 Filter  ∏ Temporal  E-­‐mage  Stream Temporal    E-­‐mage  Stream2 Aggrega0on  ⊕ K*Temporal  E-­‐mage  Stream Temporal  E-­‐mage  Stream3 Classifica0on  γ Temporal  E-­‐mage  Stream Temporal  E-­‐mage  Stream4 Characteriza0on  :  @           •  Spa0al     •  Temporal  E-­‐mage  Stream   •  Temporal  Pixel  Stream   •  Temporal   •  Temporal  Pixel  Stream •  Temporal  Pixel  Stream5 PaOern  Matching  ψ           •  Spa0al     •  Temporal  E-­‐mage  Stream   •  Temporal  Pixel  Stream   •  Temporal   •  Temporal  Pixel  Stream •  Temporal  Pixel  Stream Singh, Gao, Jain: Social Pixels: Genesis and Evaluation, ACM Multimedia 89   ‘10.  
  81. 81. •  Select  E-­‐mages  of  US  for  theme  ‘Obama’.   –  ∏spa1al(region=[24,-­‐125],[24,-­‐65])   (TEStheme=Obama)  •  Iden1fy  3  clusters  for  each  E-­‐mage  above.   –  γkmeans(3)  (∏spa1al(region=[24,-­‐125],[24,-­‐65])(TEStheme=Obama))  •  Show  me  the  speed  for  each  cluster  of  ‘Katrina’  e-­‐ mages   ( –  @speed @epicenter (γkmeans(n=3)  (∏spa1al(region=[24,-­‐125],[24,-­‐65])  (TEStheme=Katrina))))  •  How  similar  is  paBern  above  to  ‘exponen1al  increase’?   –  ψexp-­‐increase(@speed(@epicenter(γkmeans(n=3)  (∏spa1al(region=[24,-­‐125],[24,-­‐65])   (TEStheme=Katrina))))   90  
  82. 82. Personalized  situa0on:  An  ac4onable  integra4on  of  a  users  personal  context  with  surrounding  spa4otemporal  situa4on.   1)  Macro   situa0on Macro     Personal   2)  Personalized   data-­‐sources   Context   situa0on Profile  +   Preferences     3)  Personalized   alerts   User     Available   data   resources   Resource   data   IF  person  ui  <is-­‐in>  (PSj)  THEN  <connect-­‐to>  rk       91  
  83. 83. Personalized  Situa1on  Recogni1on:   Operators   Suppor1ng   Operator  Type   Data   parameter(s)   Output   Filter   …   + User  loca1on   Aggregate   …   + …   …   Classifica1on   Classifica1on   …   method …     Characteriza1on   …   Property     Growth  Rate   required   =  125%   PaBern  Matching   …   + Match=  42%   PaBern   92        
  84. 84. •  IF   𝑢𝑖   𝑖𝑠𝑖𝑛   𝑧𝑗     𝑇𝐻𝐸𝑁   𝑐𝑜𝑛𝑛𝑒𝑐𝑡  ( 𝑢𝑖,   𝑛𝑒𝑎𝑟𝑒𝑠𝑡(𝑢𝑖,   𝑟𝑘))     1)  𝑖 𝑠𝑖𝑛(𝑢𝑖,   𝑧𝑗)  → 𝑚𝑎𝑡𝑐ℎ( 𝑢𝑖,   𝑟𝑘))                         𝑓:( 𝑈× 𝑍)→( 𝑈× 𝑅)   •  U  =  Users     •  Z  =  Personalized  Situa1ons   •  R  =  Resources  2)  𝑛 𝑒𝑎𝑟𝑒𝑠𝑡(𝑢𝑖,   𝑟𝑘)= 𝑎𝑟𝑔𝑚𝑖𝑛  ( 𝑢𝑖. 𝑙𝑜𝑐,     𝑟𝑘. 𝑙𝑜𝑐𝑠)   93  
  85. 85. Billions  of  data  sources.    Selec0ng  and  combining  appropriate  sources  to  detect  situa0ons.    Interac0ons  with  different  types  of  Users    Decision  Makers                        Individuals     12/5/12   94  
  86. 86. 12/5/12   95  
  87. 87. Front  End  GUI New New E-­‐mage Alert Data Query Stream Request Source Back  End  Controller E-­‐mage  Stream Personalized   Registered Stream  Query  Processor Queries Alert  Unit E-­‐mage  Stream User  Info Registered Data Data  Ingestor Raw  Data Storage Sources API  Calls Raw  Spatial   Data  Stream Data  Cloud12/5/12   96  
  88. 88. 11/28/2012   97  
  89. 89. Emage -­‐theme  :  String STTPoint -­‐start  :  Date -­‐theme  :  String -­‐end  :  Date -­‐value  :  double -­‐latUnit  :  double Iterator -­‐start  :  Date Iterator -­‐longUnit  :  double Iterator -­‐end  :  Date -­‐swLat  :  double -­‐latUnit  :  double -­‐swLong  :  double +hasNext()  :  bool -­‐longUnit  :  double +hasNext()  :  bool -­‐neLat  :  double +hasNext()  :  bool +next()  :  <unspecified> -­‐latitude  :  double +next()  :  <unspecified> -­‐neLong  :  double +next()  :  <unspecified> -­‐longitude  :  double -­‐image FrameParameters -­‐timeWindow  :  long 1..* 1..* -­‐syncTime  :  long -­‐latUnit  :  double -­‐Parameterize STTPointIterator EmageIterator STMerger -­‐longUnit  :  double -­‐swLat  :  double +hasNext()  :  bool 1 1 +hasNext()  :  bool 1..* 1 +hasNext()  :  bool -­‐swLong  :  double 1 1 +next()  :  STTPoint +next()  :  Emage +next()  :  Emage -­‐neLat  :  double -­‐neLong  :  double 1 1 Wrapper DBSTTPointIterator VisualImageIterator ResolutionMapper -­‐initResolution +setURL() +setURL() -­‐finalResolution +setTheme() +setTheme() +setParas() +setParas() TwitterWrapper FlickrWrapper CSVWrapper KMLWrapper+setBagOfWords() +setBagOfWords()11/28/2012   +setColors() 98  
  90. 90. Macro  situa0on   Alert  Level=High   Date=12/09/10   Micro  event   Situa0onal  controller   Control  Ac0on   e.g.  “Arrgggh,  I     “Please  visit  have  a  sore  throat”   • Goal     nearest  CDC   (Loc=New  York,   • Macro  Situa1on     center  at  4th  St   Date=12/09/10)   • Rules   immediately”   Level  1  personal  threat  +  Level  3  Macro  threat  -­‐>  Immediate  ac0on     12/5/12   99  
  91. 91. •  What  personal  informa1on  can  be  shared?  •  How  should  it  be  shared  to  benefit  the  user?  •  Developing  an  architecture  for  personal   informa1on  management.  
  92. 92. Macro  situa1on  model   Asthma  Threat   c  ϵ  {Low,  mid,  high},   level   [USA,    6  hrs,   0.1x  0.1]       Air  Quality   Pollen  Count   Allergy  reports   Emage     Emage  (Pollen   Emage  (Number  of   (AQI.)   Level)   reports)   Visual-­‐   Visual-­‐   TwiOer-­‐Allergy   Air  quality   Pollen  level  Weather.com,   TwiOer  API,  [USA,    6  hrs,   Pollen.com,   [USA,    6  hrs,   [USA,    6  hrs,   0.1x  0.1]   0.1x  0.1]   0.1x  0.1]   102  
  93. 93. Personal   c  ϵ  {Low,  mid,   threat  level   high}   Classifica0on:   Thresh(30,70)   And   Physical  exer0on   Asthma  threat   level  Normalize   Normalize  (0,   (0,  100)   100)   TPS  (Asthma)   TPS  (Funf)   UserLoc   Funf-­‐ac0vity   EventShop   [USA,    6  hrs,   Phone  sensors,     0.1x  0.1]   (relaxMinder  app),   [USA,    6  hrs,   0.1x  0.1]   103    /  
  94. 94. Personal   threat  level   c  ϵ  {Low,  mid,   high}   Classifica0on:   Thresh(30,70)   And   Physical   Asthma   exer0on   threat  level  Normalize   Normalize   (0,  100)   (0,  100)   TPS   TPS  (Funf)   (Asthma)   UserLoc   Funf-­‐ac0vity   EventShop     [USA,    6  hrs,   Phone  sensors,     0.1x  0.1]   (relaxMinder  app),   [USA,    6  hrs,   0.1x  0.1]   104        
  95. 95. Classify (Flood level - Shelter Flood Shelter) Twitter Flood Level Shelter12/5/12   107  
  96. 96. 12/5/12   108  
  97. 97. Proprietary  and  Confiden1al,  Not  For  12/5/12   109   Distribu1on  
  98. 98. Outline  •  Introduc1on  •  Social  Systems  •  Real  Time  Social  Systems  •  Designing  Real  Time  Systems  •  Situa1on  Recogni1on  •  Concept  recogni1ons  •  Personalized  Situa1ons  •  EventShop  • Going  Forward  
  99. 99. •  Social  observa1ons  are  now  possible  with  liBle   latency.  •  Now  possible  to  design  social  systems  with   feedback.  •  Situa1on  Recogni1on  and  Need-­‐Availability   iden1fica1on  of  resources  becomes  a  major   challenge.  •   EventShop  is  a  step  in  the  direc1on  of   implemen1ng  Social  Life  Networks.  
  100. 100. Useful  Links  •  Demo:   –  hBp://auge.ics.uci.edu/eventshop  •  Data  Defini1on  Language  Schema   –  hBp://auge.ics.uci.edu/eventshop/documents/ EventShop_DDL_Schema  •  Query  Language  Schema   –  hBp://auge.ics.uci.edu/eventshop/documents/ EventShop_QL_Schema  •  Example  Query  in  JSON   –  hBp://auge.ics.uci.edu/eventshop/documents/ EventShop_Example_Query  11/28/2012   112  
  101. 101. Thanks  for  your  1me  and  aBen1on.   For  ques1ons:  jain@ics.uci.edu  
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×