Multimedia big data 140619

1,825 views
1,605 views

Published on

Multimedia and Big Data are closely related topic. Big data enables solving some important challenges in multimedia and basic principles of multimedia are the key issues in multimedia.

Multimedia big data 140619

  1. 1. 1  BAMMF  June  20,  2014  
  2. 2. •  Marshal  McLuhan  in  1964.     – A  medium  itself,  not  the  content  it   carries,  should  be  the  focus  of  study.     – A  medium  affects  the  society  in  which  it   plays  a  role  not  only  by  the  content   delivered  over  the  medium,  but  also  by   the  characteris@cs  of  the  medium  itself.   BAMMF  June  20,  2014   2   Media  
  3. 3. Mul+   BAMMF  June  20,  2014   3   Media   +   +   Digital  technology  allowed  combining  and   using  media  leading  to  a  major  disrupAon.  
  4. 4. Photos  are  the  killer  content  type  on  mobile.  Quick   to  consume  like  text,  but  easier  to  produce  on  a   phone.     BAMMF  June  20,  2014   4   From  Mary  Meeker’s  presenta@on  
  5. 5. Big  Data   BAMMF  June  20,  2014   5  
  6. 6. To  know  our  world,    (MulAmedia)   Data  is  the  Source.  
  7. 7. McLuhan  was  right.    UnAl  20th  Century  -­‐-­‐   Medium  was  the  message.   BAMMF  June  20,  2014   7  
  8. 8. BAMMF  June  20,  2014   8   What  is  an  Elephant?  
  9. 9. BAMMF  June  20,  2014   9   Six  Blind  Men  Exploring  an  Elephant.  
  10. 10. BAMMF  June  20,  2014   10   An  Elephant  according  to  Six  Limited   PerspecAves.   The  Blind  Men’s  Elephant  
  11. 11. BAMMF  June  20,  2014   11   But,  an  Elephant  is  an  Elephant.   A  Real  Animal.  
  12. 12. MulAmedia  and  Big  Data  are  both   the  Latest  Elephant.     BAMMF  June  20,  2014   12   Lots  of  people  excited  about  these   but  with  narrow  perspecAves.  
  13. 13. BAMMF  June  20,  2014   13  
  14. 14. Two  Major  Human  AcAviAes     •  CommunicaAons   •  ComputaAons   BAMMF  June  20,  2014   14   Both  require  MulAmedia   AND  Big  Data.  
  15. 15. CommunicaAon   BAMMF  June  20,  2014   15  
  16. 16. Nature  Gave  Us  Five  Basic  Senses   BAMMF  June  20,  2014   16   to  Experience  the  World  
  17. 17. We  promptly  created  Silos   BAMMF  June  20,  2014   17  
  18. 18. Storytelling:  The  Oldest  and  most   powerful  Art.   •  A  story  is  a  chain  of  events  that  begins  at  one   place  and  ends  at  another.   •  A  story  is  a  selecAve  batch  of  informaAon.  It   selects  details,  arranges  them,  perhaps   embellishes  them.   BAMMF  June  20,  2014   18   h6p://janefriedman.com/2011/09/27/what-­‐is-­‐a-­‐story/  
  19. 19. Where  do  Events  happen?   •  In  Real  World.   •  Images,  video,  audio,  all  other  sensors  are  just  a   means  to  detect  and  capture  some  aspect  of  the   event.   BAMMF  June  20,  2014   19  
  20. 20.  Events  are  ‘Connectors’   Events  create  ‘Context’   Event  contains  Information  and   Experiences   Spatial  Causal     Experiential   Informational   Temporal    Structural  
  21. 21. So  What  is  Digital  Storytelling?   •  Prepare  a  database  of  all  events.   –  Collect  all  experienAal  and  informaAonal  data  about   each  event.   •  Considering  audience  and  the  message,  select   important  events.   •  Render  Events   –  For  each  event  decide  what  informaAon  and   experienAal  data  should  be  used.   –  Render  all  events  and  related  data  in  the  best  format   for  highest  quality  of  experience.   BAMMF  June  20,  2014   21  
  22. 22. From  Events  to  Stories   CollecAng  Events   SelecAng  Events   Rendering  Events  
  23. 23. Oral  Storytelling   BAMMF  June  20,  2014   23  
  24. 24. Novels   BAMMF  June  20,  2014   24  
  25. 25. Video   BAMMF  June  20,  2014   25  
  26. 26. Do  you  edit  your  videos?   •  Why  is  video  ediAng  so  difficult?   – Even  a_er  so  many  products?   •  How  can  we  make  it  usable  by  consumers?   •  Is  Video  ediAng  a  real  problem?   •  Is  it  possible  to  reinvent  video  ediAng  given   all  the  ‘big  data’?   BAMMF  June  20,  2014   26  
  27. 27. Virage Video Indexing Lyn Russell 00:08:26:12 00:08:34:29 00:08:40:00 00:08:26:12 00:08:34:29 00:08:40:00 “Clinton spoke with reporters” “When She arrived” “Older citizens who have” Clinton running for senate Speaking at rally Improving Medicare Lyn Russell Hillary ClintonFace ID Speaker ID User Annotation Speech to Text Time Code Keyframes Video Time Hillary Clinton
  28. 28. Different  Data  Tracks   Ac+vity   Loca+on   Manual   Visual   Audio   OTHER   Walk   Walk   Walk   Walk   Walk   Walk   Exercise  Drive  Home   Drive   Mee+ng   Work   Exercise   Home   Drive   Mee+ng   Work  Home   Mee+ng   Work   Exercise   Event  Track  
  29. 29. Personalized  Storytelling:    Events  in  a   Football  Game.   BAMMF  June  20,  2014   29  
  30. 30. All  communicaAon  is  story  telling.   Even  scienAfic  papers.   BAMMF  June  20,  2014   30  
  31. 31. Fundamental  Opportunity  in  MulAmedia   CommunicaAon  and  CompuAng:   UAlize  complementary  and  correlated   informaAon  from  mulAple  sources  to   form  holisAc  picture  of  the  world.   BAMMF  June  20,  2014   31  
  32. 32. ComputaAon:  at  least  50,000  years   •  CounAng  was  primarily  used  to  keep  track  of   social  and  economic  data  such  as  number  of   group  members,  prey  animals,  property,  or   debts.   •  CompuAng  was  invented  for  Resource   management.   BAMMF  June  20,  2014   32  
  33. 33. What  is  Important  in  ‘Big  Data’?   MulAmedia   RealAme   Uncertainty   BAMMF  June  20,  2014   33  
  34. 34. Big  data  is  the  Rorschach  blot  of  our   Ames  –  an  incomprehensible  shape  on   to  which  we  project  our  dreams  and   nightmares,  hopes  and  fears.   BAMMF  June  20,  2014   34   Rorschach  blot    
  35. 35. BAMMF  June  20,  2014   35   You  can  only  control  what  you   measure.  
  36. 36. The most Fundamental Problem in Society: Connecting People to Resources Effectively, Efficiently, and Promptly in given Situations. 36  BAMMF  June  20,  2014  
  37. 37. Current    Social  Networks   Important   UnsaAsfied     Needs   37   Time  to  Focus   BAMMF  June  20,  2014  
  38. 38. What  differenAates:   •  Advanced  SocieAes  from  Primi6ve  SocieAes   •  Advanced  countries  from  a  Developing   countries   38  BAMMF  June  20,  2014   How  People’s  needs  are   connected  to  Resources.      
  39. 39. Hunter  Gatherer   Food  delivery   where  you   are.   39  BAMMF  June  20,  2014  
  40. 40. UnAl  2000,  you   went  to  a  phone.   Now  the  call   comes  to  you.   40  BAMMF  June  20,  2014  
  41. 41. UnAl  recently,  you   were  a  folder.   Very  Soon   You  are  Your  Data.   41  BAMMF  June  20,  2014  
  42. 42. Needs   Need = f (persona, situation) BAMMF  June  20,  2014   42  
  43. 43. Where do most Data Scientists apply their Magic? We  live  in  a  targeted  world.    AdverAsers  need   efficient  ways  to  si_  through  the  volume  of  data.     BAMMF  June  20,  2014   43  
  44. 44. Using BIG Data to solve Big PROBLEMS Time  is  right.   BAMMF  June  20,  2014   44  
  45. 45. We  Live  in  Dynamic  World.   Big  Data  will  keep  gegng  bigger  in   Cyber  Space.     BAMMF  June  20,  2014   45  
  46. 46.   Animals     Machines     SocieAes   Published  first  in  1942  
  47. 47. •  Desired  state  (Goal)   •  System  model  and  Control  Signal   (AcAons)   •  Current  State  (using  mulAmedia  data)  
  48. 48. EventShop : E. Situation Detection Predic+ve   Situa+on   Recogni+on   Evolving Global Situation Predic+ve   Personal   Situa+on   Recogni+on   Personal EventShop Evolving Personal Situation Need- Resource Matcher Recommendation Engine Persona  Database   Resources Needs Data   Inges+on   Wearable Sensors Calendar Location…. DataSources …. Data   Inges+on   and   aggrega+on   Database Systems Satellite Environmental Sensor Devices Social Network Internet of Things Actionable Information BAMMF  June  20,  2014   48  
  49. 49. CriAcal  Requirements:   •  Environmental  SituaAon  RecogniAon.   –  Detects  geo-­‐spaAal  situaAons.   –  Space  is  a  key  dimension.   –  Helps  in  idenAfying  resources  and  connecAng  to  them.   •  Personal  SituaAon  RecogniAon.   –  Centered  around  an  individual.   –  Uses  environmental  situaAon.   –  Helps  in  modeling  individuals  and  idenAfying  their  needs.   BAMMF  June  20,  2014   49  
  50. 50. Data  to  Situa+ons     Situa+ons   Event  Streams   Mul+modal  Observa+ons   Low-­‐level   Analysis   Aggrega+on   and   Classifica+on   BAMMF  June  20,  2014   50  
  51. 51. A  Grand  Challenge  for  data  has   always  been:   Sense  making  from  mulAmodal   massive  geo-­‐social  data-­‐streams.     51  BAMMF  June  20,  2014  
  52. 52. A  REAL  Challenge   •  SituaAon  Modeling   •  SituaAon  RecogniAon   (Object  recogniAon,  event  recogniAon,  speech   recogniAon,  …  dominated  last  century.)   BAMMF  June  20,  2014   52  
  53. 53. Challenge:    Unifying  MulAmodal  Big   Data   •  Spa+o-­‐temporal-­‐thema+c  (STT)    real-­‐@me  streams   •  E-­‐mage  as  a  unifying  GRID  representa+on   53   (a) Pollen levels (Source: Visual) (b) Census data (Source: text file) (c) Reports on ‘Hurricanes’ (source: Twitter stream)   d) Cloud cover (Source: Satellite imagery) (e) Predicted hurricane path (source: KML) (f) Open shelters coverage(Source: KML)   BAMMF  June  20,  2014  
  54. 54. 54   Billions  of  data  sources.   Environment  for    SelecAng,  and      Combining     appropriate  sources  to  detect  situaAons.   PredicAon  for  Pro-­‐acAve  acAons   InteracAons  with  different  types  of  Users     BAMMF  June  20,  2014  
  55. 55. EventShop   BAMMF  June  20,  2014   55  
  56. 56. 6/20/14   56   Flood level - Shelter Flood Level Shelter Twitter Classify (Flood level - Shelter)
  57. 57. 6/20/14   57  
  58. 58. The Creative Destruction of Medicine CompuAng   Power,  Data   Universe   Social   Networking   Internet   Mobile   connecAvity,   Bandwidth   InformaAon   Systems   Imaging   Wireless   Sensors   Genomics   New  Medicine   CreaAve   DestrucAon   BAMMF  June  20,  2014   58  Old  Medicine  
  59. 59. Health = Genetics + Lifestyle BAMMF  June  20,  2014   59  
  60. 60. Asthma Triggers Indoor  Allergens   Outdoor  Allergens   Smoke   Air  PolluAon   Chemical  Irritants   Asthma/Allergies   BAMMF  June  20,  2014   60  
  61. 61. Everyone’s Respiratory Health is Different Disaster   Situa+on   Assimila+on       and  Control     Environmental   Resources  and   Historic  Data   Governmental  Agencies   Internet  of  Things   Social  Sources   Experts   Users   All  Users  are   not  EQUAL.   BAMMF  June  20,  2014   62  
  62. 62. AcAviAes   CommunicaAons   Media   Persona:  Turning  Disassociated  Data   into  Meaningful  InformaAon   AGGREGATED  PERSONAL     ANALYTICS   Sensors   63   Persona   BAMMF  June  20,  2014  
  63. 63. Wearable 2013: Data Data Everywhere, nor any Insight to Use.  h6p://www.medhelp.org/   BAMMF  June  20,  2014   64  
  64. 64. HEALTH   PERSONA   Logical  Sensor   Physiological  Sensors  Fitness  Tracking  Sensors   Life   Event   Kine+c   Event   Physiological   Event   Food   Event   Personicle   (Health) Persona Framework Calendar   Ac+vity-­‐level   Heart  rate   Home   Drive   Mee+ng   Work   Exercise   Walk   Walk   BAMMF  June  20,  2014   65  
  65. 65. Time   Foursquare   Personal  Calendar   Moves   Drive   Walk   Mee+ng  in  DBH  room   3233   Walk   Drive   Mee+ng  with   trainer  in  gym   Ac+vity  Level   Personicle   Personicle   Home   Drive   Walk   Mee+ng   Work   Walk   Drive  Exercise   Work   Laleh  at  University  Club   Ea+ng  dinner   BAMMF  June  20,  2014   66  
  66. 66. Insight: Exercise Triggers My Asthma •  Red: “Severe asthma attack”. •  Orange: High activity level. •  Intense Exercise triggers asthma! Personicle   Ac+vity  Stream   t1   t2   t3   t4   t5   BAMMF  June  20,  2014   67  
  67. 67. UCI  Allergy/Asthma  App   Triggers   Recommendati on   Stop  Jogging!   Symptoms   SHORTNESS  OF  BREATH   CHEST  TIGHTNESS  OR  PAIN   COUGHING   WHEEZING   How Do You Feel?   BAMMF  June  20,  2014   68  
  68. 68. PredicAve  Techniques  for  PrevenAve  Advise   69  BAMMF  June  20,  2014  
  69. 69. From Persona to Societal Health Health  Persona  of   different  people     Environmental   factors   Disease   outbreak   Disease  Modeling   Geographical   loca+on   specifica+ons   BAMMF  June  20,  2014   70  
  70. 70. Use  ObjecAve  Massive  Volume  of  Personal  Data  for  Building   Beker  Disease  Models.   Persona and Societal Health Use  Disease  Model  and  Personal  Data  for  Beker  Quality  of   Life.   71  BAMMF  June  20,  2014  
  71. 71. Live  EventShop  and  CollaboraAon     •  Live  EventShop  Demo   – h6p://auge.ics.uci.edu/eventshop/   •  Current  Collaborators     – Cyber-­‐Physical  Cloud  Compu+ng  Project   •  NICT,  NIST   – SLN4MOP  Project   •  Sri  Lanka  Farmers;  Prof.  Ginige  in  Sydney  leading   – Open  Source  EventShop.   •  HCL   – Other  partners     BAMMF  June  20,  2014   72  
  72. 72. MulAmedia  and  Big  Data   •  MulAmedia  was  always  Big  Data.   •  Many  Major  Challenges  in  Big  Data  are  the   same  as  in  MulAmedia.   •  Time  for  MulAmedia  community  to  Accept   and  Act.   BAMMF  June  20,  2014   73  
  73. 73. Thanks  for  your  Ame  and  akenAon.   For  ques+ons:  jain@ics.uci.edu  

×