Multimedia big data 140619

1. 1 BAMMF June 20, 2014

2. •  Marshal McLuhan in 1964. – A medium itself, not the content it carries, should be the focus of study. – A medium aﬀects the society in which it plays a role not only by the content delivered over the medium, but also by the characteris@cs of the medium itself. BAMMF June 20, 2014 2 Media

3. Mul+ BAMMF June 20, 2014 3 Media + + Digital technology allowed combining and using media leading to a major disrupAon.

4. Photos are the killer content type on mobile. Quick to consume like text, but easier to produce on a phone. BAMMF June 20, 2014 4 From Mary Meeker’s presenta@on

5. Big Data BAMMF June 20, 2014 5

6. To know our world, (MulAmedia) Data is the Source.

7. McLuhan was right. UnAl 20th Century -‐-‐ Medium was the message. BAMMF June 20, 2014 7

8. BAMMF June 20, 2014 8 What is an Elephant?

9. BAMMF June 20, 2014 9 Six Blind Men Exploring an Elephant.

10. BAMMF June 20, 2014 10 An Elephant according to Six Limited PerspecAves. The Blind Men’s Elephant

11. BAMMF June 20, 2014 11 But, an Elephant is an Elephant. A Real Animal.

12. MulAmedia and Big Data are both the Latest Elephant. BAMMF June 20, 2014 12 Lots of people excited about these but with narrow perspecAves.

13. BAMMF June 20, 2014 13

14. Two Major Human AcAviAes •  CommunicaAons •  ComputaAons BAMMF June 20, 2014 14 Both require MulAmedia AND Big Data.

15. CommunicaAon BAMMF June 20, 2014 15

16. Nature Gave Us Five Basic Senses BAMMF June 20, 2014 16 to Experience the World

17. We promptly created Silos BAMMF June 20, 2014 17

18. Storytelling: The Oldest and most powerful Art. •  A story is a chain of events that begins at one place and ends at another. •  A story is a selecAve batch of informaAon. It selects details, arranges them, perhaps embellishes them. BAMMF June 20, 2014 18 h6p://janefriedman.com/2011/09/27/what-‐is-‐a-‐story/

19. Where do Events happen? •  In Real World. •  Images, video, audio, all other sensors are just a means to detect and capture some aspect of the event. BAMMF June 20, 2014 19

20. Events are ‘Connectors’ Events create ‘Context’ Event contains Information and Experiences Spatial Causal Experiential Informational Temporal Structural

21. So What is Digital Storytelling? •  Prepare a database of all events. –  Collect all experienAal and informaAonal data about each event. •  Considering audience and the message, select important events. •  Render Events –  For each event decide what informaAon and experienAal data should be used. –  Render all events and related data in the best format for highest quality of experience. BAMMF June 20, 2014 21

22. From Events to Stories CollecAng Events SelecAng Events Rendering Events

23. Oral Storytelling BAMMF June 20, 2014 23

24. Novels BAMMF June 20, 2014 24

25. Video BAMMF June 20, 2014 25

26. Do you edit your videos? •  Why is video ediAng so diﬃcult? – Even a_er so many products? •  How can we make it usable by consumers? •  Is Video ediAng a real problem? •  Is it possible to reinvent video ediAng given all the ‘big data’? BAMMF June 20, 2014 26

27. Virage Video Indexing Lyn Russell 00:08:26:12 00:08:34:29 00:08:40:00 00:08:26:12 00:08:34:29 00:08:40:00 “Clinton spoke with reporters” “When She arrived” “Older citizens who have” Clinton running for senate Speaking at rally Improving Medicare Lyn Russell Hillary ClintonFace ID Speaker ID User Annotation Speech to Text Time Code Keyframes Video Time Hillary Clinton

28. Diﬀerent Data Tracks Ac+vity Loca+on Manual Visual Audio OTHER Walk Walk Walk Walk Walk Walk Exercise Drive Home Drive Mee+ng Work Exercise Home Drive Mee+ng Work Home Mee+ng Work Exercise Event Track

29. Personalized Storytelling: Events in a Football Game. BAMMF June 20, 2014 29

30. All communicaAon is story telling. Even scienAﬁc papers. BAMMF June 20, 2014 30

31. Fundamental Opportunity in MulAmedia CommunicaAon and CompuAng: UAlize complementary and correlated informaAon from mulAple sources to form holisAc picture of the world. BAMMF June 20, 2014 31

32. ComputaAon: at least 50,000 years •  CounAng was primarily used to keep track of social and economic data such as number of group members, prey animals, property, or debts. •  CompuAng was invented for Resource management. BAMMF June 20, 2014 32

33. What is Important in ‘Big Data’? MulAmedia RealAme Uncertainty BAMMF June 20, 2014 33

34. Big data is the Rorschach blot of our Ames – an incomprehensible shape on to which we project our dreams and nightmares, hopes and fears. BAMMF June 20, 2014 34 Rorschach blot

35. BAMMF June 20, 2014 35 You can only control what you measure.

36. The most Fundamental Problem in Society: Connecting People to Resources Effectively, Efficiently, and Promptly in given Situations. 36 BAMMF June 20, 2014

37. Current Social Networks Important UnsaAsﬁed Needs 37 Time to Focus BAMMF June 20, 2014

38. What diﬀerenAates: •  Advanced SocieAes from Primi6ve SocieAes •  Advanced countries from a Developing countries 38 BAMMF June 20, 2014 How People’s needs are connected to Resources.

39. Hunter Gatherer Food delivery where you are. 39 BAMMF June 20, 2014

40. UnAl 2000, you went to a phone. Now the call comes to you. 40 BAMMF June 20, 2014

41. UnAl recently, you were a folder. Very Soon You are Your Data. 41 BAMMF June 20, 2014

42. Needs Need = f (persona, situation) BAMMF June 20, 2014 42

43. Where do most Data Scientists apply their Magic? We live in a targeted world. AdverAsers need eﬃcient ways to si_ through the volume of data. BAMMF June 20, 2014 43

44. Using BIG Data to solve Big PROBLEMS Time is right. BAMMF June 20, 2014 44

45. We Live in Dynamic World. Big Data will keep gegng bigger in Cyber Space. BAMMF June 20, 2014 45

46. Animals Machines SocieAes Published ﬁrst in 1942

47. •  Desired state (Goal) •  System model and Control Signal (AcAons) •  Current State (using mulAmedia data)

48. EventShop : E. Situation Detection Predic+ve Situa+on Recogni+on Evolving Global Situation Predic+ve Personal Situa+on Recogni+on Personal EventShop Evolving Personal Situation Need- Resource Matcher Recommendation Engine Persona Database Resources Needs Data Inges+on Wearable Sensors Calendar Location…. DataSources …. Data Inges+on and aggrega+on Database Systems Satellite Environmental Sensor Devices Social Network Internet of Things Actionable Information BAMMF June 20, 2014 48

49. CriAcal Requirements: •  Environmental SituaAon RecogniAon. –  Detects geo-‐spaAal situaAons. –  Space is a key dimension. –  Helps in idenAfying resources and connecAng to them. •  Personal SituaAon RecogniAon. –  Centered around an individual. –  Uses environmental situaAon. –  Helps in modeling individuals and idenAfying their needs. BAMMF June 20, 2014 49

50. Data to Situa+ons Situa+ons Event Streams Mul+modal Observa+ons Low-‐level Analysis Aggrega+on and Classiﬁca+on BAMMF June 20, 2014 50

51. A Grand Challenge for data has always been: Sense making from mulAmodal massive geo-‐social data-‐streams. 51 BAMMF June 20, 2014

52. A REAL Challenge •  SituaAon Modeling •  SituaAon RecogniAon (Object recogniAon, event recogniAon, speech recogniAon, … dominated last century.) BAMMF June 20, 2014 52

53. Challenge: Unifying MulAmodal Big Data •  Spa+o-‐temporal-‐thema+c (STT) real-‐@me streams •  E-‐mage as a unifying GRID representa+on 53 (a) Pollen levels (Source: Visual) (b) Census data (Source: text file) (c) Reports on ‘Hurricanes’ (source: Twitter stream) d) Cloud cover (Source: Satellite imagery) (e) Predicted hurricane path (source: KML) (f) Open shelters coverage(Source: KML) BAMMF June 20, 2014

54. 54 Billions of data sources. Environment for SelecAng, and Combining appropriate sources to detect situaAons. PredicAon for Pro-‐acAve acAons InteracAons with diﬀerent types of Users BAMMF June 20, 2014

55. EventShop BAMMF June 20, 2014 55

56. 6/20/14 56 Flood level - Shelter Flood Level Shelter Twitter Classify (Flood level - Shelter)

57. 6/20/14 57

58. The Creative Destruction of Medicine CompuAng Power, Data Universe Social Networking Internet Mobile connecAvity, Bandwidth InformaAon Systems Imaging Wireless Sensors Genomics New Medicine CreaAve DestrucAon BAMMF June 20, 2014 58 Old Medicine

59. Health = Genetics + Lifestyle BAMMF June 20, 2014 59

60. Asthma Triggers Indoor Allergens Outdoor Allergens Smoke Air PolluAon Chemical Irritants Asthma/Allergies BAMMF June 20, 2014 60

62. Everyone’s Respiratory Health is Different Disaster Situa+on Assimila+on and Control Environmental Resources and Historic Data Governmental Agencies Internet of Things Social Sources Experts Users All Users are not EQUAL. BAMMF June 20, 2014 62

63. AcAviAes CommunicaAons Media Persona: Turning Disassociated Data into Meaningful InformaAon AGGREGATED PERSONAL ANALYTICS Sensors 63 Persona BAMMF June 20, 2014

64. Wearable 2013: Data Data Everywhere, nor any Insight to Use. h6p://www.medhelp.org/ BAMMF June 20, 2014 64

65. HEALTH PERSONA Logical Sensor Physiological Sensors Fitness Tracking Sensors Life Event Kine+c Event Physiological Event Food Event Personicle (Health) Persona Framework Calendar Ac+vity-‐level Heart rate Home Drive Mee+ng Work Exercise Walk Walk BAMMF June 20, 2014 65

66. Time Foursquare Personal Calendar Moves Drive Walk Mee+ng in DBH room 3233 Walk Drive Mee+ng with trainer in gym Ac+vity Level Personicle Personicle Home Drive Walk Mee+ng Work Walk Drive Exercise Work Laleh at University Club Ea+ng dinner BAMMF June 20, 2014 66

67. Insight: Exercise Triggers My Asthma •  Red: “Severe asthma attack”. •  Orange: High activity level. •  Intense Exercise triggers asthma! Personicle Ac+vity Stream t1 t2 t3 t4 t5 BAMMF June 20, 2014 67

68. UCI Allergy/Asthma App Triggers Recommendati on Stop Jogging! Symptoms SHORTNESS OF BREATH CHEST TIGHTNESS OR PAIN COUGHING WHEEZING How Do You Feel? BAMMF June 20, 2014 68

69. PredicAve Techniques for PrevenAve Advise 69 BAMMF June 20, 2014

70. From Persona to Societal Health Health Persona of diﬀerent people Environmental factors Disease outbreak Disease Modeling Geographical loca+on speciﬁca+ons BAMMF June 20, 2014 70

71. Use ObjecAve Massive Volume of Personal Data for Building Beker Disease Models. Persona and Societal Health Use Disease Model and Personal Data for Beker Quality of Life. 71 BAMMF June 20, 2014

72. Live EventShop and CollaboraAon •  Live EventShop Demo – h6p://auge.ics.uci.edu/eventshop/ •  Current Collaborators – Cyber-‐Physical Cloud Compu+ng Project •  NICT, NIST – SLN4MOP Project •  Sri Lanka Farmers; Prof. Ginige in Sydney leading – Open Source EventShop. •  HCL – Other partners BAMMF June 20, 2014 72

73. MulAmedia and Big Data •  MulAmedia was always Big Data. •  Many Major Challenges in Big Data are the same as in MulAmedia. •  Time for MulAmedia community to Accept and Act. BAMMF June 20, 2014 73

74. Thanks for your Ame and akenAon. For ques+ons: jain@ics.uci.edu

Multimedia big data 140619

Multimedia big data 140619

Recommended

More Related Content

What's hot

What's hot(19)

Viewers also liked

Viewers also liked(9)

Similar to Multimedia big data 140619

Similar to Multimedia big data 140619(20)

More from Ramesh Jain

More from Ramesh Jain(20)

Recently uploaded

Recently uploaded(20)

Multimedia big data 140619