Successfully reported this slideshow.

SXSW2017 @NewDutchMedia Talk: Exploration is the New Search

6

Share

1 of 50
1 of 50

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

SXSW2017 @NewDutchMedia Talk: Exploration is the New Search

  1. 1. Proprietary and Confidential Exploration is the New Search / Lora Aroyo / Vrije Universiteit Amsterdam / IBM Netherlands (CAS) / Tagasauris Inc / http://lora-aroyo.org / @laroyo
  2. 2. 2 massive amount of online digital video content
  3. 3. 3 online video content is 64% of Internet traffic
  4. 4. 4 300h of video uploaded each min on YouTube
  5. 5. 5 in 2020 it would take a person more than 5 million years to watch the videos shared online in a month
  6. 6. 6 there are videos out there relevant to you …
  7. 7. 7 but …
  8. 8. 8 most online videos NOT DESCRIBED CAN’T BE FOUND WON’T BE SEEN
  9. 9. 9 what can we do to describe videos better?
  10. 10. 1 machine computation can do a lot
  11. 11. 11 recognise objects & concepts Proprietary and Confidential
  12. 12. unlock rich semantics Proprietary and Confidential 12
  13. 13. Proprietary and Confidential [setting] WESTERN [character] DOLORES [story arc] FLASHBACK 7 [object] TRAIN [action] SHOOTOUT [mood] SURREAL [audience] SHOCKED [extras] 22 [role] BYSTANDER 13 discover creative contexts
  14. 14. 14 link across online videos Proprietary and Confidential
  15. 15. 15 put all that descriptive information together to know what’s in a video on a second-by-second basis 6 so, we can … Proprietary and Confidential
  16. 16. 16 add also behavioral information to know how people watch & talk about videos 6 and, we can … Proprietary and Confidential Viewership Channels profiles Social Media Buzz Audience Demographics Audience Contexts Platform profiles
  17. 17. 17 all this will make video search better
  18. 18. 18 but we need more …
  19. 19. 19 SEARCH vs. EXPLORATION / helps only when you know what to search for / assumes you understand a topic / accuracy driven / click-through driven / helps when you don’t know what to search for / / helps you understand & deepen in a topic / / serendipity driven / / focused on engagement /
  20. 20. 20 what can we do to support video exploration?
  21. 21. 2 human computation can do a lot
  22. 22. 2 CrowdTruth.org bring more semantics of the real world
  23. 23. 2 diversity of opinions, sen)ments, contexts collected in a decentralized way aggregated views over collec1ons introduces perspectives http://crowdtruth.org
  24. 24. Proprietary and Confidential 24 consolidate experts & end users perspec)ves for con1nuous improving video descrip1ons
  25. 25. 25 humans & machines can go a long way
  26. 26. Proprietary and Confidential HUMAN-IN-THE-LOOP AI FOR VIDEO EXPLORATION / Machines help to break-down video into granular moments, i.e. shots & scenes / Machines generate multitude of paths within and across videos / Humans perform simple actions, e.g. watching, following and rating a path / Machines generalise from these actions using explicit semantics / Machines learn to evolve & improve exploration path / Orchestrate a continuous human and machine symbiosis / The ultimate aim is to reach a tipping point for video exploration, e.g. web search, speech recognition
  27. 27. 2 to transform the online video experience
  28. 28. 2 Multimedia Explorations for Smart Culture
  29. 29. 2 Interac1ve Explora)on & Discovery in Context linking objects to events and en&&es building automa1c storylines (narra1ves) on topics or objects Access to integrated online mul)media collec1ons using Linked Open Data to integrate metadata of various heritage collec1ons DIVE+ Aggregated views over the collec1on collec1ng perspec&ves from crowds & niches http://diveproject.beeldengeluid.nl/
  30. 30. 3 Erp, M. van; Oomen, J.; Segers, R.; Akker, C. van de; Aroyo, L.; Jacobs, G.; Legêne, S; Meij, L. van der;O ssenbruggen, J.R. van; Schreiber, G. AutomaDc Heritage Metadata Enrichment with Historic Events Museums and the Web 2011 hKp://www.museumsandtheweb.com/mw2011/ papers/automaDc_heritage_metadata_enrichment_with_hi
  31. 31. 31 types
  32. 32. 32 filters
  33. 33. 3 explore entities related entities
  34. 34. 3 entity exploration
  35. 35. 3 object exploration
  36. 36. 3 people filter
  37. 37. 3 event filter
  38. 38. 3 narrative
  39. 39. 3
  40. 40. 4 Video Explorations with Micro-moments
  41. 41. MOMENTS REDEFINE CURRENT VIDEO SEARCH MOBILE/SOCIAL OPTIMIZED RELEVANT SEARCH RESULTS PREVIEW SPECIFIC MOMENTS OR WATCH FULL VIDEOS DISCOVER ACTIONABLE MOMENTS WITHIN VIDEO
  42. 42. MOMENTS: PERSONAL VIDEO CHANNEL SEARCH FOR ANYTHING, WITH ANYTHING HYPERMEDIA PLAYER W/ LINKED MOMENTS & FULL VIDEOS AI ACTIVELY LEARNS USER PREFERENCES DISCOVERS MORE & MORE CONTENT
  43. 43. AI, DEEP LEARNING & NETWORK EFFECTS Ensemble algorithmic processing Video, audio & text Extract & understand entities Does what computers are good at AI & DEEP LEARNING: // // // Human assisted computing Collective intelligence (incl. fans) Waze effect Does what humans are good at NETWORK EFFECTS: // // // OBSERVE & LEARN VERIFY & EXTEND *U.S. Patent Application #13/863,751 Web-scale layer of structured, linked data. //// DOMAIN-SPECIFIC DATA MOMENTS
  44. 44. 4
  45. 45. 4
  46. 46. 4
  47. 47. Proprietary and Confidential HUMAN-IN-THE-LOOP AI FOR VIDEO EXPLORATION / Machines help to break-down video into granular moments, i.e. shots & scenes / Machines generate multitude of paths within and across videos / Humans perform simple actions, e.g. watching, following and rating a path / Machines generalise from these actions using explicit semantics / Machines learn to evolve & improve exploration path / Orchestrate a continuous human and machine symbiosis / The ultimate aim is to reach a tipping point for video exploration, e.g. web search, speech recognition
  48. 48. 49 to explore strange new worlds …. to boldly WATCH what no-one has WATCHED before
  49. 49. Proprietary and Confidential Exploration is the New Search / Lora Aroyo / Vrije Universiteit Amsterdam / IBM Netherlands / Tagasauris Inc / http://lora-aroyo.org / @laroyo

×