Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The Quest for Video Understanding
Xiaofeng Ren
Institute of Data Science and Technologies (iDST), Alibaba Group
Alibaba
2
Alibaba Ecosystem
3
CLOUD COMPUTING OPERATING SYSTEM
DATATECHNOLOGY TEAM
Online and Mobile Commerce Platform
$
BUYERSSELLE...
Alibaba Mission
4
CLOUD COMPUTING OPERATING SYSTEM
DATATECHNOLOGY TEAM
Online and Mobile Commerce Platform
$
BUYERSSELLERS...
(E-)(M-)(O-) Commerce
5
Text Image Video Interactive Media
Taobao @ 2009
7
Taobao @ 2017
8
Taobao Search: Text  Text
9
Title:茜舞2017春装新品 韩版修身公
主裙红色连衣裙 无袖打底背心裙春秋
Q:无袖蓬蓬裙连衣裙夏 a型
Title:美特斯邦威卫衣女士
春装字母拼色针织套头衫
Q:百搭灰色套头无...
Taobao Search: Text  Text
10
Title:茜舞2017春装新品 韩版修身公
主裙红色连衣裙 无袖打底背心裙春秋
Q:无袖蓬蓬裙连衣裙夏 a型
Title:美特斯邦威卫衣女士
春装字母拼色针织套头衫
Q:百搭灰色套头...
Taobao Search: Text  Image
11
Title:茜舞2017春装新品 韩版修身公
主裙红色连衣裙 无袖打底背心裙春秋
Q:无袖蓬蓬裙连衣裙夏 a型
• Joint text and image embedding fo...
Taobao Search: Image  Image
12
3 billion images over 100 million products, 12 million daily active users
Videos on Taobao
13
Videos on Taobao
14
• Promotional video
• Video guide on how to use product
• Documentary on how a product is made
… …
Live Streaming on Taobao
15
• Many top-selling stores have live streaming
• Allow user interaction (chat, FAQ)
• Conversio...
Videos on Taobao
16
• Short videos are abundant on Xianyu (second-hand trading platform)
• Demonstrations of products; som...
Videos on Taobao
17
• Authoring tools
• Video enhancements
• Content analysis and tagging
• Video-to-text matching
… …
AR/VR on Taobao
18
Youku & Tudou: “Youtube-like”
19
Founded 2006
Founded 2005
IPO 2010
IPO 2011
Merged
2012
Joined Alibaba
2016
Youku: Video Hosting
20
30M paid members
~400M active users
700-min per month
Youku: Product Identification
21
Youku: Product Identification
22
Youku: Product Placement
23
Tudou: UPGC Sharing
24
Tudou: UPGC Sharing
25
Copyright protection
Authoring tools
Search & recommendation
… …
Cainiao Logistics
27
24 hours in China
72 hours globally
1B packages on 11/11Mission:
Robots @ Cainiao
Delivery Robots Delivery Drones
Picking Robots
Sample Video Applications
30
Digital
Entertainment
Logistics
CLOUD COMPUTING
Videos Everywhere
31
CLOUD COMPUTING OPERATING SYSTEM
DATATECHNOLOGY TEAM
Online and Mobile Commerce Platform
$
BUYERSSELL...
DAMO Academy (including iDST):
$15B investment on R&D to solve “big problems”
Thank You!
iDST@Alibaba
Xiaofeng Ren at AI Frontiers: The Quest for Video Understanding
Xiaofeng Ren at AI Frontiers: The Quest for Video Understanding
Upcoming SlideShare
Loading in …5
×

Xiaofeng Ren at AI Frontiers: The Quest for Video Understanding

747 views

Published on

In this talk I will briefly discuss the ubiquitous needs of video and video understanding across Alibaba and the challenges that are being addressed and solved at iDST, Alibaba's AI R&D division. Examples include mobile shopping on Taobao, video search and recommendation on Youku and Tudou, and real-time systems for Cainiao Logistics and City Brain.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Xiaofeng Ren at AI Frontiers: The Quest for Video Understanding

  1. 1. The Quest for Video Understanding Xiaofeng Ren Institute of Data Science and Technologies (iDST), Alibaba Group
  2. 2. Alibaba 2
  3. 3. Alibaba Ecosystem 3 CLOUD COMPUTING OPERATING SYSTEM DATATECHNOLOGY TEAM Online and Mobile Commerce Platform $ BUYERSSELLERS Marketing Financing & Payment Solution Professional Service Providers Digital Entertainment Logistics Mobile Browser Location-BasedSocial Media Weibo
  4. 4. Alibaba Mission 4 CLOUD COMPUTING OPERATING SYSTEM DATATECHNOLOGY TEAM Online and Mobile Commerce Platform $ BUYERSSELLERS Marketing Financing & Payment Solution Professional Service Providers Digital Entertainment Logistics Mobile Browser Location-BasedSocial Media Weibo make it easy to do business anywhere
  5. 5. (E-)(M-)(O-) Commerce 5
  6. 6. Text Image Video Interactive Media
  7. 7. Taobao @ 2009 7
  8. 8. Taobao @ 2017 8
  9. 9. Taobao Search: Text  Text 9 Title:茜舞2017春装新品 韩版修身公 主裙红色连衣裙 无袖打底背心裙春秋 Q:无袖蓬蓬裙连衣裙夏 a型 Title:美特斯邦威卫衣女士 春装字母拼色针织套头衫 Q:百搭灰色套头无帽卫衣女
  10. 10. Taobao Search: Text  Text 10 Title:茜舞2017春装新品 韩版修身公 主裙红色连衣裙 无袖打底背心裙春秋 Q:无袖蓬蓬裙连衣裙夏 a型 Title:美特斯邦威卫衣女士 春装字母拼色针织套头衫 Q:百搭灰色套头无帽卫衣女
  11. 11. Taobao Search: Text  Image 11 Title:茜舞2017春装新品 韩版修身公 主裙红色连衣裙 无袖打底背心裙春秋 Q:无袖蓬蓬裙连衣裙夏 a型 • Joint text and image embedding for description • Leverage user click data to learn query-description similarity
  12. 12. Taobao Search: Image  Image 12 3 billion images over 100 million products, 12 million daily active users
  13. 13. Videos on Taobao 13
  14. 14. Videos on Taobao 14 • Promotional video • Video guide on how to use product • Documentary on how a product is made … …
  15. 15. Live Streaming on Taobao 15 • Many top-selling stores have live streaming • Allow user interaction (chat, FAQ) • Conversion rate much higher than normal
  16. 16. Videos on Taobao 16 • Short videos are abundant on Xianyu (second-hand trading platform) • Demonstrations of products; sometimes people showing skills • Much richer information than text descriptions
  17. 17. Videos on Taobao 17 • Authoring tools • Video enhancements • Content analysis and tagging • Video-to-text matching … …
  18. 18. AR/VR on Taobao 18
  19. 19. Youku & Tudou: “Youtube-like” 19 Founded 2006 Founded 2005 IPO 2010 IPO 2011 Merged 2012 Joined Alibaba 2016
  20. 20. Youku: Video Hosting 20 30M paid members ~400M active users 700-min per month
  21. 21. Youku: Product Identification 21
  22. 22. Youku: Product Identification 22
  23. 23. Youku: Product Placement 23
  24. 24. Tudou: UPGC Sharing 24
  25. 25. Tudou: UPGC Sharing 25 Copyright protection Authoring tools Search & recommendation … …
  26. 26. Cainiao Logistics 27 24 hours in China 72 hours globally 1B packages on 11/11Mission:
  27. 27. Robots @ Cainiao Delivery Robots Delivery Drones Picking Robots
  28. 28. Sample Video Applications 30 Digital Entertainment Logistics CLOUD COMPUTING
  29. 29. Videos Everywhere 31 CLOUD COMPUTING OPERATING SYSTEM DATATECHNOLOGY TEAM Online and Mobile Commerce Platform $ BUYERSSELLERS Marketing Financing & Payment Solution Professional Service Providers Digital Entertainment Logistics Mobile Browser Location-BasedSocial Media Weibo
  30. 30. DAMO Academy (including iDST): $15B investment on R&D to solve “big problems”
  31. 31. Thank You! iDST@Alibaba

×