2018030 @ National Cheng-Chi University, Management Information Systems Department Ph.D. program seminar

1. Big video data revolution,
challenges unresolved!
Albert Y. C. Chen, Ph.D.
Vice President, R&D
Viscovery

2. Albert Y. C. Chen, Ph.D.
陳彥呈博⼠士
albert@viscovery.com
http://slideshare.net/albertycchen
• Experience
2017-present: Vice President of R&D @ Viscovery
2015-2017: Chief Scientist @ Viscovery
2015-2015: Principal Scientist @ Nervve Technologies
2013-2014 Senior Scientist @ Tandent Vision Science
2011-2012 @ GE Global Research, Computer Vision Lab
• Education
Ph.D. in Computer Science, SUNY-Buffalo
M.S. in Computer Science, NTNU
B.S. in Computer Science, NTHU

3. LONG TIME AGO,
IN A GALAXY
FAR FAR AWAY

4. • Humans on Earth dreamed of doing Content
Based Image Retrieval (CBIR), to become the
next "Video Google". Most failed...
= ?

5. If we can't tag content automatically, why
not let everyone upload and tag?

6. Many new businesses were born
Make others tag for you!
Optimize Tags for search engines

7. Any better way to mange and utilize the
exploding amount of images and videos?
Deep Neural Network (DNN) helps us to extract much more
useful information reliably from images and videos.

8. Aspiring to become the next X?

9. Managing & Balancing
between Tech and Market Risk
• Technology Risk
• How much can you afford to burn on R&D?
• How to manage uncertainties in R&D?
• Market Risk
• Prototyping?
• Pretotying!!!

10. Artificial Neural Network (ANN)
Why didn’t it work; why now?
• MNIST digit data：28x28
• LeCunn’s 3 layer ANN:
1170 variables.
• Require tens of
thousands of samples.
• Only learn simple line/
curve combinations

11. AI Winter (1970-1980, 1990-2000)
• Early NN problems:
• redundant structure,
• slow learning speed
• need too much data
• bad learning
stability.

12. What’s in a NN
( )zσ+
( )zσ+
( )zσ+
( )zσ+
Input
weights
bias
activation
function

13. NN breakthroughs since 1970’s
1. Better Network Structure
• Convolutional Neural Network greatly reduces the number
of variables in NN’s designed for images and videos. —>
Improved convergence speed, reduced data requirements.
Upper-left corner
Bird Beak
Detector
Center Bird Beak
Detector
Almost identical, can be shared across regions

14. NN breakthroughs since 1970’s
1. Network Structure

15. NN breakthroughs since 1970’s
2. Improved Activation Functions
Large
Small
1x
2x
……
Nx
……
……
……
……
……
……
……
y1
y2
yM

16. NN breakthroughs since 1970’s
3. Effective Backpropagation
w1
w2
Clipping
[Razvan Pascanu, ICML’13]

17. NN breakthroughs since 1970’s
4. Efficient Training Methods
• Mini-batch
• Adaptive
Learning Rate
• Dropout, Batch-
normalization
minibatchno minibatch
1 epoch 所
做之優化

18. Deep Neural Networks (DNN)
way more complex and capable!

19. What do DNNs learn?
• Neurons act like “custom-trained filters”; react to
very different visual cues, depending on data.

20. What do DNNs learn?
• Neurons act like “custom-trained filters”; react to
very different visual cues, depending on data.

21. • Does not “memorize” millions of viewed images.
• Extracts greatly reduced number of features that
are vital to classify different classes of data.
• Classifying data becomes a simple task when
the features measured are “”good”.
What do DNNs learn?

22. Recent breakthroughs powered
by DNN
• Autonomous cars
• Cashier-free stores
• Advanced manufacturing
• ...

23. • Not this one...
OK. We have a hammer. Now what?
• This one!
However, don't run around and try to hit everything!!!

24. AI revolution landscape overview
(BCG AI Report, 2016/10)
appl.
layer
tech
layer
infra
layer
solution
platform
libraries
modules
data
machine computing power
data accumulation via open API
AI/DNN library AI/DNN library
general purpose
platforms
general purpose
platforms
app-specific
platforms
app-specific
platforms
app app app app app
HW
co.
VerticalAIStartups
agri. manu. med. fin. retail trans.
E.g., 1: Google, Amazon, FB, 2: IBM, 3: Walmart, 5: NVidia

25. Why startups?
Fail early, fail often, fail forward!

26. The Venture Capital Funnel is
extremely efficient at filtering ideas

27. Vertical AI Startups
Solving industry-specific problems by combining
AI and Subject Matter Expertise.
• Full Stack Products
• Subject Matter Expertise
• Proprietary Data
• AI delivers core value
(Bradford Cross, 2017/06/14)

28. Why Go Vertical?
• Don’t get ripped off—don’t get disconnected
from key customer/providers.
• Tasks get commoditized
• Software is eating the world—every company in
every industry needs to be a tech company.
• Enterprise exits come in cohorts
(Bradford Cross, 2017/06/14)

29. So, which vertical?

30. (Bradford Cross, 2017/06/14)
• choose large TAM
with high margin
• avoid 1% fallacy
• avoid confirmation
bias
So, which vertical?

31. (Bradford Cross, 2017/06/14)
AI startups total investmentUnicorn tally

33. OK, back to the market challenge
Traditional
Media
Free Content,
Ad Revenue
Subscription
Revenue
Do Nothing?
Sitting Duck!
Improve Ad
Revenue
Content-
related video
ads
Own platform
Shared/
congregated
platform
customized
recommendatio
ns
ad measure /
brand safety
Image/Video content owners struggling to monetize
Can AI solve it all?

34. Why not go ahead and analyze all videos? COST!
0600120018002400
2014 2015 2016 2017 2018 2019
Revenue Hardware Cost Data Collection Cost Model Training Cost
Technology Inflection Point:
• Computer vision cost has dropped significantly
while market demand has increased beyond
exploratory stages.
• Overall cost reduction resulted from accumulated
domain expertise, efficient machines learning
techniques, and lower computing cost.
Case study on leading e-commerce site in APAC, with 1M SKU and >20M calls/month.

35. • Business model vs videos analyzed
Need to deliver value while analyzing!
o2o marketing
ad effectiveness
analysis
campaign
effectiveness
analytics
direct-sell
content-related ads
programmatic
content-related ads
Big Data Insights
0 2.5 5 7.5 10
videos
analyzed
(10^N) • Extremely costly to
analyze video content.
Google charges
$1.5/1000 images. with
3600 seconds/hour, it
will cost $5.4 / hour
video / single single
recognition task.
• Costly to store videos
and their recognition
results.

36. Hindsight on "right approach" for media
Increase user’s time spent & stickiness
Multiply traffic w/o subsidizing growth.
(4) Video content-related ads & e-commerce
(3) Ad exposure measure
(2) Content recommendation
Grow user & advertiser
budget organically with
new services
(1) Image/Video Content Recognition as a Service
Focus on
Revenue
Growth
(5)DataMiningandAnalyticsasaService
Goal
Media
Size

37. Value proposition to media:
Start making money the first day!
user/time/ad/budgetincrease(times)
0
1
2
3
4
TotalSiteRevenueIncrease(times)
04.5913.518
current Better recommendation
Improve user stickiness
Better measurement
Grow user & ad budget
Content-related ad, EC
Focus on Revenue
# users time / user ad / user*time budget / ad total site revenue

38. Viscovery's Complete Solution
for Media and Beyond
Media
AdRetail
Archive & Search
Content
Recommendation
Video Generator*
Brand Insight
Ad Effectiveness
Measure
Brand Safety
Assurance
Visual
Content
Recognition
Smart Checkout
Video to EC
Consumer Insights

39. Media AI: Video content archive & search

40. Media AI: Content Recommendation
Article-to-Article, Article-to-Video, Video-to-Video
The 31 Tiniest Cat Breeds
Cats are notoriously small
household pets, but some are
even tinier than the norm. If you
have tight living quarters or
simply can’t get enough of cats
little enough to resemble kittens
forever, there are plenty of small
cat breeds to suit you. When it
comes to cats, bigger isn’t
always better, as some of the
most charming felines come in
small packages.
Meaningful
Terms Re-weighting
High-dimension
Mapping
Terms (Attribute)
Extraction
Para

41. Media AI: Video Generator
powering the "pivot to video" movement!
Text Analysis
(NLP)
Image
Recognition
Key Phrases
Sentiment
Audio
Subtitles
F/O/S
Concept
Media
Database
Generation Engine

42. Ad AI: ad exposure measure
# Appearances 68 times
# Appearances with
Duration > 5 sec
2 times
# Appearances with
Duration > 15 sec
0
# Appearances with
Duration > 30 sec
0
Longest Duration of
Appearance
9 sec
Average Duration of
Appearance
2.12 sec
Average Appearance
Interval
37.79 sec
Total Duration of
Appearance
144 sec
Ad Exposure for Audi @ China GT (Shanghai) 2017
# times of logo
occupying >5%
screen size
8 times
Duration of logo
occupying > 5%
screen size
16 sec
# times of logo
occupying > 10%
screen size
0 times
Duration of logo
occupying > 10%
screen size
0 sec
Average logo size
(relative to screen)
2.12%
Maximum logo size
(relative to screen)
11.05%

43. Ad AI: Content-related ads

44. Ad AI: Brand safety assurance
Terroris Violence
DrugPolitical Explicit Content

45. Retail AI: o2o redirection

46. Retail AI: smart checkout
Oreo cookie ……………
$4.95
Bounty Paper Towel … $3.45
Cal. Red wine………… $12.44
--------------------------
Total
$20.84
Scan payment QR Code:
wechat Alipay

47. Smart checkout demo

48. Retail AI: merchandise management
•2D+3D cameras for head and eye tracking
•Detect Point-Of-Interest
•Consumer profiling and data collection
•Integrate with kiosk big screen devices
•ESL Integration
Smart POS
• Smart check out
• Client-facing POS screen
with 3D camera
• POS-on-the-run
• Customer Big Data
• ESL integration
Smart Shelf

49. Retail AI: Cunsumer Insights
Female, 30-35
Ms. Mary Rice
Status: VIP
Housewife
Brand: P&G,
LV, Nike,
Tiffany
Already
holding
product A
Reaching for
product B on
the shelf
E.g., People who appear interested in milk powder will
get an eDM containing relevant promo info.
Recognize customer appearance and behavior characteristics
to establish a buying recommendation database.

50. • Clear segment focus: Any corporation smaller
than Google, Facebook, Baidu, Alibaba,
Tencent.
• Need to go from 0-80 fast. Then, help them go
from 80-90 and catch up quickly.
• How can we make the "0-80" part scalable?
Building Technology Barrier

51. Sample-less Face Recognition

52. Merchandise Recognition

53. Abstract and Extended Concepts

54. • Top 5 Factors in Success (across >200 startups
with successful exit)
By the way...

55. Timing, timing, timing.
Rule of Thumb:
• Customers within the target vertical having
immediate unmet needs.
• VCs scouting that vertical ready to invest.
2000
.com bubble burst
2002 2004 2006 2008 2010 2012 2014 2016
housing bubble burst
broadband penetration > 50%

56. Building up Competitive
Advantage for Vertical AI Startups
Business
DataTechnology
Speed
Team
Speed
Speed

57. Some Hard Lessons Learned
• Swallowed alive by giants (Speed, Timing)
• Aiming for a narrow exit. (Business)
• Failing to disrupt the market (Technology)
• Everybody in the field is lying (Data)
• Don’t underestimate the cost of going long in
any vertical.

58. Unstructured
videos
⼴广告主
DSP SSP
⼴广代
媒代
DMP
Viscovery SDK
互动层Ad Exchange
⼴广告版位
(Key moment)
Categories
Users
User segment
Viscovery Server
Video id and key
moments
Publishers
Video crawler
VSP
ranking
system
One video
w/ its id
Videos
Categories User profile
From DMP or publishers
Asynchronous processes
Filter
Recommander engine
Category of ad
Tracking Server
监控链结(互动层)之资讯
监控链结(互动
层)之资讯
6
53
2
14
vad.json
Tag2AD
expensive cheaper
ad request
vtag.raw
vtag.raw
Database
Engine
api
api
VDSAgent
web
api
web
api
ad op {(vid, {ti, ci})}
Video id
Key moment
AD categoryhttps://vsp.viscovery.com/
Viscovery went long in AdTech

59. When something is important enough,
you do it even if the odds are not in your favor.
Elon Musk
Falcon 9
takeoff
Falcon 9
decelerate
Falcon 9
vertical
touchdown