Machine Learning & Data Science come to DAM

© 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe .
Machine Learning & Data Science come to DAM
Tuesday November 29th , 2016
Sponsored by:
Elliot Sedegah
Strategy & Product Marketing
Adobe Experience Manager
Adobe
Dr. Jonas Dahl
Product Manager
Machine Learning & Innovation
Adobe

© 2016 Adobe Systems Incorporated. All Rights Reserved.
Machine Learning & Data Science come to DAM
November 29, 2016

Agenda
3
Understanding
Content Velocity
Challenges
Demystify Machine
Learning & Data
Science
5 Practical Applications
with DAM
Tips & Tricks

It’s very getting more costly & complex.
4
85%
SLOW TIME TO
MARKET
71%
HIGHER COST
& COMPLEXITY
IDC InfoBrief, Proving the Value of Digital Asset Management
Forrester, Benchmark Your B2B Content Marketing Strategy And Maturity
Creative Team
(In-house)
Partner
Stock
Content
Employees
Content
Community &
User Generated
Content
Creative Agency
(External)
Creative Agency
(External)
Creative Agency
(External)
Your
Organization
Under pressure to create
assets and deliver more
campaigns, more quickly
Creating over 10x the
assets today to support
increasing channels

We’ve all heard the challenges.
“eCom is growing and current
process cannot support it”
“everything about assets is
manual, slow and tedious”
“I can’t seem to find
anything digital asset that I
KNOW exist”
“we need a better process to
manage assets”
“multiple teams are involved in the
creation of content with different
storage media”
“I wish I knew which assets were being
used and how they were performing”
“cannot scale personalization
across channels”
“we have no idea how much we
spend on creating assets”

Maintaining Content Velocity is hard –we will need some help.
6© 2016 Adobe Systems Incorporated. All Rights Reserved. Adobe .

“We are surrounded by data,
But starved for insights”
7
- Jay Baer

Digital Asset
Data is the bedrock of DAM
We need lots of Metadata:
• Descriptive metadata
• Structural metadata
• Technical metadata
• Administrative metadata
• Rights management metadata
• Preservation metadata…

© 2016 Adobe Systems Incorporated. All Rights Reserved.© 2016 Adobe Systems Incorporated. All Rights Reserved.
Analytics & Data can enrich every digital asset.
DAM
ANALYTICS
Performance & Usage data:
• How many times, where an asset has been
used?
• Which assets are being used the most?
• Which assets have not been used?
• Are the top assets being deployed across all
channels?
• Which assets are resonating the most,
creating max impact?

The DAM of the future will be different.
ANALYTICS INTELLIGENCE

The DAM of the future will be different.
ANALYTICS INTELLIGENCE
Scale & Automation
Data & Machine learning will automate some manual tasks.
Insights
Every single asset is a ‘signal’ that can provide insights &
recommendations to your business”

How did we get here?
12
Source: Nvidia Blog: https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/
ARTIFICIAL
INTELLIGENCE
Early artificial intelligence
stris excitement.
1950’s 1960’s 1970’s 1980’s 1990’s 2000’s 2010’s
MACHINE
LEARNING
Machine learning begins
to flourish Deep Learning
Deep Learning breakthroughs
Drive Al boom.
0010101010001001001
0010101010001001001
0010101010001001001
0010101010001001001
0010101010001001001
0010101010001001001

Haven’t we tried this before? What’s changed?
13
CLOUD
Highly available and scalable
compute
DATA REVOLUTION
We have much more data
to use
ALGORITHMS
Faster and more intelligent

The old approach: image recognition was difficult and less accurate
14
You must tell the system what to look for….
“This is face because…..”
A face has:
q 2 eyes
q 1 Nose
q 1 Mouth
q Teeth
q 2 Ears
Is this a Face?
ü2 eyes
ü1 Nose
ü1 Mouth
üTeeth
X Ears
This method has a higher error rate & is
more labor intensive:
• Higher Error Rate
• Model Creation: Manual
No?

Deep Learning is more accurate and can generalize concepts
15
Instruction to AI Platform:
(1) Here are many faces…
(2) Teach yourself to to “see”
This is a Face.
DAM
Yes (95%)
model

Deep Learning “trains” itself to learn concepts
16
Source: http://www.andrewng.org/publications/
Faces
Source: www.eecs.umich.edu/~honglak/icml09-ConvolutionalDeepBeliefNetworks.pdf

Study: Deep Learning Drops Error Rate for Breast Cancer Diagnoses by 85%
17
Source: https://blogs.nvidia.com/blog/2016/09/19/deep-learning-breast-cancer-diagnosis/
350.00%
290.00%
50%
0.00%
50.00%
100.00%
150.00%
200.00%
250.00%
300.00%
350.00%
400.00%
Study Pathologist All Model All Model + Study
Pathologist
(Al + Pathologist)> Pathologist
Study Pathologist All Model All Model + Study Pathologist

How Deep Learning classifies images with tags
• How it works for images
• Takes a Low Res image, pushes it to a neural network (likely in the cloud)
• The Cloud returns tags with predictions
18
Deep Learning model

What type of predictive “smart” Tags can be added?
19
Deep learning technology can analyze
and extract the semantic content of
images based on a training set
§ Automatically tag images with keywords :
§ Photo type (macro, portrait, etc.)
§ Activities (running, skiing, hiking),
§ Emotions (smiling, crying)
§ Objects (car, roads, people, etc.)
§ Locations, Animals, primary colors, and
more

Machine-generated and User-generated metadata
§ Existing metadata is preserved
§ Smart tags are stored separately
from user-generated tags
20
Digital Asset
User-generated
metadata
Machine-
Generated
Digital Asset
User-
Generated,
ADD APPEND
Digital Asset
User-generated
metadata
Machine-
Generated
REPLACE
Machine-
Generated
§ Smart tags are added to existing
metadata entries
§ Existing metadata is replaced with
machine generated

© 2016 Adobe Systems Incorporated. All Rights Reserved .
Use confidence scores to drive search results
Sunset (88) Sunset (75) Sunset (71)
IMAGE
17.jpg
5 days ago 45.8 KB
612 x 320
IMAGE
15.jpg
5 days ago 33.4 KB
612 x 320
IMAGE
27.jpg
5 days ago 732 KB
1600 x 1067

How to best set confidence
threshold
• Confidence threshold
• Max. number of Smart Tags per asset
22
Balance between search performance and
richness
Typically 10-30 tags per asset
Coffee (93) Café (75)
Cup (92) Tea (74)
Drink (83) Mug (72)
Beverage (76) White (72)
Breakfast (75) Expresso (71)
Table (75)
Brown (69) Black (67) Caffeine (67) Morning (66)
Aroma (66) Close-up (65) Saucer (65) Spoon (63)
Break (63) Food (62)
Italian Culture (32) Agile (32)

Example 1: Stock Photography contribution
23
Benefits:
- Reduces under-tagged assets
- Search Relevancy
- Boost search If manual tag
matches generated tag
How it works:
1. Upload an image
2. Algorithm will analyze it
3. Generating keywords from top
images
4. Reorder keywords by relevancy
5. Display the top 5 to review, edit
& reorder

Example 2: Sourcing Community Content with Filtering & Moderation
Moderation during bulk ingest
Auto-populate social media content with tags
24
Filter: NSWF images, Off-Brand
DISCOVER
SOCIAL CONTENT ORGANIZE
TOPIC
CAMPAIGN
LOCATION
APPROVED
ADMIN MODERATOR
RIGHTS

We are creating more content than ever before.
25
3,500
3,000
2,500
2,000
1,500
1,000
500
0
#ofPhotosSharedperDay(MM)
Facebook
Owned
properties
2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Daily Number of Photos Shared on Select Platforms, Global, 2005-2015
Source: Snapchat, Company disclosed information, KPCB estimates
Note: Snapchat data includes images and video. Snapchat stories are a completion of images and video. WhatsApp data estimated based on average of photos shared disclosed in Q115 and Q1:16
Instagram data per Instagram press release. Messenger data per Facebook (~9.5B photos per months). Facebook shares ~2B photos per day across Facebook, Instagram, Messenger, and WhatsApp (2015)
@KPCB
Snapchat
Facebook Messenger (2015 only)
Instagram
WhatsApp (2013 onward only)
Facebook

Example 3: Global Hotel
26
Challenges with content:
• Unable to keep up with new content (up to 1,000) per day
• Incomplete metadata – “taxonomy doesn’t naturally match users search
terms” – Nobody adds metadata for ‘subjective’ descriptions. Such as “calm”
• Duplicate/Similar content
• Unable to easily filter Customer & User Generated Content
Sources content from:
• Internal & External Agencies – Photoshoots of global owned- hotel properties,
Lifestyle content, Global marketing campaigns
• Franchise Partners – Publish local content
• Commercial Partners
• Branded Products (e.g. credit cards, events,)
• Local Tour Guides – Submitting “What’s happening this week” content for approval
Users/Customers – Shots from destinations, vacations, conference events

Example 4: Global Publisher
Sources content from:
• Content Producers
• Licensed Content
• Shared Content from internal brands
• Users/Readers – Social Feeds
27
Future Need to
Repurpose Example:
Tags you need are:
“Woman”, “Calm”,
“Fitness”
Challenges
• Not enough resources to tag - Too much content (200-400) per day
• Undertagged assets - Giant pool of content that was built up over the
years that were never tagged.
• Incomplete metadata – “taxonomy doesn’t naturally match users
search terms” – No metadata for ‘subjective’ descriptions. (e.g “calm”)
Example:
Original Asset
List Tags:
Article #123
Studio, Rights, “Yoga
Studio”

Example 5: Consumer Electronics Company
Assets are used in multiple places
• Report Asset usage on Main Website,
Mobile Apps, Regional Websites & 3rd-
Party & Partner Sites
Assets commissioned from
multiple agencies
• Need an understanding of the
performance of the assets as attributed
to the creating entities.
28
Opportunities with Asset Insights
• Tracking usage & performance of assets
• use data to drive search results
• Understand agency and subcontractor
performance
• Compare Campaign performance
• Provide feedback to content creators
• Discover which assets are being
reused/repurposed/high-performing

Other applications for this technology
Custom Smart tags
• User tags a small
number of images
• System is trained based
on these images and
can now tag similar
images
Similarity Search
• Deep learning based
similarity search is
much better than
traditional similarity
search because it is
based on concepts
Recommendations
• Recommendations
will be a more
important part of how
marketers discover
assets
Video
• Smart tagging for
video will also account
for the temporal
aspect of video
• Analyze and Identify
key frames, audio, and
more.

Artificial Intelligence & Data Science will be an essential tool to increase
the value of the assets we manage
30
MOVE FASTER MORE EFFICIENT MORE INTELLIGENCE

Audience Participation:
Can you predict the tags?
31

Audience Participation: What tags would you assign to this image?
32

Next Steps & Practical Advice
Build the Business Case
• Think productivity, content reuse/savings
33
Determine what data would make a difference for your stakeholders
• Performance data? Search data?
Don’t abandon your metadata & taxonomy strategy !
• Instead, think about how to add and improve it!
Start Small & Take a Phased Approach
• Start with a Focused POC that will expand in scope over time
Don’t create another data silo with your DAM
§ Connect to your broader organization’s data sources

Questions?
34

Machine Learning & Data Science come to DAM

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Machine Learning & Data Science come to DAM

Similar to Machine Learning & Data Science come to DAM (20)

Recently uploaded

Recently uploaded (20)

Machine Learning & Data Science come to DAM