Artificial Intelligence (AI) is fast becoming the cornerstone of digital transformation in business and industry. A widespread misconception about AI is that its power derives from the algorithms, or the techniques, used to implement learning and modeling, the messaging being that sophisticated techniques lead to better outcomes. As it turns out, the nature of the data used to train AI algorithms, specifically its quality, depth and breadth, impacts outcomes far more than the technique itself.

1. What Every Executive
Must Know Before Using AI
© Mobilewalla, Inc. 2018
Anindya Datta, Ph.D.
Founder, CEO, Chairman - Mobilewalla

2. AI is Pervasive
Investments
$26-39B
SPENT BY COMPANIES IN AI
IN 2016
$15B
FUNDING TO AI STARTUPS AS
OF 2017
AUTO, TELCO,
FINANCE
LEADING AI ADOPTERS
Trends Tokyo Olympics 2020
TOP TOPIC IN 2017
LINKEDIN INSIGHTS
200K+ ACADEMIC
PAPERS
9X INCREASE SINCE 1996
ML, DL and NLP
3 MOST IN-DEMAND JOB SKILLS
FACIAL RECOGNITION
DRIVERLESS CARS
SPECTATOR GUIDE
SYSTEM
SOURCES:McKinsey'sStateOfMachineLearningAndAI2017,CBInsightsAITrendsToWatchin
2018,LinkedInContentInsightsAnnual2017,AIIndex2017Report
© Mobilewalla, Inc. 2018

3. AI & Machine Learning
Strictly speaking, Machine Learning (ML) is a
sub-field of Artificial Intelligence (AI)
However, when applied to solving business
problems of interest to organizations, ML and AI
are virtually synonymous
© Mobilewalla, Inc. 2018

4. How
Machine
Learning
Works
© Mobilewalla, Inc. 2018

5. How Machine Learning Works
It all starts with a
problem
© Mobilewalla, Inc. 2018

6. How Machine Learning Works
Email
Automated
Spam
Filter
Spam
Non-Spam
© Mobilewalla, Inc. 2018

7. Two Core Components: Data & ML Algorithm
Training Data
© Mobilewalla, Inc. 2018
ML Algorithm

8. Binary Classification
Two Core Components: Data & ML Algorithm © Mobilewalla, Inc. 2018
Training Data ML Algorithm
Features
1. Originating Country
2. Number of Words
3. Presence of Key-phrases

9. TRAIN
How Machine Learning Works: #1 Train
Training Data ML Technique
© Mobilewalla, Inc. 2018
Features
1. Originating Country
2. Number of Words
3. Presence of Key-phrases
1
Binary
Classification

10. How Machine Learning Works: #2 Build Model
Model
Training Data ML Technique
© Mobilewalla, Inc. 2018
Intermediate Decision
Model
TRAIN
1
BUILD
MODEL
2
Features
1. Originating country
2. Number of Words
3. Presence of Key-phrases
Binary
Classification

11. How Machine Learning Works: #3 Validate © Mobilewalla, Inc. 2018
Features
1. Originating country
2. Number of Words
3. Presence of Key-phrases
TRAIN
1
BUILD
MODEL
2
Training Data ML Technique Intermediate Decision Model
VALIDATE
3
Hold-Out Validation Set
Binary
Classification

12. VALIDATE
3
How Machine Learning Works
Training Data ML Technique
© Mobilewalla, Inc. 2018
Intermediate Decision Model
Features
1. Originating country
2. Number of Words
3. Presence of Key-phrases
Prediction
Accuracy
Acceptable?
Final Decision Model
YES
NO
TRAIN
1
BUILD
MODEL
2
REFINE
4
Hold-Out Validation Set
Binary
Classification

13. Component 1: Training Data © Mobilewalla, Inc. 2018
ML AlgorithmTraining Data

14. Component 1: Training Data © Mobilewalla, Inc. 2018
Breadth
Depth
Training Data

15. Component 1: Training Data © Mobilewalla, Inc. 2018
Length of History
How many accurately classified
emails are included in the training
set?
What is the length of history over
which we have selected emails to
include in the training set?
Depth
Training Data
Volume

16. Component 1: Training Data © Mobilewalla, Inc. 2018
Length of History
How many accurately classified
emails are included in the training
set?
What is the length of history over
which we have selected emails to
include in the training set?
Depth
Training Data
Volume
Breadth Attribute Set
What specific attributes
regarding emails are part of the
training set?

17. Component 1: Training Data © Mobilewalla, Inc. 2018
Length of History
How many accurately classified
emails are included in the training
set?
What is the length of history over
which we have selected emails to
include in the training set?
Depth
Training Data
Volume
Breadth Attribute Set
What specific attributes
regarding emails are part of the
training set?
Features
1. Originating Country
2. Number of Words
3. Presence of Key-phrases

18. Component 2: ML Algorithm © Mobilewalla, Inc. 2018
ML AlgorithmTraining Data

19. Component 2: ML Algorithm © Mobilewalla, Inc. 2018
Linear Regression
Logistic Regression
Naive Bayes
K-Nearest Neighbours
Support Vector machines (linear kernel)
Random Forest
Gradient Boosting Tree
Reinforcement learning
ML Algorithm

20. Two Core Components: Data and ML Algorithm © Mobilewalla, Inc. 2018
ML AlgorithmTraining Data
Depth
Breadth
(Features)

21. The
question
is…
© Mobilewalla, Inc. 2018
?

22. What is more contributive
towards outcome?

23. What is more contributive
towards outcome?

24. What is more contributive
towards outcome?

25. The current AI battles
are completely fought
over techniques
Sophisticated
techniques lead to
better outcomes – this
is the common wisdom ARTIFICIAL
INTELLIGENCE
Technology Landscape

26. The current AI battles
are completely fought
over techniques
Sophisticated
techniques lead to
better outcomes – this
is the common wisdom ARTIFICIAL
INTELLIGENCE
Technology Landscape
AUTONOMOUS
SYSTEMS
MACHINE
LEARNING
DEEP LEARNING
NEURAL NETWORKS
PATTERN
RECOGNITION
NATURAL
LANGUAGE PROCESSING
CHAT BOTS
REAL TIME EMOTION ANALYTICS
VIRTUAL COMPANIONS
REAL TIME UNIVERSAL
TRANSLATION
THOUGHT
CONTROLLED
GAMING
NEXT GEN
CLOUD
ROBOTICS
AUTONOMOUS
SURGICAL
ROBOTICS
ROBOTIC
PERSONAL
ASSISTANTS
COGNITIVE CYBER
SECURITY
NEUROMORPHIC
COMPUTING

27. The current AI battles
are completely fought
over techniques
Sophisticated
techniques lead to
better outcomes – this
is the common wisdom ARTIFICIAL
INTELLIGENCE
Technology Landscape
AUTONOMOUS
SYSTEMS
MACHINE
LEARNING
DEEP LEARNING
NEURAL NETWORKS
PATTERN
RECOGNITION
NATURAL
LANGUAGE PROCESSING
CHAT BOTS
REAL TIME EMOTION ANALYTICS
VIRTUAL COMPANIONS
REAL TIME UNIVERSAL
TRANSLATION
THOUGHT
CONTROLLED
GAMING
NEXT GEN
CLOUD
ROBOTICS
AUTONOMOUS
SURGICAL
ROBOTICS
ROBOTIC
PERSONAL
ASSISTANTS
COGNITIVE CYBER
SECURITY
NEUROMORPHIC
COMPUTING
Nothing could
be further
from the
truth

28. Real World Use Cases
© Mobilewalla, Inc. 2018

29. Identifying and Building Portraits of
High-Value Customers (HVC) © Mobilewalla, Inc. 2018
Most businesses get 80% of their revenues from 20% of their customers.
These are their High-Value Customers that they want to retain, acquire and grow.
RETAIN
I must retain my
existing HVC.
ACQUIRE
I wish to acquire new
customers who are likely
to be HVC.
GROW
How do I increase the
ARPU of my HVC?
80%
20%
High-Value
CUSTOMERS
80%
20%
REVENUE

30. © Mobilewalla, Inc. 2018
Use Case # 1: HVC for Food Delivery Company
Training Input Features
Algorithm HVC
Portrait
Benefits

31. © Mobilewalla, Inc. 2018
Use Case # 1: HVC for Food Delivery Company
Training Input
Customer ID Cuisine Type Date Order Amount
1 Tandoori 08 /15/2018 $53.80
2 Pizza 08/12/2018 $27.86
3 Chinese O7/23/2018 $29.50
Customer ID Demographic
Attributes
Home
Location
Work Location Restaurant
Visit
Frequency
Supermarket
Visit
Frequency
1 Male, Single,
30-45 years
old
Paya Lebar,
Singapore
Raffles City 3x per week 1x per week
2 Male, Single,
18-34 years
old
Jurong East,
Singapore
Anson Road 4x per week 1x per week
3 Female,
Married, 25-
35 years old
Balestier,
Singapore
Marina Bay 2x per week 2x per week

32. © Mobilewalla, Inc. 2018
Training Input Features
Algorithm HVC
Portrait
Benefits
Data Data
Use Case # 1: HVC for Food Delivery Company
Demographic Attributes Both
Home Location Mobilewalla
Work Location Mobilewalla
Restaurant Visit Frequency Mobilewalla
Supermarket Visit Frequency Mobilewalla
Order Amt. Distribution DeliverX
Order Freq. Distribution DeliverX

33. © Mobilewalla, Inc. 2018
Training Input Features
Algorithm HVC
Portrait
Benefits
Data Data
Use Case # 1: HVC for Food Delivery Company
Demographic Attributes Both
Home Location Mobilewalla
Work Location Mobilewalla
Restaurant Visit Frequency Mobilewalla
Supermarket Visit Frequency Mobilewalla
Order Amt. Distribution DeliverX
Order Freq. Distribution DeliverX
• Logistic Regression (Simple)
• Gradient Boosting (Sophisticated)

34. © Mobilewalla, Inc. 2018
Training Input Features
Algorithm HVC
Portrait
Benefits
Data Data
Use Case # 1: HVC for Food Delivery Company
Demographic Attributes Both
Home Location Mobilewalla
Work Location Mobilewalla
Restaurant Visit Frequency Mobilewalla
Supermarket Visit Frequency Mobilewalla
Order Amt. Distribution DeliverX
Order Freq. Distribution DeliverX
• Logistic Regression (Simple)
• Gradient Boosting (Sophisticated)
• Married 25-34
• Has kids
• Working Spouse
• Home to Work Commute Distance > 15
KM

35. © Mobilewalla, Inc. 2018
Training Input Features
Algorithm HVC
Portrait
Benefits
Data Data
Use Case # 1: HVC for Food Delivery Company
Demographic Attributes Both
Home Location Mobilewalla
Work Location Mobilewalla
Restaurant Visit Frequency Mobilewalla
Supermarket Visit Frequency Mobilewalla
Order Amt. Distribution DeliverX
Order Freq. Distribution DeliverX
• Logistic Regression (Simple)
• Gradient Boosting (Sophisticated)
• Married 25-34
• Has kids
• Working Spouse
• Home to Work Commute Distance > 15
KM
• Increased Avg. Rev. per customer
• Increased Avg. Order Amt. per customer
• Increased Avg. LTV
• Increase in Return on Acquisition spend

36. © Mobilewalla, Inc. 2018
Quality of Outcome: Data (Depth) vs. Algorithm
PredictionQuality
Data Depth (Length of History)
1 Quarter 2 Quarters 3 Quarters 4 Quarters
0
1
0.7
2X
1.3X
1.1X 1.05X
Logistic Regression (Simple)
Gradient Boosting (Sophisticated)

37. © Mobilewalla, Inc. 2018
Quality of Outcome: Data (Depth) vs. Algorithm
PredictionQuality
Data Depth (Length of History)
1 Quarter 2 Quarters 3 Quarters 4 Quarters
0
1
0.7
2X
1.3X
1.1X 1.05X
Logistic Regression (Simple)
Gradient Boosting (Sophisticated)
✓ The deeper the data incorporated in training, the less sense it
makes to pay a premium for the technique
✓ If you haven’t incorporated deep data in training, you are likely
not getting best prediction possible

38. © Mobilewalla, Inc. 2018
Quality of Outcome : Data (Depth) vs. Algorithm
PredictionQuality
1 Quarter
0
1
1.4X
Logistic Regression (Simple)
Gradient Boosting (Sophisticated)
3 Quarters

39. © Mobilewalla, Inc. 2018
Quality of Outcome : Data (Depth) vs. Algorithm
PredictionQuality
1 Quarter
0
1
1.4X
Logistic Regression (Simple)
Gradient Boosting (Sophisticated)
3 Quarters
✓Data depth beats Algorithm hands down

40. © Mobilewalla, Inc. 2018
Quality of Outcome : Data (Breadth) vs. Algorithm
PredictionQuality
1
Only with DeliverX Data
With all Data
Full Featured
Wide Data
Feature Constrained
Narrow Data
1.5X
0
Logistic Regression (Simple)
Gradient Boosting (Sophisticated)
Demographic Attributes Both
Home Location Mobilewalla
Work Location Mobilewalla
Restaurant Visit Frequency Mobilewalla
Supermarket Visit Frequency Mobilewalla
Order Amt. Distribution DeliverX
Order Freq. Distribution DeliverX

41. © Mobilewalla, Inc. 2018
Quality of Outcome : Data (Breadth) vs. Algorithm
PredictionQuality
1
Only with DeliverX Data
With all Data
Full Featured
Wide Data
Feature Constrained
Narrow Data
1.5X
0
Logistic Regression (Simple)
Gradient Boosting (Sophisticated)
✓ Lack of breadth is very costly. Using only internal data for the project, the sophisticated technique is 50%
less effective than the simplest solution.
✓ As companies have limited breadth of 1st party customer data, enriching data sets are a must to drive
high ROI of your AI investment

42. © Mobilewalla, Inc. 2018
Depth vs. Breadth
PredictionQuality
1
2 Quarters of History
+
Full Features 4 Quarters of History
+
Only DeliverX Data
1.5X
0
4 Quarters of History
+
Only DeliverX Data
2 Quarters of History
+
Full Features
1.6X
Logistic Regression (Simple)
Gradient Boosting (Sophisticated)

43. © Mobilewalla, Inc. 2018
Depth vs. Breadth
PredictionQuality
1
2 Quarters of History
+
Full Features 4 Quarters of History
+
Only DeliverX Data
1.5X
0
4 Quarters of History
+
Only DeliverX Data
2 Quarters of History
+
Full Features
1.6X
Logistic Regression (Simple)
Gradient Boosting (Sophisticated)✓Breadth beats depth

44. © Mobilewalla, Inc. 2018Use Case # 2: Brand Propensity Modelling
Training Input
Features
Algorithm Benefits
Data
• Historical QSR Visitation Data
• Brand POI data – location of the
QSR outlets
• Collaborative filtering (Simple) - CF
• Matrix Factorization (Sophisticated) –
ALS-WR
• Precise campaign planning &
targeting
• ROI maximization
• Provides guidance regarding how
consumer characteristics, both
demographic (age, gender,
marital status etc.) and
personality-specific (e.g.,
introvert/extrovert) impact
brand preference.
Results: CF and ALS WR performed equally with deeper historical data
The Results - Recall
Comparison between a simple and a sophisticated algorithm
0%
10%
20%
30%
40%
50%
60%
70%
3 Quarters -CF 1 Quarter -ALS-WR
1.3X
0%
10%
20%
30%
40%
50%
60%
70%
CF ALS-WR CF ALS-WR CF ALS-WR
1 Quarter 1 Quarter 2 Quarters 2 Quarters 3 Quarters 3 Quarters
1.42X
1.03X
1.2X

45. © Mobilewalla, Inc. 2018Use Case # 3: Gender Prediction
• Activity based features (1)
Device engagement, app usage, active days,
device age
• Environment based features (2)
Device type, internet connectivity, device
mobility
• App engagement based features (3)
Category of interest, demographic affinity
• Logistic Regression (Simple)
• Random forest (Sophisticated)
• Precise campaign planning &
targeting
• Personalized Messaging
• ROI maximization
Results: Logistic Regression and Random forest performed equally with
deeper historical data (3 quarters)
The Results – Nielsen Digital Ad Ratings
Comparison between a simple and a sophisticated algorithm
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Logistic
Regression
Random forest Logistic
Regression
Random forest Logistic
Regression
Random forest
1 Quarter 1 Quarter 2 Quarter 2 Quarter 3 Quarters 3 Quarters
1.22X
1.12X
1.01X
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Logistic
Regression
Logistic
Regression
Random forest Random forest
1 Quarters of
History + Full
features (1,2 & 3)
3 Quarters of
History + features
(1)
1 Quarters of
History + Full
features (1,2 & 3)
3 Quarters of
History + features
(1)
1.3X
1.22X
Breadth is
important than
history
Training Input
Algorithm Benefits
Data Features

46. Parting
Thoughts
© Mobilewalla, Inc. 2018

47. Parting Thoughts © Mobilewalla, Inc. 2018
If you are using AI/ML, are you paying attention to how your techniques
are trained?
Are you aware of the depth and breadth of first-party customer data in
your organization?
Because organizations have limited visibility into their customers, our experience is that most
first-party data is significantly breadth-constrained, and therefore depth-challenged as well.
Layering 3rd party data on your first-party information can result in meaningful increases in
quality of training data (depth and breadth), and can yield greatly increased return on your
investments in AI/ML.

48. 48
Consumer Intelligence Solutions
SCALE
3+ years
of consumer
behavior data
1.5 B+
unique devices
30+
countries
Data from 75K
Mobile Apps
IDENTITY
Persistent Key
Online Behaviors
Offline Behaviors
Cross-Channel
INSIGHTS
SOLUTIONS
Data Enrichment
Analytics
Audience Segments
Identity
Demographic and Behavioral
Geographic and Location
Competitive
TV Attribution

49. © Mobilewalla, Inc. 2018