Machine Learning in Big Data

Machine Learning in Big Data
- Look forward or be left behind
V. William Porto
Hadoop Summit June 2015

2  RedPoint Global Inc. 2015 Confidential
Machine Learning – keeping ahead of the curve
Three basic tenants for success in today’s world
Prediction - you need to learn and use what you’ve learned
Optimization - the world is a dynamic place
Automation - because people don’t scale well

Machine Learning – why bother?
If you have always done it that way, it is probably wrong” - Charles Kettering

Machine Learning – what really is it all about?
Learning vs. instruction
Humans learn instinctively – computers not so much
Intelligent Systems
Memory
Prediction (modeling)
Assessment
Feedback
Adaptation

Data Modeling – what, why, how
Regression – what happened in the past
Prediction – what will happen in the future
“Prediction is very difficult – especially if it’s about the future”
- Nihls Bohr

Data Modeling – what, why, how
Choices, choices - the wide world of data modeling
Supervised models
you have historical data and known correlated outputs (truth)
Unsupervised models
historical data, but may not have (or trust) associated outputs

Supervised vs. Unsupervised Models

Linear Models
Major Assumption: the world is linear
Pros:
the math is easy!
fast execution
Cons:
the real world isn’t really linear
all errors aren’t all equal
easy to generate misleading results

Decision Trees
Major Assumption: the world is discrete
Pros:
easy to understand
fast execution
no linearity assumptions
Cons:
lots of ‘human time’ to create
bias in unbalanced trees
some concepts need very large trees

Non-Linear Models
Major Assumption: data is representative
Pros:
‘universal’ modeling tools
fast execution
no linearity assumptions
Cons:
lots of parameters, many techniques
training can be slow
difficult to explain and understand
Artificial Neural Network
Bayesian Network

Clustering/Segmentation
Basic Question – which one describes the data the best?
Raw data

Clustering/Segmentation – group think
Collaborative Filtering
Relationship Matrix

Clustering/Segmentation with Statistics
Statistical Techniques:
K-Means
Vector Quantization
Pros:
relatively simple
statistically-backed results
Cons:
assumptions: data distribution
how many clusters really are there?
K-Means Clustering
Vector Quantization

Clustering/Segmentation – data driven
Feature Maps:
Pros:
lets data speaks for itself
useful boundary relationships
Cons:
slow to train
Customer Demographics

Model Selection – how to choose?
Basic Model Type (prediction or segmentation)
inputs + correlated outputs
inputs only?
Basic Questions:
which one to use for my problem?
parameters?
is this the best choice?
could I do better, and how?

Optimization – making the best choices
Standard (old-school) Techniques:
PCA, Partial Least Squares, etc.
Pros:
because the math is easy !
Cons:
lots of (usually incorrect) assumptions
new data = start from scratch

Optimization – is that the only way?

Optimization – Evolving better solutions
Simulated Evolution
Pros:
fast, efficient search
always have a solution
arbitrary ‘evaluation’ functions
can start with existing solution(s)
Cons:
CPU time + memory – but that’s why we have distributed processing!

Optimization – Evolving Models
What does a ‘solution’ look like?
model type
parameters
data (training + testing)
Variation – alter model type, parameters
Assessment – how well does the model work?
Selection – survival of the fittest

Evolutionary Optimization in a Hadoop Environment
Challenges:
data partitioning
distributed computation
communication
MapReduce

Optimization in a Hadoop Environment – what really works
MapReduce:
algorithmic task partitioning
iterative tasks vs. fully compartmented tasks
aggregation – distribution tasks
communication / synchronization costs

ML in a Hadoop Environment – Single Algorithm Architecture
Multi-Core Machine (per Chu and Kim, et. al 2006, Stanford NLPG)
ML Algorithm
Engine
Master
Mapper Mapper Mapper Mapper
Data Reducer
input
reduce
query info
result
query info
map (split data)intermediate data

Machine Learning in a Hadoop Environment
ML Algorithms:
Locally Weighted Linear Regression
K-Means Nearest Neighbor (KNN)
Feed-forward Multi-layer Neural Network (MLP)
Principal Component Analysis (PCA)
Support Vector Machine (SVM)

Machine Learning in a Hadoop Environment – example
Hadoop Multi-Core Tests (per Chu and Kim, et. al 2006, Stanford NLPG)
# Processors
Speed
increase

ML in a Hadoop Environment – Evolutionary Optimization Architecture
Offspring Partition
Offspring Partition Map
Initial (seed)
Population
Coordinator
Map
...
...
Offspring Partition
Master
(Variation)
Reducer
Reducer
...
1st reduction
stage
(local selection)
2nd reduction
stage
(global selection)
Reducer
Nth
generation
solutions
map stage
(evaluation)

Machine Learning – Hadoop, MPI, GPU?
query info
Analyze the algorithmic bottlenecks
Use Hadoop / MapReduce if:
large number of features
relatively few inter-process communication steps
e.g., on-line training
Use MPI, GPUs if:
large number of training samples
e.g., batch training

Optimization – Don’t Stop Now
Adaptation
update models regularly
drop old data, retrain
Model with different time scales
daily, weekly, seasonal, yearly, multi-year
Automate the process !

A Word about RedPoint Global
Launched 2006
Founded and staffed by industry
veterans
Headquarters: Wellesley,
Massachusetts
Offices in US, UK, Australia, Philippines
Global customer base
Serves most major industries
MAGIC QUADRANT
Data Quality
MAGIC QUADRANT
Multichannel Campaign
Management
MAGIC QUADRANT
Integrated Marketing
Management

Time for Q&A
For more information contact:
Bill Porto
RedPoint Global Inc.
36 Washington St., Suite 120
Wellesley Hills, MA 02481
vwporto@redpoint.net

Machine Learning in Big Data

More Related Content

What's hot

Viewers also liked

Similar to Machine Learning in Big Data

More from DataWorks Summit

Recently uploaded

Machine Learning in Big Data