More Related Content Similar to WSO2Con USA 2015: Keynote - The Future of Real-Time Analytics and IoT (20) WSO2Con USA 2015: Keynote - The Future of Real-Time Analytics and IoT1. The Future Of Analytics And IoT
It Can’t Happen Without You
Mike Gualtieri, Principal Analyst
November 3, 2015 San Francisco
Twitter: @mgualtieri
3. © 2015 Forrester Research, Inc. Reproduction Prohibited 3
52%
53%
53%
54%
58%
64%
64%
65%
66%
73%
75%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Better leverage big data and analytics in business decision-making
Create a comprehensive strategy for addressing digital technologies like mobile,
social & smart products
Create a comprehensive digital marketing strategy
Better comply with regulations and requirements
Improve differentiation in the market
Increase influence and brand reach in the market
Address rising customer expectations
Improve our ability to innovate
Reduce costs
Improve our products /services
Improve the experience of our customers
Customer experience is a top priority for
business leaders
› Base: 3,005 global data and analytics decision-makers
› Source: Global Business Technographics Data And Analytics Online Survey, 2015
4. For you For all For segments For you
Demographic
Relationships
Hyper-Personal,
Real-Time
Relationships
Personal
Relationships
Mass
Relationships
CustomerExperience
1800 1900 1950 2000 2015
7. • Learn individual customer
characteristics and
behaviors
• Detect customer needs and
desires in real-time
• Adapt applications to serve
an individual customer
Celebrity experiences must:
9. © 2015 Forrester Research, Inc. Reproduction Prohibited 9
Learn Model Detect Adapt
True BI means having four kinds of analytics
Predictive
Analytics
Streaming
Analytics
Descriptive
Analytics
(Advanced Analytics)
Prescriptive
Analytics
Batch Real-time
Most firms invest here They must invest here too
10. © 2015 Forrester Research, Inc. Reproduction Prohibited 10
Source: Forrester Research
Advanced analytics is rightly surging
“What is your firm's/business unit's current use of the following technologies?”
Source: Forrester's Global Business Technographics Data And Analytics Survey, 2015 and 2014
Base: 1805 (2015), 1063 (2014)
19%
19%
24%
31%
34%
22%
22%
35%
31%
43%
53%
54%
50%
50%
69%
39%
42%
42%
42%
42%
43%
43%
46%
48%
52%
54%
55%
56%
57%
69%
Non modeled data exploration and discovery
Search/interactive discovery
Streaming analytics
Metadata generated analytics
OLAP
Advanced visualization
Text analytics
Location analytics
Predictive analytics
Process analytics
Embedded analytics
Web analytics
Dashboards
Performance analytics
Reporting
2015
2014
Sweet!Most of your
competitors
still haven’t
started!
14. © 2015 Forrester Research, Inc. Reproduction Prohibited 14
Every industry is graced with more data
› Richer transactional data from portfolio of hundreds of
business applications
› Usage and behavior data from web and mobile apps
› Social media data
› Log data
› IoT device sensor and event data
› Data economy – firms buying and selling data
15. Using your best estimate, what is the size of
all data stored within your company?
Source: Forrester Research, September 2015
Base: 100 US Managers and above currently using Hadoop for processing and analyzing data.
Enterprises have plenty of data from both internal and
external sources
10-49
Terabytes
5%
50-99
Terabytes
12%
100-500
Terabytes
54%
Greater than
500
Terabytes
29%
Internal
business
data
49%
External
source data
51%
What % of the data available is from internal
business applications (ERP and business
applications) versus external sources
(social, IoT)?
21. How can you prevent this dude from fleecing
you right now?
22. What are movers and shakers saying about
equities that we cover right now?
23. How can you know if your baby is sleeping
soundly or if something is wrong right now?
24. How can you warn other drivers that the
road is slippery to avoid a crash right now?
25. What offers should you make to your customer if
they are within proximity of your store right now?
28. © 2015 Forrester Research, Inc. Reproduction Prohibited 28
Most firms struggle make insights actionable
Insights for lines of
business:
Source: October 16, 2014, “The Customer-Activated Enterprise” Forrester report
30. © 2015 Forrester Research, Inc. Reproduction Prohibited 30
Learn Model Detect Adapt
Build analytics into your applications
Predictive
Analytics
Streaming
Analytics
Descriptive
Analytics
(Advanced Analytics)
Prescriptive
Analytics
Batch Real-time
Most firms invest here They must invest here too
32. © 2015 Forrester Research, Inc. Reproduction Prohibited 32
Top 3 most read Forrester
research reports in Q3 2015.
#1 Does Customer Experience Really Drive
Business Success?
#2 Forrester Wave: Big Data Predictive
Analytics, Q2 2015
#3 The US Customer Experience Index,
Q12015
34. © 2015 Forrester Research, Inc. Reproduction Prohibited 34
Ways to create predictive models
› Handcraft business rules or program code to
create a predictive model based on human
experts.
› Use big data predictive analytics tools to build
models that analyze data with machine learning
algorithms to build a predictive model.
› Sometimes you need both.
35. © 2015 Forrester Research, Inc. Reproduction Prohibited 35
Predictive models can be very powerful and
profitable, but understand that:
› Predictive models are about probabilities, not absolutes
• E.g. 78% chance you will like Better Call Saul
› Accurate predictive models may not exist for every
question
• E.g. Economists have a very poor record
› Prediction models are not necessarily and probably not
causative
37. Data scientists use a
combination of statistical
and machine learning
algorithms to find
patterns and predictive
models.
38. © 2015 Forrester Research, Inc. Reproduction Prohibited 38
Are you a data scientist?
38
K-means clustering
Association rules
Boosting trees
CHAID
Cluster analysis
Feature selection
Independent components analysis
Kohonen Networks (SOFM)
Neural networks
Social network analysis (SNA)
Random forests
Mars regression splines
Linear and logistic regression
Naïve Bayesian classifiers
Optimal binning
Partial least squares
Response Optimization
Root cause analysis
Support vector machines
Natural language processing
39. © 2013 Forrester Research, Inc. Reproduction Prohibited 39
Big data is the fuel and machine learning is the
engine
› Classifiers
• Predict a specific event, characteristic, or behavior
› Recommenders
• Make a recommendation
› Clusters
• Find groups that share common characteristics
43. ANALYTICS
PRESCRIPTIVETools, techniques, and technologies that are
used to determine the next best decision or
action using a combination of methods including
business rules, descriptive/predictive/streaming
analytics, and optimization.
44. How can you influence swing voters to vote for
you?
Image source: iStockphoto
45. 45© 2015 Forrester Research, Inc. Reproduction Prohibited
You must support both analytical and
intuitive decision makers.
Analytical
Intuitive
Automated
Decision can be modeled
Information is available
Algorithm can be defined
Expert
Decision is not modeled
Unknown or fuzzy information
Experience and judgment
Image sources: Cryteria, Eurogamer.net, Business Insider, CBS Entertainment, Cinema Blend
46. 46© 2015 Forrester Research, Inc. Reproduction Prohibited
Prescriptive analytics relies on a
combination of approaches
• Business rules and/or program code based on
human expertise and experience.
• Use previously built predictive models.
• Use previously solved mathematical
optimizations.
You will likely need a combination of all of
these to make the best possible decision.
50. © 2015 Forrester Research, Inc. Reproduction Prohibited 50
Real-time means business-time
› A customer walks into a shopping mall
› A shopper clicks on an online add
› A temperature sensor spikes
› A stock price rises
› A customer uses a credit card
› A customer wakes up
51. © 2015 Forrester Research, Inc. Reproduction Prohibited 51
Thinking in streams is different…
› Ingest
› Filter
› Transform
› Normalize
› Link
› Enrich
› Correlate
› Location/motion (geofencing)
› Time windows
› Temporal pattern detection
› Business logic/rules execution
› Action interfaces
Continuous ETL Continuous Analytics
52. How can an online retailer
sell more motorcycle
helmets and optimize
profits?
› Temporal pattern detection
› Time windows
› Business logic/rules
execution
› Action interfaces
53. © 2015 Forrester Research, Inc. Reproduction Prohibited 53
Streaming analytics enables an entirely new real-
time selling model
› Analytic: When has this user
viewed at least three
motorcycle safety products
including at least one helmet?
› Action: Display most profitable
motorcycle helmets.
› Analytic: What is the real-time
daily total sales of motorcycle
helmets?
› Action: If sales trending lower
than usual, then dynamically
lower price.
Temporal Pattern Detection Time Window
54. © 2015 Forrester Research, Inc. Reproduction Prohibited 54
Try doing that with plain-old, conventional SQL
SELECT SUM(CustomerViews.view)
WHERE
CustomerViews.productCategory =
‘MCSafety’ AND …
58. Spark on Hadoop minimizes the latency of loading
HDFS data into memory, and DevOps can use
YARN to tune performance for simultaneous jobs.
59. © 2015 Forrester Research, Inc. Reproduction Prohibited 59
Spark and Hadoop can coexist in the same
cluster.
61. SELECT * FROM Hadoop;
Pure SQL for
Hadoop
Boosted SQL for
Hadoop
• Apache Hive
• Apache Drill
• Apache Phoenix
(for HBase)
• Cloudera Impala
• Presto
• Spark SQL
(through Spark)
• Actian Vortex
• HP Vertica SQL
on Hadoop
• IBM Big SQL
• JethroData
• Pivotal HAWQ
• Microsoft
Polybase
• Oracle Big Data
SQL
• Teradata
QueryGrid
Database+ SQL for
Hadoop
63. © 2015 Forrester Research, Inc. Reproduction Prohibited 63
Spark also includes a growing number of
specialized tools
65. © 2015 Forrester Research, Inc. Reproduction Prohibited 65
Learn Model Detect Adapt
Only the analytical enterprise can compete and
win in the age of celebrity
Predictive
Analytics
Streaming
Analytics
Descriptive
Analytics
(Real-time)
Prescriptive
Analytics
(Continuous Batch)
þ þ þ þ
Invest here Invest here Invest here
66. © 2015 Forrester Research, Inc. Reproduction Prohibited 66
Combine application middleware and analytics to
create industrial strength “lambda” applications
74. Embed and act on real-time analytics in
existing and new applications.
78. What kind of celebrity
customer experiences
can you create with a
great open source
platform?
79. © 2015 Forrester Research, Inc. Reproduction Prohibited 79
Think like a venture capitalist to find
opportunities in your business.
1. Walk through critical or challenging business processes
- At each step of the business process ask how analytics could
improve the process
2. Walk through customer experience to improve customer
experience
- At each step of the customer journey, ask how analytics could
help create celebrity customer experiences