Spark Driving Business Results with Real-Time Analytics

www.ovum.com
© Copyright Ovum 2015. All rights reserved.
Spark & the Enterprise
Tony Baer
tony.baer@ovum.com
Presentation for Spark Summit East 2016

2© Copyright Ovum 2015. All rights reserved.
Spark eating the Big Data world
§ 40+ committers
§ 1000 contributors
§ 179 projects using Spark
engine
§ 370k+ LOC
Source: Databricks, January 2015
The most active Apache project

“The leading candidate for
‘successor’ to MapReduce today is
Apache Spark.”
Mike Olson
Chief Strategy Officer, Cloudera 12/30/2013
Bob Picciano
SVP, Analytics, IBM 6/15/2015
When IBM put its muscle behind Linux in 1999, that move
marked the beginning of its ascendancy in corporations and
Internet-class data centers. The same sort of thing could
happen now with Spark.

What’s there to like?
Ease of development
Performance
FlexibilityExtensibility
Versatility
10 – 100x faster than
MapReduce
Batch + Real timeCore projects & 80+ libraries
Orchestrate multiple
analytic processes
Higher level programming abstraction

So what?

What’s really there to like?
Ease of development
Performance
Versatility
MapReduce
analytic processes
Batch + Real time
Handle different &
complex scenarios
Better Productivity
Wider tool
selection
Smarter
predictions
Handle more varied
scenarios
Better programming
Core projects & 80+ libraries

Ease of development
Performance
VersatilityExtensibility
Versatility
MapReduce
analytic processes
Batch + Real time
Handle more complex
scenarios
Better Productivity
Wider tool
selection
Smarter
predictions
Handle more varied &
complex scenarios
Better programming

Ease of development
Performance
VersatilityExtensibility
Versatility
MapReduce
analytic processes
Batch + Real time
Handle more complex
scenarios
Better Productivity
Wider tool
selection
Smarter
predictions
Handle more varied &
complex scenarios
Better programming
So what?

How will this impact
the business?

Focus on the results
§ What use cases/business
scenarios/business problems can Spark
address?
§ How does Spark impact analytics?
§ What questions are asked?
§ How questions are asked?
§ Types of analytics that are performed?
§ Timeliness of results?
§ The insights that can be obtained?

Common analytics use cases
Workload shift
Customer
Engagement
Risk/Fraud/Security
Operations
Customer Retention
Customer Experience
Upsell/Cross-Sell
Social Tribe Influence
Real-time Customer Offer
Risk Mitigation
Fraud Detection/Prevention
Intrusion Detection
Operational Efficiency
Process Optimization
Asset & Service Mgmt.
Performance Mgmt.
ETL processes
Batch analytics
Many use cases are familiar… the results are different

Smart City:
Manage Traffic flow
From Sense & respond to….
Real-time analytics + interactive query + long running ML =
better insights for managing traffic

Monitoring Automotive product performance
From:
§ Track warranty & repair trends (after the
fact)
To:
§ Identify signals from social media to
prepare auto mfr & dealer network to
anticipate performance issues
§ Use Spark MLlib machine learning
capabilities
Benefits:
§ Provided advance warning of customer
feedback
§ MLlib libraries eliminated need for
custom programming ML functions Source: Toyota 12-week pilot program

Data wrangling to spot financial fraud
From:
§ DW populated with data from internal sources
(mostly OLTP data)
To:
§ Broadening data set to widely varying sources
(transactions, text messages, social media) with
10s or 100s of millions of records
§ Use Spark-based ML-powered data prep tool to
harmonize data to ID outliers & patterns
Benefits:
§ Spark performance enabled team to expand
data pool, query interactively & run more what-if
scenarios for spotting fraud

Customer Experience (CX) Management
From:
§ Surveys, focus groups, CRM data
To:
§ Predictive analytics for improving the
customer experience
§ Spark-enabled machine learning for
identifying CX trends, customer
satisfaction levels; Graph analytics for
connecting customer experiences
across different channels and ID’ing
influencers & followers
Benefits:
§ Changes CX management from
reactive to proactive

Why Spark?
From the tech argument
Ease of development
Performance
Versatility
MapReduce
Batch + Real timeCore projects & 80+ libraries
analytic processes

Why Spark?
To: Business Benefits
§ Automotive product performance
§ Machine learning enables the automotive OEM to be proactive in deciphering the
signals to anticipate consumer sentiment/perceptions of product performance
§ Business Benefit: Head off product complaints/potential liability/reputational issues
before they explode
§ Financial fraud detection
§ Spark’s scalability allows crunching of more complete data sets; performance
produces more timely results; machine learning IDs emergent outliers of interest
§ Business Benefit: More thorough, timely detection of fraud
§ Customer Experience
§ Machine learning allows proactive deciphering of signals; graph computing
identifies social tribes & influencers
§ Business Benefit: Keep more in sync with customers. Act, not react to events,
trends, changes in customer climate

Takeaways
§ Spark enthusiasm in practitioner community has gone
viral
§ Spark community highly successfulin sparking vendor
support.
§ Spark practitioners must take the message on Spark to
higher level: Talk to the business
§ Keep your message real:
§ Business benefits
§ Don’t promise the sky
§ Spark is not the only path to ML, graph, streaming, etc.
But API compatibility provides accessibility, enables
flexibility & versatility
§ Spark is still in adolescence.

Spark Driving Business Results with Real-Time Analytics

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (9)

Similar to Spark Driving Business Results with Real-Time Analytics

Similar to Spark Driving Business Results with Real-Time Analytics (20)

More from Spark Summit

More from Spark Summit (20)

Recently uploaded

Recently uploaded (20)

Spark Driving Business Results with Real-Time Analytics