1.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
Piloting Big Data
Where to Start?
29 May 2014 – StampedeCon – St. Louis
John Akred (@BigDataAnalysis),
www.svds.com @SVDataScience

2.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
2
Solving
difficult
problems
with
technology,
data,
and
science
Cross-­‐func<onal
teams
Agile
delivery
methods
Business-­‐driven
technology
strategy
and
advisory

3.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
3
1 Why big data?
2 What is a pilot?
3 Choosing a use case
4 Defining success
Doing a Big Data Pilot
Fielding a team
Delivering

4.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
http://svds.com/post/
successful-data-teams-are-agile-and-cross-functional
4

5.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
Why Big Data?
5
1. New Capabilities
2. Economic Scalability

6.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
6
DATA PLATFORMS
FOR
NEW CAPABILITES

7.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
7
THE DATA VALUE CHAIN
Acquire Ingest Process Persist Integrate Analyze Expose

8.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
8
DATA
PLATFORMS
FOR
ECONOMIC
SCALABILITY
at NetApp

9.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
9
UP OR OUT? The SaaS Edition
Users
Revenue
scale-out
cost
good times bummer
Different products and
features put different
demands on the data
infrastructure
• Profitable
• Unprofitable
Increasing cost per user from
scale-up architectures causes
a barrier to economic
expansion of the product user
base.

10.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
10
UP OR OUT? in the enterprise
Different use cases put different
demands on the data
infrastructure
• UC1
• UC2
• UC3
• UC4
• UCn
Increasing cost per unit of
capability from scale-up
architectures causes rationing of
resources. Only the most
valuable use cases are pursued.
Data Resource Usage
Value
scale-out
cost
UC 1 UC2 UC3 UC4 UCn

11.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
11
StampedeCon
1 Why big data?
2
What is a pilot?
3 Choosing a use case
4 Defining success

12.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
12
From idea to production
Agile: Iterate to value, answering the most valuable questions
as quickly as possible
Plan Prototype Pilot Production
þ
þ
þ
þ
þ

13.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
What is a Pilot?
Plan and
Initialize
• Define
architectural
approach
• Identify
resources
• Provision
training
• Choose use
case
• Define success
• Populate initial
backlog and
sprint plans
Prototype
and Prove
• Identify poorly
understood
functionality
• Isolate and
experiment
• Determine
solution
approaches
• Evaluate
solution(s)
• Correctness
• Scale
• Economics
Pilot
• Define end-to-
end “steel
thread”
• Partition off
pilot
population
• Build and
integrate
system
components
• Modify
associated
processes
• Train pilot user
team
Production
• Expand to
entire user/
customer/
partner/ etc
population
• Industrialize
monitoring
capabilities
• Re-engineer
processes
• Train user
community
13

14.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
14
1 Why big data?
2 What is a pilot?
3
Choosing a use case
4 Defining success
StampedeCon

15.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
15
STRATEGIC
IMPERATIVES
BUSINESS
OBJECTIVES
MAP OBJECTIVES TO
TECHNICAL WORKLOADS
RATIONALIZE
WORKLOADS
Strategic Workloads

16.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
16
BUILDING A
DATA
PLATFORM
External
Systems
Data
Acquisition
Internal
Data
Sources
Data Management
Security, Operations, Data Quality, Meta Data Management and Data Lineage
Analytics
Data
Ingestion
Data
Repository
External
Data
Sources
Persistence
Offline
Processing
Real Time
Processing
Batch
Processing
Data
Services

17.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
17
1 Why big data?
2 What is a pilot?
3 Choosing a use case
4
Defining success
StampedeCon

18.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
• Incremental revenue
• Time to market
• Economic functional
implementation
• Cost avoidance
• Brand benefit
• Goodwill
✔
18
Defining
Success

19.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
thank you!
19
Yes, we’re hiring
www.svds.com/join-us