SlideShare a Scribd company logo
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
Piloting Big Data
Where to Start?
29 May 2014 – StampedeCon – St. Louis
John Akred (@BigDataAnalysis),
www.svds.com @SVDataScience
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
2
Solving	
  difficult	
  problems	
  with	
  
technology,	
  data,	
  and	
  science	
  
Cross-­‐func<onal	
  teams	
  
Agile	
  delivery	
  methods	
  
Business-­‐driven	
  technology	
  
strategy	
  and	
  advisory	
  
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
3
1 Why big data?
2 What is a pilot?
3 Choosing a use case
4 Defining success
Doing a Big Data Pilot
Fielding a team
Delivering
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
http://svds.com/post/
successful-data-teams-are-agile-and-cross-functional
4
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
Why Big Data?
5
1. New Capabilities
2. Economic Scalability
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
6
DATA PLATFORMS
FOR
NEW CAPABILITES
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
7
THE DATA VALUE CHAIN
Acquire Ingest Process Persist Integrate Analyze Expose
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
8
DATA
PLATFORMS
FOR
ECONOMIC
SCALABILITY
at NetApp
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
9
UP OR OUT? The SaaS Edition
Users
Revenue
scale-out
cost
good times bummer
Different products and
features put different
demands on the data
infrastructure
•  Profitable
•  Unprofitable
Increasing cost per user from
scale-up architectures causes
a barrier to economic
expansion of the product user
base.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
10
UP OR OUT? in the enterprise
Different use cases put different
demands on the data
infrastructure
•  UC1
•  UC2
•  UC3
•  UC4
•  UCn
Increasing cost per unit of
capability from scale-up
architectures causes rationing of
resources. Only the most
valuable use cases are pursued.
Data Resource Usage
Value
scale-out
cost
UC 1 UC2 UC3 UC4 UCn
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
11
StampedeCon
1 Why big data?
2
What is a pilot?
3 Choosing a use case
4 Defining success
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
12
From idea to production
Agile: Iterate to value, answering the most valuable questions
as quickly as possible
Plan Prototype Pilot Production
þ
þ
þ
þ
þ
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
What is a Pilot?
Plan and
Initialize
• Define
architectural
approach
• Identify
resources
• Provision
training
• Choose use
case
• Define success
• Populate initial
backlog and
sprint plans
Prototype
and Prove
• Identify poorly
understood
functionality
• Isolate and
experiment
• Determine
solution
approaches
• Evaluate
solution(s)
• Correctness
• Scale
• Economics
Pilot
• Define end-to-
end “steel
thread”
• Partition off
pilot
population
• Build and
integrate
system
components
• Modify
associated
processes
• Train pilot user
team
Production
• Expand to
entire user/
customer/
partner/ etc
population
• Industrialize
monitoring
capabilities
• Re-engineer
processes
• Train user
community
13
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
14
1 Why big data?
2 What is a pilot?
3
Choosing a use case
4 Defining success
StampedeCon
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
15
STRATEGIC
IMPERATIVES
BUSINESS
OBJECTIVES
MAP OBJECTIVES TO
TECHNICAL WORKLOADS
RATIONALIZE
WORKLOADS
Strategic Workloads
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
16
BUILDING A
DATA
PLATFORM
External
Systems
Data
Acquisition
Internal
Data
Sources
Data Management
Security, Operations, Data Quality, Meta Data Management and Data Lineage
Analytics
Data
Ingestion
Data
Repository
External
Data
Sources
Persistence
Offline
Processing
Real Time
Processing
Batch
Processing
Data
Services
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
17
1 Why big data?
2 What is a pilot?
3 Choosing a use case
4
Defining success
StampedeCon
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
•  Incremental revenue
•  Time to market
•  Economic functional
implementation
•  Cost avoidance
•  Brand benefit
•  Goodwill
✔
18
Defining
Success
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
thank you!
19
Yes, we’re hiring
www.svds.com/join-us

More Related Content

Similar to Piloting Big Data: Where To Start? - StampedeCon 2014

TC028SN_Spencer_FINAL
TC028SN_Spencer_FINALTC028SN_Spencer_FINAL
TC028SN_Spencer_FINAL
Terri Spencer
 
Leverage Data Strategy as a Catalyst for Innovation
Leverage Data Strategy as a Catalyst for InnovationLeverage Data Strategy as a Catalyst for Innovation
Leverage Data Strategy as a Catalyst for Innovation
Glorium Tech
 
ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj
ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna SelvarajANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj
ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj
AgileNetwork
 
Kythera BioPharma Commercial Infrastructure 2015 05 28 final
Kythera BioPharma Commercial Infrastructure 2015 05 28 finalKythera BioPharma Commercial Infrastructure 2015 05 28 final
Kythera BioPharma Commercial Infrastructure 2015 05 28 final
Michael W. Hughes
 
The Future of Data Management: The Enterprise Data Hub
The Future of Data Management: The Enterprise Data HubThe Future of Data Management: The Enterprise Data Hub
The Future of Data Management: The Enterprise Data Hub
Cloudera, Inc.
 
Integrating Project Management with Service Management Best Practices Event B...
Integrating Project Management with Service Management Best Practices Event B...Integrating Project Management with Service Management Best Practices Event B...
Integrating Project Management with Service Management Best Practices Event B...
Google
 
Software is Eating the World, And You're For Lunch"
Software is Eating the World, And You're For Lunch"Software is Eating the World, And You're For Lunch"
Software is Eating the World, And You're For Lunch"
Extract Data Conference
 
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICSBig Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
Matt Stubbs
 
Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...
Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...
Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...
StampedeCon
 
CSC - Presentation at Hortonworks Booth - Strata 2014
CSC - Presentation at Hortonworks Booth - Strata 2014CSC - Presentation at Hortonworks Booth - Strata 2014
CSC - Presentation at Hortonworks Booth - Strata 2014
Hortonworks
 
Event Sponsor ScienceLogic - CTO Antonio Piraino
Event Sponsor ScienceLogic - CTO Antonio Piraino Event Sponsor ScienceLogic - CTO Antonio Piraino
Event Sponsor ScienceLogic - CTO Antonio Piraino
Hostway|HOSTING
 
Big Data: Myths and Realities
Big Data: Myths and RealitiesBig Data: Myths and Realities
Big Data: Myths and Realities
Toronto-Oracle-Users-Group
 
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web ServicesDickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Precisely
 
Analytics Rising: Plan for Success
Analytics Rising: Plan for SuccessAnalytics Rising: Plan for Success
Analytics Rising: Plan for Success
Lewandog, Inc,
 
Creating your Center of Excellence (CoE) for data driven use cases
Creating your Center of Excellence (CoE) for data driven use casesCreating your Center of Excellence (CoE) for data driven use cases
Creating your Center of Excellence (CoE) for data driven use cases
Frank Vullers
 
Data Science in the Enterprise
Data Science in the EnterpriseData Science in the Enterprise
Data Science in the Enterprise
The Hive
 
Project Management at Cisco Ben Rus TIM 101 Seminar – .docx
Project Management at Cisco Ben Rus TIM 101 Seminar – .docxProject Management at Cisco Ben Rus TIM 101 Seminar – .docx
Project Management at Cisco Ben Rus TIM 101 Seminar – .docx
woodruffeloisa
 
High-Performance Analytics in the Cloud with Apache Impala
High-Performance Analytics in the Cloud with Apache ImpalaHigh-Performance Analytics in the Cloud with Apache Impala
High-Performance Analytics in the Cloud with Apache Impala
Cloudera, Inc.
 
Accelerating Data Lakes and Streams with Real-time Analytics
Accelerating Data Lakes and Streams with Real-time AnalyticsAccelerating Data Lakes and Streams with Real-time Analytics
Accelerating Data Lakes and Streams with Real-time Analytics
Arcadia Data
 
Breed data scientists_ A Presentation.pptx
Breed data scientists_ A Presentation.pptxBreed data scientists_ A Presentation.pptx
Breed data scientists_ A Presentation.pptx
GautamPopli1
 

Similar to Piloting Big Data: Where To Start? - StampedeCon 2014 (20)

TC028SN_Spencer_FINAL
TC028SN_Spencer_FINALTC028SN_Spencer_FINAL
TC028SN_Spencer_FINAL
 
Leverage Data Strategy as a Catalyst for Innovation
Leverage Data Strategy as a Catalyst for InnovationLeverage Data Strategy as a Catalyst for Innovation
Leverage Data Strategy as a Catalyst for Innovation
 
ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj
ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna SelvarajANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj
ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj
 
Kythera BioPharma Commercial Infrastructure 2015 05 28 final
Kythera BioPharma Commercial Infrastructure 2015 05 28 finalKythera BioPharma Commercial Infrastructure 2015 05 28 final
Kythera BioPharma Commercial Infrastructure 2015 05 28 final
 
The Future of Data Management: The Enterprise Data Hub
The Future of Data Management: The Enterprise Data HubThe Future of Data Management: The Enterprise Data Hub
The Future of Data Management: The Enterprise Data Hub
 
Integrating Project Management with Service Management Best Practices Event B...
Integrating Project Management with Service Management Best Practices Event B...Integrating Project Management with Service Management Best Practices Event B...
Integrating Project Management with Service Management Best Practices Event B...
 
Software is Eating the World, And You're For Lunch"
Software is Eating the World, And You're For Lunch"Software is Eating the World, And You're For Lunch"
Software is Eating the World, And You're For Lunch"
 
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICSBig Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
 
Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...
Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...
Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...
 
CSC - Presentation at Hortonworks Booth - Strata 2014
CSC - Presentation at Hortonworks Booth - Strata 2014CSC - Presentation at Hortonworks Booth - Strata 2014
CSC - Presentation at Hortonworks Booth - Strata 2014
 
Event Sponsor ScienceLogic - CTO Antonio Piraino
Event Sponsor ScienceLogic - CTO Antonio Piraino Event Sponsor ScienceLogic - CTO Antonio Piraino
Event Sponsor ScienceLogic - CTO Antonio Piraino
 
Big Data: Myths and Realities
Big Data: Myths and RealitiesBig Data: Myths and Realities
Big Data: Myths and Realities
 
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web ServicesDickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
 
Analytics Rising: Plan for Success
Analytics Rising: Plan for SuccessAnalytics Rising: Plan for Success
Analytics Rising: Plan for Success
 
Creating your Center of Excellence (CoE) for data driven use cases
Creating your Center of Excellence (CoE) for data driven use casesCreating your Center of Excellence (CoE) for data driven use cases
Creating your Center of Excellence (CoE) for data driven use cases
 
Data Science in the Enterprise
Data Science in the EnterpriseData Science in the Enterprise
Data Science in the Enterprise
 
Project Management at Cisco Ben Rus TIM 101 Seminar – .docx
Project Management at Cisco Ben Rus TIM 101 Seminar – .docxProject Management at Cisco Ben Rus TIM 101 Seminar – .docx
Project Management at Cisco Ben Rus TIM 101 Seminar – .docx
 
High-Performance Analytics in the Cloud with Apache Impala
High-Performance Analytics in the Cloud with Apache ImpalaHigh-Performance Analytics in the Cloud with Apache Impala
High-Performance Analytics in the Cloud with Apache Impala
 
Accelerating Data Lakes and Streams with Real-time Analytics
Accelerating Data Lakes and Streams with Real-time AnalyticsAccelerating Data Lakes and Streams with Real-time Analytics
Accelerating Data Lakes and Streams with Real-time Analytics
 
Breed data scientists_ A Presentation.pptx
Breed data scientists_ A Presentation.pptxBreed data scientists_ A Presentation.pptx
Breed data scientists_ A Presentation.pptx
 

More from StampedeCon

Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...
Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...
Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...
StampedeCon
 
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
StampedeCon
 
Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017
Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017
Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017
StampedeCon
 
Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...
Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...
Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...
StampedeCon
 
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
StampedeCon
 
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
StampedeCon
 
Foundations of Machine Learning - StampedeCon AI Summit 2017
Foundations of Machine Learning - StampedeCon AI Summit 2017Foundations of Machine Learning - StampedeCon AI Summit 2017
Foundations of Machine Learning - StampedeCon AI Summit 2017
StampedeCon
 
Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...
Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...
Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...
StampedeCon
 
Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...
Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...
Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...
StampedeCon
 
Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017
Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017
Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017
StampedeCon
 
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017
AI in the Enterprise: Past,  Present &  Future - StampedeCon AI Summit 2017AI in the Enterprise: Past,  Present &  Future - StampedeCon AI Summit 2017
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017
StampedeCon
 
A Different Data Science Approach - StampedeCon AI Summit 2017
A Different Data Science Approach - StampedeCon AI Summit 2017A Different Data Science Approach - StampedeCon AI Summit 2017
A Different Data Science Approach - StampedeCon AI Summit 2017
StampedeCon
 
Graph in Customer 360 - StampedeCon Big Data Conference 2017
Graph in Customer 360 - StampedeCon Big Data Conference 2017Graph in Customer 360 - StampedeCon Big Data Conference 2017
Graph in Customer 360 - StampedeCon Big Data Conference 2017
StampedeCon
 
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
StampedeCon
 
Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017
Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017
Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017
StampedeCon
 
Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...
Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...
Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...
StampedeCon
 
Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...
Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...
Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...
StampedeCon
 
Innovation in the Data Warehouse - StampedeCon 2016
Innovation in the Data Warehouse - StampedeCon 2016Innovation in the Data Warehouse - StampedeCon 2016
Innovation in the Data Warehouse - StampedeCon 2016
StampedeCon
 
Creating a Data Driven Organization - StampedeCon 2016
Creating a Data Driven Organization - StampedeCon 2016Creating a Data Driven Organization - StampedeCon 2016
Creating a Data Driven Organization - StampedeCon 2016
StampedeCon
 
Using The Internet of Things for Population Health Management - StampedeCon 2016
Using The Internet of Things for Population Health Management - StampedeCon 2016Using The Internet of Things for Population Health Management - StampedeCon 2016
Using The Internet of Things for Population Health Management - StampedeCon 2016
StampedeCon
 

More from StampedeCon (20)

Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...
Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...
Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...
 
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
 
Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017
Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017
Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017
 
Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...
Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...
Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...
 
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
 
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
 
Foundations of Machine Learning - StampedeCon AI Summit 2017
Foundations of Machine Learning - StampedeCon AI Summit 2017Foundations of Machine Learning - StampedeCon AI Summit 2017
Foundations of Machine Learning - StampedeCon AI Summit 2017
 
Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...
Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...
Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...
 
Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...
Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...
Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...
 
Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017
Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017
Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017
 
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017
AI in the Enterprise: Past,  Present &  Future - StampedeCon AI Summit 2017AI in the Enterprise: Past,  Present &  Future - StampedeCon AI Summit 2017
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017
 
A Different Data Science Approach - StampedeCon AI Summit 2017
A Different Data Science Approach - StampedeCon AI Summit 2017A Different Data Science Approach - StampedeCon AI Summit 2017
A Different Data Science Approach - StampedeCon AI Summit 2017
 
Graph in Customer 360 - StampedeCon Big Data Conference 2017
Graph in Customer 360 - StampedeCon Big Data Conference 2017Graph in Customer 360 - StampedeCon Big Data Conference 2017
Graph in Customer 360 - StampedeCon Big Data Conference 2017
 
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
 
Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017
Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017
Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017
 
Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...
Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...
Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...
 
Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...
Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...
Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...
 
Innovation in the Data Warehouse - StampedeCon 2016
Innovation in the Data Warehouse - StampedeCon 2016Innovation in the Data Warehouse - StampedeCon 2016
Innovation in the Data Warehouse - StampedeCon 2016
 
Creating a Data Driven Organization - StampedeCon 2016
Creating a Data Driven Organization - StampedeCon 2016Creating a Data Driven Organization - StampedeCon 2016
Creating a Data Driven Organization - StampedeCon 2016
 
Using The Internet of Things for Population Health Management - StampedeCon 2016
Using The Internet of Things for Population Health Management - StampedeCon 2016Using The Internet of Things for Population Health Management - StampedeCon 2016
Using The Internet of Things for Population Health Management - StampedeCon 2016
 

Recently uploaded

Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
saastr
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
LucaBarbaro3
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Tatiana Kojar
 
Operating System Used by Users in day-to-day life.pptx
Operating System Used by Users in day-to-day life.pptxOperating System Used by Users in day-to-day life.pptx
Operating System Used by Users in day-to-day life.pptx
Pravash Chandra Das
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Jeffrey Haguewood
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
Dinusha Kumarasiri
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 

Recently uploaded (20)

Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
 
Operating System Used by Users in day-to-day life.pptx
Operating System Used by Users in day-to-day life.pptxOperating System Used by Users in day-to-day life.pptx
Operating System Used by Users in day-to-day life.pptx
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 

Piloting Big Data: Where To Start? - StampedeCon 2014

  • 1. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience Piloting Big Data Where to Start? 29 May 2014 – StampedeCon – St. Louis John Akred (@BigDataAnalysis), www.svds.com @SVDataScience
  • 2. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 2 Solving  difficult  problems  with   technology,  data,  and  science   Cross-­‐func<onal  teams   Agile  delivery  methods   Business-­‐driven  technology   strategy  and  advisory  
  • 3. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 3 1 Why big data? 2 What is a pilot? 3 Choosing a use case 4 Defining success Doing a Big Data Pilot Fielding a team Delivering
  • 4. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience http://svds.com/post/ successful-data-teams-are-agile-and-cross-functional 4
  • 5. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience Why Big Data? 5 1. New Capabilities 2. Economic Scalability
  • 6. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 6 DATA PLATFORMS FOR NEW CAPABILITES
  • 7. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 7 THE DATA VALUE CHAIN Acquire Ingest Process Persist Integrate Analyze Expose
  • 8. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 8 DATA PLATFORMS FOR ECONOMIC SCALABILITY at NetApp
  • 9. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 9 UP OR OUT? The SaaS Edition Users Revenue scale-out cost good times bummer Different products and features put different demands on the data infrastructure •  Profitable •  Unprofitable Increasing cost per user from scale-up architectures causes a barrier to economic expansion of the product user base.
  • 10. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 10 UP OR OUT? in the enterprise Different use cases put different demands on the data infrastructure •  UC1 •  UC2 •  UC3 •  UC4 •  UCn Increasing cost per unit of capability from scale-up architectures causes rationing of resources. Only the most valuable use cases are pursued. Data Resource Usage Value scale-out cost UC 1 UC2 UC3 UC4 UCn
  • 11. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 11 StampedeCon 1 Why big data? 2 What is a pilot? 3 Choosing a use case 4 Defining success
  • 12. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 12 From idea to production Agile: Iterate to value, answering the most valuable questions as quickly as possible Plan Prototype Pilot Production þ þ þ þ þ
  • 13. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience What is a Pilot? Plan and Initialize • Define architectural approach • Identify resources • Provision training • Choose use case • Define success • Populate initial backlog and sprint plans Prototype and Prove • Identify poorly understood functionality • Isolate and experiment • Determine solution approaches • Evaluate solution(s) • Correctness • Scale • Economics Pilot • Define end-to- end “steel thread” • Partition off pilot population • Build and integrate system components • Modify associated processes • Train pilot user team Production • Expand to entire user/ customer/ partner/ etc population • Industrialize monitoring capabilities • Re-engineer processes • Train user community 13
  • 14. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 14 1 Why big data? 2 What is a pilot? 3 Choosing a use case 4 Defining success StampedeCon
  • 15. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 15 STRATEGIC IMPERATIVES BUSINESS OBJECTIVES MAP OBJECTIVES TO TECHNICAL WORKLOADS RATIONALIZE WORKLOADS Strategic Workloads
  • 16. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 16 BUILDING A DATA PLATFORM External Systems Data Acquisition Internal Data Sources Data Management Security, Operations, Data Quality, Meta Data Management and Data Lineage Analytics Data Ingestion Data Repository External Data Sources Persistence Offline Processing Real Time Processing Batch Processing Data Services
  • 17. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 17 1 Why big data? 2 What is a pilot? 3 Choosing a use case 4 Defining success StampedeCon
  • 18. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience •  Incremental revenue •  Time to market •  Economic functional implementation •  Cost avoidance •  Brand benefit •  Goodwill ✔ 18 Defining Success
  • 19. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience thank you! 19 Yes, we’re hiring www.svds.com/join-us