SlideShare a Scribd company logo
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
Piloting Big Data
Where to Start?
29 May 2014 – StampedeCon – St. Louis
John Akred (@BigDataAnalysis),
www.svds.com @SVDataScience
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
2
Solving	
  difficult	
  problems	
  with	
  
technology,	
  data,	
  and	
  science	
  
Cross-­‐func<onal	
  teams	
  
Agile	
  delivery	
  methods	
  
Business-­‐driven	
  technology	
  
strategy	
  and	
  advisory	
  
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
3
1 Why big data?
2 What is a pilot?
3 Choosing a use case
4 Defining success
Doing a Big Data Pilot
Fielding a team
Delivering
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
http://svds.com/post/
successful-data-teams-are-agile-and-cross-functional
4
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
Why Big Data?
5
1. New Capabilities
2. Economic Scalability
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
6
DATA PLATFORMS
FOR
NEW CAPABILITES
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
7
THE DATA VALUE CHAIN
Acquire Ingest Process Persist Integrate Analyze Expose
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
8
DATA
PLATFORMS
FOR
ECONOMIC
SCALABILITY
at NetApp
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
9
UP OR OUT? The SaaS Edition
Users
Revenue
scale-out
cost
good times bummer
Different products and
features put different
demands on the data
infrastructure
•  Profitable
•  Unprofitable
Increasing cost per user from
scale-up architectures causes
a barrier to economic
expansion of the product user
base.
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
10
UP OR OUT? in the enterprise
Different use cases put different
demands on the data
infrastructure
•  UC1
•  UC2
•  UC3
•  UC4
•  UCn
Increasing cost per unit of
capability from scale-up
architectures causes rationing of
resources. Only the most
valuable use cases are pursued.
Data Resource Usage
Value
scale-out
cost
UC 1 UC2 UC3 UC4 UCn
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
11
StampedeCon
1 Why big data?
2
What is a pilot?
3 Choosing a use case
4 Defining success
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
12
From idea to production
Agile: Iterate to value, answering the most valuable questions
as quickly as possible
Plan Prototype Pilot Production
þ
þ
þ
þ
þ
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
What is a Pilot?
Plan and
Initialize
• Define
architectural
approach
• Identify
resources
• Provision
training
• Choose use
case
• Define success
• Populate initial
backlog and
sprint plans
Prototype
and Prove
• Identify poorly
understood
functionality
• Isolate and
experiment
• Determine
solution
approaches
• Evaluate
solution(s)
• Correctness
• Scale
• Economics
Pilot
• Define end-to-
end “steel
thread”
• Partition off
pilot
population
• Build and
integrate
system
components
• Modify
associated
processes
• Train pilot user
team
Production
• Expand to
entire user/
customer/
partner/ etc
population
• Industrialize
monitoring
capabilities
• Re-engineer
processes
• Train user
community
13
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
14
1 Why big data?
2 What is a pilot?
3
Choosing a use case
4 Defining success
StampedeCon
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
15
STRATEGIC
IMPERATIVES
BUSINESS
OBJECTIVES
MAP OBJECTIVES TO
TECHNICAL WORKLOADS
RATIONALIZE
WORKLOADS
Strategic Workloads
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
16
BUILDING A
DATA
PLATFORM
External
Systems
Data
Acquisition
Internal
Data
Sources
Data Management
Security, Operations, Data Quality, Meta Data Management and Data Lineage
Analytics
Data
Ingestion
Data
Repository
External
Data
Sources
Persistence
Offline
Processing
Real Time
Processing
Batch
Processing
Data
Services
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
17
1 Why big data?
2 What is a pilot?
3 Choosing a use case
4
Defining success
StampedeCon
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
•  Incremental revenue
•  Time to market
•  Economic functional
implementation
•  Cost avoidance
•  Brand benefit
•  Goodwill
✔
18
Defining
Success
© 2014 Silicon Valley Data Science LLC
All Rights Reserved.
www.svds.com @SVDataScience
thank you!
19
Yes, we’re hiring
www.svds.com/join-us

More Related Content

Similar to Piloting Big Data: Where To Start? - StampedeCon 2014

TC028SN_Spencer_FINAL
TC028SN_Spencer_FINALTC028SN_Spencer_FINAL
TC028SN_Spencer_FINAL
Terri Spencer
 
Leverage Data Strategy as a Catalyst for Innovation
Leverage Data Strategy as a Catalyst for InnovationLeverage Data Strategy as a Catalyst for Innovation
Leverage Data Strategy as a Catalyst for Innovation
Glorium Tech
 
ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj
ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna SelvarajANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj
ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj
AgileNetwork
 
Kythera BioPharma Commercial Infrastructure 2015 05 28 final
Kythera BioPharma Commercial Infrastructure 2015 05 28 finalKythera BioPharma Commercial Infrastructure 2015 05 28 final
Kythera BioPharma Commercial Infrastructure 2015 05 28 final
Michael W. Hughes
 
The Future of Data Management: The Enterprise Data Hub
The Future of Data Management: The Enterprise Data HubThe Future of Data Management: The Enterprise Data Hub
The Future of Data Management: The Enterprise Data Hub
Cloudera, Inc.
 
Integrating Project Management with Service Management Best Practices Event B...
Integrating Project Management with Service Management Best Practices Event B...Integrating Project Management with Service Management Best Practices Event B...
Integrating Project Management with Service Management Best Practices Event B...
Google
 
Software is Eating the World, And You're For Lunch"
Software is Eating the World, And You're For Lunch"Software is Eating the World, And You're For Lunch"
Software is Eating the World, And You're For Lunch"
Extract Data Conference
 
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICSBig Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
Matt Stubbs
 
Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...
Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...
Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...
StampedeCon
 
CSC - Presentation at Hortonworks Booth - Strata 2014
CSC - Presentation at Hortonworks Booth - Strata 2014CSC - Presentation at Hortonworks Booth - Strata 2014
CSC - Presentation at Hortonworks Booth - Strata 2014
Hortonworks
 
Event Sponsor ScienceLogic - CTO Antonio Piraino
Event Sponsor ScienceLogic - CTO Antonio Piraino Event Sponsor ScienceLogic - CTO Antonio Piraino
Event Sponsor ScienceLogic - CTO Antonio Piraino
Hostway|HOSTING
 
Big Data: Myths and Realities
Big Data: Myths and RealitiesBig Data: Myths and Realities
Big Data: Myths and Realities
Toronto-Oracle-Users-Group
 
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web ServicesDickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Precisely
 
Analytics Rising: Plan for Success
Analytics Rising: Plan for SuccessAnalytics Rising: Plan for Success
Analytics Rising: Plan for Success
Lewandog, Inc,
 
Creating your Center of Excellence (CoE) for data driven use cases
Creating your Center of Excellence (CoE) for data driven use casesCreating your Center of Excellence (CoE) for data driven use cases
Creating your Center of Excellence (CoE) for data driven use cases
Frank Vullers
 
Data Science in the Enterprise
Data Science in the EnterpriseData Science in the Enterprise
Data Science in the Enterprise
The Hive
 
Project Management at Cisco Ben Rus TIM 101 Seminar – .docx
Project Management at Cisco Ben Rus TIM 101 Seminar – .docxProject Management at Cisco Ben Rus TIM 101 Seminar – .docx
Project Management at Cisco Ben Rus TIM 101 Seminar – .docx
woodruffeloisa
 
High-Performance Analytics in the Cloud with Apache Impala
High-Performance Analytics in the Cloud with Apache ImpalaHigh-Performance Analytics in the Cloud with Apache Impala
High-Performance Analytics in the Cloud with Apache Impala
Cloudera, Inc.
 
Accelerating Data Lakes and Streams with Real-time Analytics
Accelerating Data Lakes and Streams with Real-time AnalyticsAccelerating Data Lakes and Streams with Real-time Analytics
Accelerating Data Lakes and Streams with Real-time Analytics
Arcadia Data
 
Breed data scientists_ A Presentation.pptx
Breed data scientists_ A Presentation.pptxBreed data scientists_ A Presentation.pptx
Breed data scientists_ A Presentation.pptx
GautamPopli1
 

Similar to Piloting Big Data: Where To Start? - StampedeCon 2014 (20)

TC028SN_Spencer_FINAL
TC028SN_Spencer_FINALTC028SN_Spencer_FINAL
TC028SN_Spencer_FINAL
 
Leverage Data Strategy as a Catalyst for Innovation
Leverage Data Strategy as a Catalyst for InnovationLeverage Data Strategy as a Catalyst for Innovation
Leverage Data Strategy as a Catalyst for Innovation
 
ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj
ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna SelvarajANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj
ANIn Coimbatore Sep 2023 | Agile for data science by Venkatesa Prasanna Selvaraj
 
Kythera BioPharma Commercial Infrastructure 2015 05 28 final
Kythera BioPharma Commercial Infrastructure 2015 05 28 finalKythera BioPharma Commercial Infrastructure 2015 05 28 final
Kythera BioPharma Commercial Infrastructure 2015 05 28 final
 
The Future of Data Management: The Enterprise Data Hub
The Future of Data Management: The Enterprise Data HubThe Future of Data Management: The Enterprise Data Hub
The Future of Data Management: The Enterprise Data Hub
 
Integrating Project Management with Service Management Best Practices Event B...
Integrating Project Management with Service Management Best Practices Event B...Integrating Project Management with Service Management Best Practices Event B...
Integrating Project Management with Service Management Best Practices Event B...
 
Software is Eating the World, And You're For Lunch"
Software is Eating the World, And You're For Lunch"Software is Eating the World, And You're For Lunch"
Software is Eating the World, And You're For Lunch"
 
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICSBig Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
Big Data LDN 2018: MICROLISE: USING BIG DATA AND AI IN TRANSPORT AND LOGISTICS
 
Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...
Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...
Beyond a Big Data Pilot: Building a Production Data Infrastructure - Stampede...
 
CSC - Presentation at Hortonworks Booth - Strata 2014
CSC - Presentation at Hortonworks Booth - Strata 2014CSC - Presentation at Hortonworks Booth - Strata 2014
CSC - Presentation at Hortonworks Booth - Strata 2014
 
Event Sponsor ScienceLogic - CTO Antonio Piraino
Event Sponsor ScienceLogic - CTO Antonio Piraino Event Sponsor ScienceLogic - CTO Antonio Piraino
Event Sponsor ScienceLogic - CTO Antonio Piraino
 
Big Data: Myths and Realities
Big Data: Myths and RealitiesBig Data: Myths and Realities
Big Data: Myths and Realities
 
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web ServicesDickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
Dickey's Barbecue Pit Heats Up Analytics with Amazon Web Services
 
Analytics Rising: Plan for Success
Analytics Rising: Plan for SuccessAnalytics Rising: Plan for Success
Analytics Rising: Plan for Success
 
Creating your Center of Excellence (CoE) for data driven use cases
Creating your Center of Excellence (CoE) for data driven use casesCreating your Center of Excellence (CoE) for data driven use cases
Creating your Center of Excellence (CoE) for data driven use cases
 
Data Science in the Enterprise
Data Science in the EnterpriseData Science in the Enterprise
Data Science in the Enterprise
 
Project Management at Cisco Ben Rus TIM 101 Seminar – .docx
Project Management at Cisco Ben Rus TIM 101 Seminar – .docxProject Management at Cisco Ben Rus TIM 101 Seminar – .docx
Project Management at Cisco Ben Rus TIM 101 Seminar – .docx
 
High-Performance Analytics in the Cloud with Apache Impala
High-Performance Analytics in the Cloud with Apache ImpalaHigh-Performance Analytics in the Cloud with Apache Impala
High-Performance Analytics in the Cloud with Apache Impala
 
Accelerating Data Lakes and Streams with Real-time Analytics
Accelerating Data Lakes and Streams with Real-time AnalyticsAccelerating Data Lakes and Streams with Real-time Analytics
Accelerating Data Lakes and Streams with Real-time Analytics
 
Breed data scientists_ A Presentation.pptx
Breed data scientists_ A Presentation.pptxBreed data scientists_ A Presentation.pptx
Breed data scientists_ A Presentation.pptx
 

More from StampedeCon

Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...
Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...
Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...
StampedeCon
 
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
StampedeCon
 
Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017
Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017
Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017
StampedeCon
 
Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...
Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...
Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...
StampedeCon
 
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
StampedeCon
 
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
StampedeCon
 
Foundations of Machine Learning - StampedeCon AI Summit 2017
Foundations of Machine Learning - StampedeCon AI Summit 2017Foundations of Machine Learning - StampedeCon AI Summit 2017
Foundations of Machine Learning - StampedeCon AI Summit 2017
StampedeCon
 
Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...
Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...
Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...
StampedeCon
 
Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...
Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...
Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...
StampedeCon
 
Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017
Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017
Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017
StampedeCon
 
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017
AI in the Enterprise: Past,  Present &  Future - StampedeCon AI Summit 2017AI in the Enterprise: Past,  Present &  Future - StampedeCon AI Summit 2017
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017
StampedeCon
 
A Different Data Science Approach - StampedeCon AI Summit 2017
A Different Data Science Approach - StampedeCon AI Summit 2017A Different Data Science Approach - StampedeCon AI Summit 2017
A Different Data Science Approach - StampedeCon AI Summit 2017
StampedeCon
 
Graph in Customer 360 - StampedeCon Big Data Conference 2017
Graph in Customer 360 - StampedeCon Big Data Conference 2017Graph in Customer 360 - StampedeCon Big Data Conference 2017
Graph in Customer 360 - StampedeCon Big Data Conference 2017
StampedeCon
 
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
StampedeCon
 
Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017
Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017
Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017
StampedeCon
 
Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...
Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...
Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...
StampedeCon
 
Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...
Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...
Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...
StampedeCon
 
Innovation in the Data Warehouse - StampedeCon 2016
Innovation in the Data Warehouse - StampedeCon 2016Innovation in the Data Warehouse - StampedeCon 2016
Innovation in the Data Warehouse - StampedeCon 2016
StampedeCon
 
Creating a Data Driven Organization - StampedeCon 2016
Creating a Data Driven Organization - StampedeCon 2016Creating a Data Driven Organization - StampedeCon 2016
Creating a Data Driven Organization - StampedeCon 2016
StampedeCon
 
Using The Internet of Things for Population Health Management - StampedeCon 2016
Using The Internet of Things for Population Health Management - StampedeCon 2016Using The Internet of Things for Population Health Management - StampedeCon 2016
Using The Internet of Things for Population Health Management - StampedeCon 2016
StampedeCon
 

More from StampedeCon (20)

Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...
Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...
Why Should We Trust You-Interpretability of Deep Neural Networks - StampedeCo...
 
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
 
Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017
Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017
Predicting Outcomes When Your Outcomes are Graphs - StampedeCon AI Summit 2017
 
Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...
Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...
Novel Semi-supervised Probabilistic ML Approach to SNP Variant Calling - Stam...
 
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
How to Talk about AI to Non-analaysts - Stampedecon AI Summit 2017
 
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017
 
Foundations of Machine Learning - StampedeCon AI Summit 2017
Foundations of Machine Learning - StampedeCon AI Summit 2017Foundations of Machine Learning - StampedeCon AI Summit 2017
Foundations of Machine Learning - StampedeCon AI Summit 2017
 
Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...
Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...
Don't Start from Scratch: Transfer Learning for Novel Computer Vision Problem...
 
Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...
Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...
Bringing the Whole Elephant Into View Can Cognitive Systems Bring Real Soluti...
 
Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017
Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017
Automated AI The Next Frontier in Analytics - StampedeCon AI Summit 2017
 
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017
AI in the Enterprise: Past,  Present &  Future - StampedeCon AI Summit 2017AI in the Enterprise: Past,  Present &  Future - StampedeCon AI Summit 2017
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017
 
A Different Data Science Approach - StampedeCon AI Summit 2017
A Different Data Science Approach - StampedeCon AI Summit 2017A Different Data Science Approach - StampedeCon AI Summit 2017
A Different Data Science Approach - StampedeCon AI Summit 2017
 
Graph in Customer 360 - StampedeCon Big Data Conference 2017
Graph in Customer 360 - StampedeCon Big Data Conference 2017Graph in Customer 360 - StampedeCon Big Data Conference 2017
Graph in Customer 360 - StampedeCon Big Data Conference 2017
 
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
End-to-end Big Data Projects with Python - StampedeCon Big Data Conference 2017
 
Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017
Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017
Doing Big Data Using Amazon's Analogs - StampedeCon Big Data Conference 2017
 
Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...
Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...
Enabling New Business Capabilities with Cloud-based Streaming Data Architectu...
 
Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...
Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...
Big Data Meets IoT: Lessons From the Cloud on Polling, Collecting, and Analyz...
 
Innovation in the Data Warehouse - StampedeCon 2016
Innovation in the Data Warehouse - StampedeCon 2016Innovation in the Data Warehouse - StampedeCon 2016
Innovation in the Data Warehouse - StampedeCon 2016
 
Creating a Data Driven Organization - StampedeCon 2016
Creating a Data Driven Organization - StampedeCon 2016Creating a Data Driven Organization - StampedeCon 2016
Creating a Data Driven Organization - StampedeCon 2016
 
Using The Internet of Things for Population Health Management - StampedeCon 2016
Using The Internet of Things for Population Health Management - StampedeCon 2016Using The Internet of Things for Population Health Management - StampedeCon 2016
Using The Internet of Things for Population Health Management - StampedeCon 2016
 

Recently uploaded

Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Neo4j
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Neo4j
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Zilliz
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Aggregage
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 

Recently uploaded (20)

Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 

Piloting Big Data: Where To Start? - StampedeCon 2014

  • 1. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience Piloting Big Data Where to Start? 29 May 2014 – StampedeCon – St. Louis John Akred (@BigDataAnalysis), www.svds.com @SVDataScience
  • 2. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 2 Solving  difficult  problems  with   technology,  data,  and  science   Cross-­‐func<onal  teams   Agile  delivery  methods   Business-­‐driven  technology   strategy  and  advisory  
  • 3. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 3 1 Why big data? 2 What is a pilot? 3 Choosing a use case 4 Defining success Doing a Big Data Pilot Fielding a team Delivering
  • 4. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience http://svds.com/post/ successful-data-teams-are-agile-and-cross-functional 4
  • 5. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience Why Big Data? 5 1. New Capabilities 2. Economic Scalability
  • 6. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 6 DATA PLATFORMS FOR NEW CAPABILITES
  • 7. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 7 THE DATA VALUE CHAIN Acquire Ingest Process Persist Integrate Analyze Expose
  • 8. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 8 DATA PLATFORMS FOR ECONOMIC SCALABILITY at NetApp
  • 9. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 9 UP OR OUT? The SaaS Edition Users Revenue scale-out cost good times bummer Different products and features put different demands on the data infrastructure •  Profitable •  Unprofitable Increasing cost per user from scale-up architectures causes a barrier to economic expansion of the product user base.
  • 10. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 10 UP OR OUT? in the enterprise Different use cases put different demands on the data infrastructure •  UC1 •  UC2 •  UC3 •  UC4 •  UCn Increasing cost per unit of capability from scale-up architectures causes rationing of resources. Only the most valuable use cases are pursued. Data Resource Usage Value scale-out cost UC 1 UC2 UC3 UC4 UCn
  • 11. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 11 StampedeCon 1 Why big data? 2 What is a pilot? 3 Choosing a use case 4 Defining success
  • 12. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 12 From idea to production Agile: Iterate to value, answering the most valuable questions as quickly as possible Plan Prototype Pilot Production þ þ þ þ þ
  • 13. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience What is a Pilot? Plan and Initialize • Define architectural approach • Identify resources • Provision training • Choose use case • Define success • Populate initial backlog and sprint plans Prototype and Prove • Identify poorly understood functionality • Isolate and experiment • Determine solution approaches • Evaluate solution(s) • Correctness • Scale • Economics Pilot • Define end-to- end “steel thread” • Partition off pilot population • Build and integrate system components • Modify associated processes • Train pilot user team Production • Expand to entire user/ customer/ partner/ etc population • Industrialize monitoring capabilities • Re-engineer processes • Train user community 13
  • 14. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 14 1 Why big data? 2 What is a pilot? 3 Choosing a use case 4 Defining success StampedeCon
  • 15. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 15 STRATEGIC IMPERATIVES BUSINESS OBJECTIVES MAP OBJECTIVES TO TECHNICAL WORKLOADS RATIONALIZE WORKLOADS Strategic Workloads
  • 16. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 16 BUILDING A DATA PLATFORM External Systems Data Acquisition Internal Data Sources Data Management Security, Operations, Data Quality, Meta Data Management and Data Lineage Analytics Data Ingestion Data Repository External Data Sources Persistence Offline Processing Real Time Processing Batch Processing Data Services
  • 17. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience 17 1 Why big data? 2 What is a pilot? 3 Choosing a use case 4 Defining success StampedeCon
  • 18. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience •  Incremental revenue •  Time to market •  Economic functional implementation •  Cost avoidance •  Brand benefit •  Goodwill ✔ 18 Defining Success
  • 19. © 2014 Silicon Valley Data Science LLC All Rights Reserved. www.svds.com @SVDataScience thank you! 19 Yes, we’re hiring www.svds.com/join-us