SlideShare a Scribd company logo
1 of 10
a d v i s o r s
mwd
helping you create business improvement from IT investment
Navigating the Big Data landscape
Craig Wentworth
Principal Analyst
Big Data’s moonshot program
© MWD Advisors 2014 www.mwdadvisors.com 2
Growth of Google,
Facebook, etc
Cost of memory,
hardware, etc
Time
Pb
Amount of data
Time to value
$
$bn
Number
of users
ms
How Hadoop’s march has dominated Big
Data market development
© MWD Advisors 2014 www.mwdadvisors.com 3
2004
Google’s MapReduce
research paper
2005
Yahoo!
develops
Hadoop
2013
Hadoop 2
goes GA
2007
Hadoop goes
open source
at Apache
2011
Hadoop 1
goes GA
Hadoop
NoSQL
Streaming
Data
warehouse
Specialist
sources
Big Data’s open source army is about much
more than Hadoop, in fact
© MWD Advisors 2014 www.mwdadvisors.com 4
PostgreSQL
MySQL
Drill Spark
Storm
Samza
Accumulo
HBase
Cassandra
Couchbase
MongoDB
Redis
Different use cases have dramatically
different requirements
© MWD Advisors 2014 www.mwdadvisors.com 5Extreme
Scale
Extreme
Speed
“Everything
else”
National security,
Fraud detection
Transport, Utilities
management
Customer
analytics for
next best
action
Predictive
maintenance
Speed
Scale
Different capability clusters are most suited
to different use cases
© MWD Advisors 2014 www.mwdadvisors.com 6
CEP, Real-time,
Streaming
databases
Hadoop 1
(MapReduce)
EDW, Data Marts
NoSQL
Wide
Column
Key- Value
Document
Graph
Specialised
sources
Hadoop 2
and OSS
ecosystem
Extreme
Scale
Extreme
Speed
“Everything
else”
Speed
Scale
Four aspects of delivery models you need
to consider
© MWD Advisors 2014 www.mwdadvisors.com 7
Licensing model Physical data
storage model
Operational
model
Analytics
architecture
Things you need to do
© MWD Advisors 2014 www.mwdadvisors.com 8
Watch Hadoop Be aware of the
shifting boundaries
Make sure you can
hold it all together
a d v i s o r s
mwd
helping you create business improvement from IT investment
Thank you
Any questions?
Craig Wentworth
craig@mwdadvisors.com
@craigwentworth
Check out our free reports:
Big Data for analytics: Vendor landscape
http://goo.gl/CJ2YRV
Hadoop: A driving force in the Big Data technology landscape
http://goo.gl/nf0lY1
Big Data: What is it and why should I care?
http://goo.gl/Sckh2v
Neil Ward-Dutton
neilwd@mwdadvisors.com
@neilwd
Hadoop logo (Slides 3, 8) – Intel Free Press
(http://commons.wikimedia.org/wiki/File:Apache_Hadoop_Elephant.jpg) CC-BY-SA-2.0
Apache Software Foundation logo (Slide 7) – José Carlos Gallego
(http://commons.wikimedia.org/wiki/File:The_Apache_Software_Foundation.jpg) CC-BY-SA-3.0
Locker metal security closed steel key hold (Slide 7) – Nemo (http://pixabay.com/en/locker-metal-security-
closed-steel-310533/) CC0
Intel SS4000-E Entry Storage Systems, excerpt (Slide 7) – Axel Schwenke
(https://www.flickr.com/photos/schwenke/1016958982/) CC-BY-SA-2.0
Clouds (Slide 7) – Clue (http://openclipart.org/detail/4112/simple-clouds-by-clue) PD
Plus disks (Slide 7) – Open storage pod (http://www.flickr.com/photos/openstoragepod/4392476653/) CC-BY-2.0
RAM-Not22233 (Slide 7) – mateusz (http://de.wikipedia.org/wiki/Arbeitsspeicher#mediaviewer/Datei:Ram-
not22233.jpg) CC-BY-SA-3.0
Database symbol (Slide 7) – rg1024 (http://openclipart.org/image/300px/svg_to_png/94723/db.png) PD
Signpost (Slide 8) – Succo (http://pixabay.com/en/directory-signposts-shield-note-230724/) CC0
Piecing it together (Slide 8) – Hans (http://pixabay.com/en/puzzle-play-share-piecing-together-318110/) CC0
Photo credits
© MWD Advisors 2014
(Except images, where indicated)
www.mwdadvisors.com 10

More Related Content

What's hot

Cognitive Systems
Cognitive SystemsCognitive Systems
Cognitive SystemsLukas Ott
 
Scaling Data overview
Scaling Data overviewScaling Data overview
Scaling Data overviewWade Malone
 
How Virtual Reality and Machine Learning Are Powering the New Age of Network ...
How Virtual Reality and Machine Learning Are Powering the New Age of Network ...How Virtual Reality and Machine Learning Are Powering the New Age of Network ...
How Virtual Reality and Machine Learning Are Powering the New Age of Network ...DataStax
 
Neo4j Health Care & Life Sciences Workshop 2021
Neo4j Health Care & Life Sciences Workshop 2021Neo4j Health Care & Life Sciences Workshop 2021
Neo4j Health Care & Life Sciences Workshop 2021Neo4j
 
The crusade for big data in the AAL domain
The crusade for big data in the AAL domainThe crusade for big data in the AAL domain
The crusade for big data in the AAL domainAALForum
 
Hadoop und IoT
Hadoop und IoTHadoop und IoT
Hadoop und IoTLukas Ott
 
Global Azure Bootcamp 2019 - Modernize your Data Platform with Azure
Global Azure Bootcamp 2019 - Modernize your Data Platform with AzureGlobal Azure Bootcamp 2019 - Modernize your Data Platform with Azure
Global Azure Bootcamp 2019 - Modernize your Data Platform with AzureSergio Zenatti Filho
 
Trillion graph : Distribuer les données connectées sur des centaines d'instan...
Trillion graph : Distribuer les données connectées sur des centaines d'instan...Trillion graph : Distribuer les données connectées sur des centaines d'instan...
Trillion graph : Distribuer les données connectées sur des centaines d'instan...Neo4j
 
Introduction Data Warehouse With BigQuery
Introduction Data Warehouse With BigQueryIntroduction Data Warehouse With BigQuery
Introduction Data Warehouse With BigQueryYatno Sudar
 
Webinar: Rearchitecting Storage for the Next Wave of Splunk Data Growth
Webinar: Rearchitecting Storage for the Next Wave of Splunk Data GrowthWebinar: Rearchitecting Storage for the Next Wave of Splunk Data Growth
Webinar: Rearchitecting Storage for the Next Wave of Splunk Data GrowthStorage Switzerland
 
Druid meetup @ Netflix (11/14/2018 )
Druid meetup @ Netflix  (11/14/2018 )Druid meetup @ Netflix  (11/14/2018 )
Druid meetup @ Netflix (11/14/2018 )Jaebin Yoon
 

What's hot (11)

Cognitive Systems
Cognitive SystemsCognitive Systems
Cognitive Systems
 
Scaling Data overview
Scaling Data overviewScaling Data overview
Scaling Data overview
 
How Virtual Reality and Machine Learning Are Powering the New Age of Network ...
How Virtual Reality and Machine Learning Are Powering the New Age of Network ...How Virtual Reality and Machine Learning Are Powering the New Age of Network ...
How Virtual Reality and Machine Learning Are Powering the New Age of Network ...
 
Neo4j Health Care & Life Sciences Workshop 2021
Neo4j Health Care & Life Sciences Workshop 2021Neo4j Health Care & Life Sciences Workshop 2021
Neo4j Health Care & Life Sciences Workshop 2021
 
The crusade for big data in the AAL domain
The crusade for big data in the AAL domainThe crusade for big data in the AAL domain
The crusade for big data in the AAL domain
 
Hadoop und IoT
Hadoop und IoTHadoop und IoT
Hadoop und IoT
 
Global Azure Bootcamp 2019 - Modernize your Data Platform with Azure
Global Azure Bootcamp 2019 - Modernize your Data Platform with AzureGlobal Azure Bootcamp 2019 - Modernize your Data Platform with Azure
Global Azure Bootcamp 2019 - Modernize your Data Platform with Azure
 
Trillion graph : Distribuer les données connectées sur des centaines d'instan...
Trillion graph : Distribuer les données connectées sur des centaines d'instan...Trillion graph : Distribuer les données connectées sur des centaines d'instan...
Trillion graph : Distribuer les données connectées sur des centaines d'instan...
 
Introduction Data Warehouse With BigQuery
Introduction Data Warehouse With BigQueryIntroduction Data Warehouse With BigQuery
Introduction Data Warehouse With BigQuery
 
Webinar: Rearchitecting Storage for the Next Wave of Splunk Data Growth
Webinar: Rearchitecting Storage for the Next Wave of Splunk Data GrowthWebinar: Rearchitecting Storage for the Next Wave of Splunk Data Growth
Webinar: Rearchitecting Storage for the Next Wave of Splunk Data Growth
 
Druid meetup @ Netflix (11/14/2018 )
Druid meetup @ Netflix  (11/14/2018 )Druid meetup @ Netflix  (11/14/2018 )
Druid meetup @ Netflix (11/14/2018 )
 

Viewers also liked

BPM for Mobile, Mobile for BPM
BPM for Mobile, Mobile for BPMBPM for Mobile, Mobile for BPM
BPM for Mobile, Mobile for BPMBP3 Global, Inc.
 
Social collaboration in transition
Social collaboration in transitionSocial collaboration in transition
Social collaboration in transitionMWD Advisors
 
Neil Ward-Dutton, Founder & Research Director at MWD Advisors - Innovating di...
Neil Ward-Dutton, Founder & Research Director at MWD Advisors - Innovating di...Neil Ward-Dutton, Founder & Research Director at MWD Advisors - Innovating di...
Neil Ward-Dutton, Founder & Research Director at MWD Advisors - Innovating di...Global Business Events
 
The Digital Enterprise: CIO perspectives
The Digital Enterprise: CIO perspectivesThe Digital Enterprise: CIO perspectives
The Digital Enterprise: CIO perspectivesMWD Advisors
 
Remixing BPM for the digital age
Remixing BPM for the digital ageRemixing BPM for the digital age
Remixing BPM for the digital ageMWD Advisors
 
Six key business / tech trends and how they're impacting IT investment and go...
Six key business / tech trends and how they're impacting IT investment and go...Six key business / tech trends and how they're impacting IT investment and go...
Six key business / tech trends and how they're impacting IT investment and go...MWD Advisors
 

Viewers also liked (7)

BPM for Mobile, Mobile for BPM
BPM for Mobile, Mobile for BPMBPM for Mobile, Mobile for BPM
BPM for Mobile, Mobile for BPM
 
Social collaboration in transition
Social collaboration in transitionSocial collaboration in transition
Social collaboration in transition
 
Schrodinger's BPM
Schrodinger's BPMSchrodinger's BPM
Schrodinger's BPM
 
Neil Ward-Dutton, Founder & Research Director at MWD Advisors - Innovating di...
Neil Ward-Dutton, Founder & Research Director at MWD Advisors - Innovating di...Neil Ward-Dutton, Founder & Research Director at MWD Advisors - Innovating di...
Neil Ward-Dutton, Founder & Research Director at MWD Advisors - Innovating di...
 
The Digital Enterprise: CIO perspectives
The Digital Enterprise: CIO perspectivesThe Digital Enterprise: CIO perspectives
The Digital Enterprise: CIO perspectives
 
Remixing BPM for the digital age
Remixing BPM for the digital ageRemixing BPM for the digital age
Remixing BPM for the digital age
 
Six key business / tech trends and how they're impacting IT investment and go...
Six key business / tech trends and how they're impacting IT investment and go...Six key business / tech trends and how they're impacting IT investment and go...
Six key business / tech trends and how they're impacting IT investment and go...
 

Similar to Navigating the big data landscape

Hadoop for beginners free course ppt
Hadoop for beginners   free course pptHadoop for beginners   free course ppt
Hadoop for beginners free course pptNjain85
 
IRJET- Systematic Review: Progression Study on BIG DATA articles
IRJET- Systematic Review: Progression Study on BIG DATA articlesIRJET- Systematic Review: Progression Study on BIG DATA articles
IRJET- Systematic Review: Progression Study on BIG DATA articlesIRJET Journal
 
Cloud Expo 2015: DICE: Developing Data-Intensive Cloud Applications with Iter...
Cloud Expo 2015: DICE: Developing Data-Intensive Cloud Applications with Iter...Cloud Expo 2015: DICE: Developing Data-Intensive Cloud Applications with Iter...
Cloud Expo 2015: DICE: Developing Data-Intensive Cloud Applications with Iter...DICE-H2020
 
Big Data & Open Source - Neil Jadhav
Big Data & Open Source - Neil JadhavBig Data & Open Source - Neil Jadhav
Big Data & Open Source - Neil JadhavSwapnil (Neil) Jadhav
 
Big Data on Public Cloud
Big Data on Public CloudBig Data on Public Cloud
Big Data on Public CloudIMC Institute
 
Big Data Performance and Capacity Management
Big Data Performance and Capacity ManagementBig Data Performance and Capacity Management
Big Data Performance and Capacity Managementrightsize
 
Big Data Management: A Unified Approach to Drive Business Results
Big Data Management: A Unified Approach to Drive Business ResultsBig Data Management: A Unified Approach to Drive Business Results
Big Data Management: A Unified Approach to Drive Business ResultsCA Technologies
 
Solving the Really Big Tech Problems with IoT
 Solving the Really Big Tech Problems with IoT Solving the Really Big Tech Problems with IoT
Solving the Really Big Tech Problems with IoTEric Kavanagh
 
Top 5 Tasks Of A Hadoop Developer Webinar
Top 5 Tasks Of A Hadoop Developer WebinarTop 5 Tasks Of A Hadoop Developer Webinar
Top 5 Tasks Of A Hadoop Developer WebinarSkillspeed
 
Big Data and Fast Data – Big and Fast Combined, is it Possible?
Big Data and Fast Data – Big and Fast Combined, is it Possible?Big Data and Fast Data – Big and Fast Combined, is it Possible?
Big Data and Fast Data – Big and Fast Combined, is it Possible?Guido Schmutz
 
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...Denodo
 
Connecta Event: Big Query och dataanalys med Google Cloud Platform
Connecta Event: Big Query och dataanalys med Google Cloud PlatformConnecta Event: Big Query och dataanalys med Google Cloud Platform
Connecta Event: Big Query och dataanalys med Google Cloud PlatformConnectaDigital
 
"Big Data beyond Apache Hadoop - How to Integrate ALL your Data" - JavaOne 2013
"Big Data beyond Apache Hadoop - How to Integrate ALL your Data" - JavaOne 2013"Big Data beyond Apache Hadoop - How to Integrate ALL your Data" - JavaOne 2013
"Big Data beyond Apache Hadoop - How to Integrate ALL your Data" - JavaOne 2013Kai Wähner
 
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)Sascha Dittmann
 
Self-Tuning Data Centers
Self-Tuning Data CentersSelf-Tuning Data Centers
Self-Tuning Data CentersReza Rahimi
 

Similar to Navigating the big data landscape (20)

Sree
SreeSree
Sree
 
Hadoop for beginners free course ppt
Hadoop for beginners   free course pptHadoop for beginners   free course ppt
Hadoop for beginners free course ppt
 
GDG Varna - Hadoop
GDG Varna - HadoopGDG Varna - Hadoop
GDG Varna - Hadoop
 
IRJET- Systematic Review: Progression Study on BIG DATA articles
IRJET- Systematic Review: Progression Study on BIG DATA articlesIRJET- Systematic Review: Progression Study on BIG DATA articles
IRJET- Systematic Review: Progression Study on BIG DATA articles
 
Cloud Expo 2015: DICE: Developing Data-Intensive Cloud Applications with Iter...
Cloud Expo 2015: DICE: Developing Data-Intensive Cloud Applications with Iter...Cloud Expo 2015: DICE: Developing Data-Intensive Cloud Applications with Iter...
Cloud Expo 2015: DICE: Developing Data-Intensive Cloud Applications with Iter...
 
Big Data & Open Source - Neil Jadhav
Big Data & Open Source - Neil JadhavBig Data & Open Source - Neil Jadhav
Big Data & Open Source - Neil Jadhav
 
Big Data on Public Cloud
Big Data on Public CloudBig Data on Public Cloud
Big Data on Public Cloud
 
3rd day big data
3rd day   big data3rd day   big data
3rd day big data
 
Big Data Performance and Capacity Management
Big Data Performance and Capacity ManagementBig Data Performance and Capacity Management
Big Data Performance and Capacity Management
 
Big Data Management: A Unified Approach to Drive Business Results
Big Data Management: A Unified Approach to Drive Business ResultsBig Data Management: A Unified Approach to Drive Business Results
Big Data Management: A Unified Approach to Drive Business Results
 
Solving the Really Big Tech Problems with IoT
 Solving the Really Big Tech Problems with IoT Solving the Really Big Tech Problems with IoT
Solving the Really Big Tech Problems with IoT
 
Top 5 Tasks Of A Hadoop Developer Webinar
Top 5 Tasks Of A Hadoop Developer WebinarTop 5 Tasks Of A Hadoop Developer Webinar
Top 5 Tasks Of A Hadoop Developer Webinar
 
Big Data and Fast Data – Big and Fast Combined, is it Possible?
Big Data and Fast Data – Big and Fast Combined, is it Possible?Big Data and Fast Data – Big and Fast Combined, is it Possible?
Big Data and Fast Data – Big and Fast Combined, is it Possible?
 
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
 
Seminarppt
SeminarpptSeminarppt
Seminarppt
 
Connecta Event: Big Query och dataanalys med Google Cloud Platform
Connecta Event: Big Query och dataanalys med Google Cloud PlatformConnecta Event: Big Query och dataanalys med Google Cloud Platform
Connecta Event: Big Query och dataanalys med Google Cloud Platform
 
"Big Data beyond Apache Hadoop - How to Integrate ALL your Data" - JavaOne 2013
"Big Data beyond Apache Hadoop - How to Integrate ALL your Data" - JavaOne 2013"Big Data beyond Apache Hadoop - How to Integrate ALL your Data" - JavaOne 2013
"Big Data beyond Apache Hadoop - How to Integrate ALL your Data" - JavaOne 2013
 
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)
SQLSaturday #230 - Introduction to Microsoft Big Data (Part 1)
 
TSE_Pres12.pptx
TSE_Pres12.pptxTSE_Pres12.pptx
TSE_Pres12.pptx
 
Self-Tuning Data Centers
Self-Tuning Data CentersSelf-Tuning Data Centers
Self-Tuning Data Centers
 

Recently uploaded

Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 

Recently uploaded (20)

Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 

Navigating the big data landscape

  • 1. a d v i s o r s mwd helping you create business improvement from IT investment Navigating the Big Data landscape Craig Wentworth Principal Analyst
  • 2. Big Data’s moonshot program © MWD Advisors 2014 www.mwdadvisors.com 2 Growth of Google, Facebook, etc Cost of memory, hardware, etc Time Pb Amount of data Time to value $ $bn Number of users ms
  • 3. How Hadoop’s march has dominated Big Data market development © MWD Advisors 2014 www.mwdadvisors.com 3 2004 Google’s MapReduce research paper 2005 Yahoo! develops Hadoop 2013 Hadoop 2 goes GA 2007 Hadoop goes open source at Apache 2011 Hadoop 1 goes GA
  • 4. Hadoop NoSQL Streaming Data warehouse Specialist sources Big Data’s open source army is about much more than Hadoop, in fact © MWD Advisors 2014 www.mwdadvisors.com 4 PostgreSQL MySQL Drill Spark Storm Samza Accumulo HBase Cassandra Couchbase MongoDB Redis
  • 5. Different use cases have dramatically different requirements © MWD Advisors 2014 www.mwdadvisors.com 5Extreme Scale Extreme Speed “Everything else” National security, Fraud detection Transport, Utilities management Customer analytics for next best action Predictive maintenance Speed Scale
  • 6. Different capability clusters are most suited to different use cases © MWD Advisors 2014 www.mwdadvisors.com 6 CEP, Real-time, Streaming databases Hadoop 1 (MapReduce) EDW, Data Marts NoSQL Wide Column Key- Value Document Graph Specialised sources Hadoop 2 and OSS ecosystem Extreme Scale Extreme Speed “Everything else” Speed Scale
  • 7. Four aspects of delivery models you need to consider © MWD Advisors 2014 www.mwdadvisors.com 7 Licensing model Physical data storage model Operational model Analytics architecture
  • 8. Things you need to do © MWD Advisors 2014 www.mwdadvisors.com 8 Watch Hadoop Be aware of the shifting boundaries Make sure you can hold it all together
  • 9. a d v i s o r s mwd helping you create business improvement from IT investment Thank you Any questions? Craig Wentworth craig@mwdadvisors.com @craigwentworth Check out our free reports: Big Data for analytics: Vendor landscape http://goo.gl/CJ2YRV Hadoop: A driving force in the Big Data technology landscape http://goo.gl/nf0lY1 Big Data: What is it and why should I care? http://goo.gl/Sckh2v Neil Ward-Dutton neilwd@mwdadvisors.com @neilwd
  • 10. Hadoop logo (Slides 3, 8) – Intel Free Press (http://commons.wikimedia.org/wiki/File:Apache_Hadoop_Elephant.jpg) CC-BY-SA-2.0 Apache Software Foundation logo (Slide 7) – José Carlos Gallego (http://commons.wikimedia.org/wiki/File:The_Apache_Software_Foundation.jpg) CC-BY-SA-3.0 Locker metal security closed steel key hold (Slide 7) – Nemo (http://pixabay.com/en/locker-metal-security- closed-steel-310533/) CC0 Intel SS4000-E Entry Storage Systems, excerpt (Slide 7) – Axel Schwenke (https://www.flickr.com/photos/schwenke/1016958982/) CC-BY-SA-2.0 Clouds (Slide 7) – Clue (http://openclipart.org/detail/4112/simple-clouds-by-clue) PD Plus disks (Slide 7) – Open storage pod (http://www.flickr.com/photos/openstoragepod/4392476653/) CC-BY-2.0 RAM-Not22233 (Slide 7) – mateusz (http://de.wikipedia.org/wiki/Arbeitsspeicher#mediaviewer/Datei:Ram- not22233.jpg) CC-BY-SA-3.0 Database symbol (Slide 7) – rg1024 (http://openclipart.org/image/300px/svg_to_png/94723/db.png) PD Signpost (Slide 8) – Succo (http://pixabay.com/en/directory-signposts-shield-note-230724/) CC0 Piecing it together (Slide 8) – Hans (http://pixabay.com/en/puzzle-play-share-piecing-together-318110/) CC0 Photo credits © MWD Advisors 2014 (Except images, where indicated) www.mwdadvisors.com 10