SlideShare a Scribd company logo
1 of 40
This is Not Your Father’s Database: Everything You Need to Know Now About Cloud Computing and Emerging Database Technology  Guy Harrison Director Research and Development, Melbourne guy.harrison@quest.com www.guyharrison.net
Introductions
Mainframes After the gold rush Minicomputers Client Server Internet/Y2K Boom
Current Day Trends Big Data Cloud computing Solid State Disk
Big Data The Industrial Revolution of data*  User generated data: Twitter, Facebook, Amazon  Machine generated data: RFID, POS, cell phones, GPS Traditional RDBMS neither economic or capable * http://radar.oreilly.com/2008/11/the-commoditization-of-massive.html
Big data 1:  Google
Map Reduce  Map Map Map Map Map Map Map Map Map Map Map Start Reduce Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map
Hadoop: Open source Map-reduce  Yahoo! Hadoop cluster: 4000 nodes 16PB disk 64 TB of RAM 32,000 Cores Very Low $/TB
Hive SQL Java Results
Big Data 2:  Web 2.0
Twitter Growth
The fail whale
Web Servers Memcached Servers Database Servers  Read Only Slaves  Shard (G-O) Shard (P-Z) Shard (A-F)
Clouds and Elastic provisioning Capacity / Demand Demand Hardware upgrade Under provisioned Capacity Over provisioned Time
CAP Theorem Availability RD B M S Consistency NO GO NoSQL Partition Tolerance
In search of the elastic database	 Big Web sites AND Cloud applications need servers that scale up (and down) on demand Elastic provisioning works fine for web servers, application servers, etc. However RDBMS does not scale easily: SQL Azure limited to one database <50GB on a single host Oracle’s RAC not supported in cloud environments MySQL sharding “obnoxious” Many are willing to sacrifice relational database features for scalability and operational simplicity
The NoSQL movement
NoSQL (A.K.A.) Cloud databases Generally DO NOT support SQL Transactions Immediate consistency  Usually DO support: Elasticity (scale out AND in) Eventual consistency Inherent redundancy and fault tolerance
NoSQL Data Models
MemcacheDB Azure Table Services Key Value Stores Redis Tokyo Cabinet SimpleDB Riak Amazon Dynamo Voldemort Google BigTable Cassandra Hbase Hypertable CouchDB Document DB JSON/XML DB MongoDB Neo4J Graph Databases FlockDB
Not so easy to get the data out....
Amazon AWS Cloud On-Premise  (AKA private Cloud) MySQL Data Hub SQL HBase SimpleDB SQL Data Hub Microsoft Azure Cloud SQL Azure Table Services SQL Server Oracle
Big Data 3:  Data Warehousing
Data Warehouse players
DATAllegro architecture
Column Databases (Vertica, Sybase) Data is stored together in columns Very fast answers to analytic aggregate queries Better compression Not write optimized
Disk drives and Moore’s law Transistor density doubles every 18 months Exponential growth is observed in most electronic components: CPU clock speeds RAM Hard Disk Drive storage density  But not in mechanical components Service time (Seek latency) – limited by actuator arm speed and disk circumference  Throughput (rotational latency) – limited by speed of rotation, circumference and data density
Big Data vs. Fast Data Disk trends 2001-2009
SSD to the rescue?
Power consumption
Economics of SSD
Fast reads but slow writes
Hierarchical storage management  $/GB $/IOP
In Memory Databases:  VoltDB & H-Store In Memory Distributed (“Sharded”) Database No transactional IO ACID transactions (k-safety) Single Threaded (no latches or locks) Java Stored Procedure transactions Hierarchical data model  ,[object Object]
Spool out to DW for ad-hoc analysis
Very high TPS for suitable applications,[object Object]
The Next Generation?

More Related Content

What's hot

Best Practices for Deploying Hadoop (BigInsights) in the Cloud
Best Practices for Deploying Hadoop (BigInsights) in the CloudBest Practices for Deploying Hadoop (BigInsights) in the Cloud
Best Practices for Deploying Hadoop (BigInsights) in the CloudLeons Petražickis
 
Výhody a benefity nasazení Oracle Database Appliance
Výhody a benefity nasazení Oracle Database ApplianceVýhody a benefity nasazení Oracle Database Appliance
Výhody a benefity nasazení Oracle Database ApplianceMarketingArrowECS_CZ
 
Dell Lustre Storage Architecture Presentation - MBUG 2016
Dell Lustre Storage Architecture Presentation - MBUG 2016Dell Lustre Storage Architecture Presentation - MBUG 2016
Dell Lustre Storage Architecture Presentation - MBUG 2016Andrew Underwood
 
FAQ on Dedupe NetApp
FAQ on Dedupe NetAppFAQ on Dedupe NetApp
FAQ on Dedupe NetAppAshwin Pawar
 
Sun Oracle Exadata Technical Overview V1
Sun Oracle Exadata Technical Overview V1Sun Oracle Exadata Technical Overview V1
Sun Oracle Exadata Technical Overview V1jenkin
 
ZFS for Databases
ZFS for DatabasesZFS for Databases
ZFS for Databasesahl0003
 
Lustre Releases Update from LAD'14
Lustre Releases Update from LAD'14Lustre Releases Update from LAD'14
Lustre Releases Update from LAD'14inside-BigData.com
 
Hi Speed Datawarehousing
Hi Speed DatawarehousingHi Speed Datawarehousing
Hi Speed DatawarehousingJos van Dongen
 
Accelerating analytics workloads with Alluxio data orchestration and Intel® O...
Accelerating analytics workloads with Alluxio data orchestration and Intel® O...Accelerating analytics workloads with Alluxio data orchestration and Intel® O...
Accelerating analytics workloads with Alluxio data orchestration and Intel® O...Alluxio, Inc.
 
Database 101 on IBM i
Database 101 on IBM iDatabase 101 on IBM i
Database 101 on IBM iHelpSystems
 
Database server comparison: Dell PowerEdge R630 vs. Lenovo ThinkServer RD550
Database server comparison: Dell PowerEdge R630 vs. Lenovo ThinkServer RD550Database server comparison: Dell PowerEdge R630 vs. Lenovo ThinkServer RD550
Database server comparison: Dell PowerEdge R630 vs. Lenovo ThinkServer RD550Principled Technologies
 
Aerospike Hybrid Memory Architecture
Aerospike Hybrid Memory ArchitectureAerospike Hybrid Memory Architecture
Aerospike Hybrid Memory ArchitectureAerospike, Inc.
 
QCon2016--Drive Best Spark Performance on AI
QCon2016--Drive Best Spark Performance on AIQCon2016--Drive Best Spark Performance on AI
QCon2016--Drive Best Spark Performance on AILex Yu
 
Oracle Cloud Infrastructure – Storage
Oracle Cloud Infrastructure – StorageOracle Cloud Infrastructure – Storage
Oracle Cloud Infrastructure – StorageMarketingArrowECS_CZ
 
DDN: Massively-Scalable Platforms and Solutions Engineered for the Big Data a...
DDN: Massively-Scalable Platforms and Solutions Engineered for the Big Data a...DDN: Massively-Scalable Platforms and Solutions Engineered for the Big Data a...
DDN: Massively-Scalable Platforms and Solutions Engineered for the Big Data a...inside-BigData.com
 
Netapp online training-30Hours Classes, 30Hours Lab,Assignments, Project
Netapp online training-30Hours Classes, 30Hours Lab,Assignments, ProjectNetapp online training-30Hours Classes, 30Hours Lab,Assignments, Project
Netapp online training-30Hours Classes, 30Hours Lab,Assignments, ProjectVidhyalive
 

What's hot (20)

Best Practices for Deploying Hadoop (BigInsights) in the Cloud
Best Practices for Deploying Hadoop (BigInsights) in the CloudBest Practices for Deploying Hadoop (BigInsights) in the Cloud
Best Practices for Deploying Hadoop (BigInsights) in the Cloud
 
Výhody a benefity nasazení Oracle Database Appliance
Výhody a benefity nasazení Oracle Database ApplianceVýhody a benefity nasazení Oracle Database Appliance
Výhody a benefity nasazení Oracle Database Appliance
 
Dell Lustre Storage Architecture Presentation - MBUG 2016
Dell Lustre Storage Architecture Presentation - MBUG 2016Dell Lustre Storage Architecture Presentation - MBUG 2016
Dell Lustre Storage Architecture Presentation - MBUG 2016
 
ODA X6-2 family
ODA X6-2 familyODA X6-2 family
ODA X6-2 family
 
FAQ on Dedupe NetApp
FAQ on Dedupe NetAppFAQ on Dedupe NetApp
FAQ on Dedupe NetApp
 
Sun Oracle Exadata Technical Overview V1
Sun Oracle Exadata Technical Overview V1Sun Oracle Exadata Technical Overview V1
Sun Oracle Exadata Technical Overview V1
 
ZFS appliance
ZFS applianceZFS appliance
ZFS appliance
 
ZFS for Databases
ZFS for DatabasesZFS for Databases
ZFS for Databases
 
Exadata
ExadataExadata
Exadata
 
Lustre Releases Update from LAD'14
Lustre Releases Update from LAD'14Lustre Releases Update from LAD'14
Lustre Releases Update from LAD'14
 
Hi Speed Datawarehousing
Hi Speed DatawarehousingHi Speed Datawarehousing
Hi Speed Datawarehousing
 
Accelerating analytics workloads with Alluxio data orchestration and Intel® O...
Accelerating analytics workloads with Alluxio data orchestration and Intel® O...Accelerating analytics workloads with Alluxio data orchestration and Intel® O...
Accelerating analytics workloads with Alluxio data orchestration and Intel® O...
 
Database 101 on IBM i
Database 101 on IBM iDatabase 101 on IBM i
Database 101 on IBM i
 
Database server comparison: Dell PowerEdge R630 vs. Lenovo ThinkServer RD550
Database server comparison: Dell PowerEdge R630 vs. Lenovo ThinkServer RD550Database server comparison: Dell PowerEdge R630 vs. Lenovo ThinkServer RD550
Database server comparison: Dell PowerEdge R630 vs. Lenovo ThinkServer RD550
 
Aerospike Hybrid Memory Architecture
Aerospike Hybrid Memory ArchitectureAerospike Hybrid Memory Architecture
Aerospike Hybrid Memory Architecture
 
QCon2016--Drive Best Spark Performance on AI
QCon2016--Drive Best Spark Performance on AIQCon2016--Drive Best Spark Performance on AI
QCon2016--Drive Best Spark Performance on AI
 
Oracle Cloud Infrastructure – Storage
Oracle Cloud Infrastructure – StorageOracle Cloud Infrastructure – Storage
Oracle Cloud Infrastructure – Storage
 
Avoid the SAN Trap
Avoid the SAN TrapAvoid the SAN Trap
Avoid the SAN Trap
 
DDN: Massively-Scalable Platforms and Solutions Engineered for the Big Data a...
DDN: Massively-Scalable Platforms and Solutions Engineered for the Big Data a...DDN: Massively-Scalable Platforms and Solutions Engineered for the Big Data a...
DDN: Massively-Scalable Platforms and Solutions Engineered for the Big Data a...
 
Netapp online training-30Hours Classes, 30Hours Lab,Assignments, Project
Netapp online training-30Hours Classes, 30Hours Lab,Assignments, ProjectNetapp online training-30Hours Classes, 30Hours Lab,Assignments, Project
Netapp online training-30Hours Classes, 30Hours Lab,Assignments, Project
 

Viewers also liked

Happy Father's Day
Happy Father's DayHappy Father's Day
Happy Father's Daylindajlord
 
STEM in Motion - BASA Conference
STEM in Motion - BASA ConferenceSTEM in Motion - BASA Conference
STEM in Motion - BASA ConferenceOHM Advisors
 
Awkward Family Photos Slide Show
Awkward Family Photos   Slide ShowAwkward Family Photos   Slide Show
Awkward Family Photos Slide ShowRobinNicole621
 
Linha 0i - Comparativo e opções
Linha 0i - Comparativo e opçõesLinha 0i - Comparativo e opções
Linha 0i - Comparativo e opçõesPrestus®
 
Wastewater Treatment Systems-Public And Private
Wastewater Treatment Systems-Public And PrivateWastewater Treatment Systems-Public And Private
Wastewater Treatment Systems-Public And PrivateOHM Advisors
 
Asset Management for Small Systems - AWWA Conference
Asset Management for Small Systems - AWWA ConferenceAsset Management for Small Systems - AWWA Conference
Asset Management for Small Systems - AWWA ConferenceOHM Advisors
 
Rosa Et Al. 2010
Rosa Et Al. 2010Rosa Et Al. 2010
Rosa Et Al. 2010sabrinarosa
 
Information Skills: 4. Evaluating Sources (Natural Sciences, Bangor University)
Information Skills: 4. Evaluating Sources (Natural Sciences, Bangor University)Information Skills: 4. Evaluating Sources (Natural Sciences, Bangor University)
Information Skills: 4. Evaluating Sources (Natural Sciences, Bangor University)Vashti Zarach
 
The Top 4 risks in P4P (Pay for Performance) 20120611
The Top 4 risks in P4P (Pay for Performance) 20120611The Top 4 risks in P4P (Pay for Performance) 20120611
The Top 4 risks in P4P (Pay for Performance) 20120611PERFORMENSATION
 
Carpet Advantage 2
Carpet Advantage 2Carpet Advantage 2
Carpet Advantage 2rlantz
 
Adventures in freemium
Adventures in freemiumAdventures in freemium
Adventures in freemiumNavin Ganeshan
 
Linha Vivo - Comparativo e opções
Linha Vivo - Comparativo e opçõesLinha Vivo - Comparativo e opções
Linha Vivo - Comparativo e opçõesPrestus®
 
Perception of Public Works - APWA Conference
Perception of Public Works - APWA Conference Perception of Public Works - APWA Conference
Perception of Public Works - APWA Conference OHM Advisors
 
Sim House Example Dogwood
Sim House Example   DogwoodSim House Example   Dogwood
Sim House Example Dogwoodyog_live
 
7 key ways to drive revenue using li updated
7 key ways to drive revenue using li updated7 key ways to drive revenue using li updated
7 key ways to drive revenue using li updatedmbieler
 
Pró-Labore - Como aumentar o seu
Pró-Labore - Como aumentar o seuPró-Labore - Como aumentar o seu
Pró-Labore - Como aumentar o seuPrestus®
 
5, 4, 3, 2, 1: The Code to Better Compensation Planning
5, 4, 3, 2, 1:  The Code to Better Compensation Planning5, 4, 3, 2, 1:  The Code to Better Compensation Planning
5, 4, 3, 2, 1: The Code to Better Compensation PlanningPERFORMENSATION
 
Green Stormwater: LID with GIS
Green Stormwater: LID with GISGreen Stormwater: LID with GIS
Green Stormwater: LID with GISOHM Advisors
 
Building a Community Around your Blog 2 - Let the Comments be your Content!
Building a Community Around your Blog 2 - Let the Comments be your Content!Building a Community Around your Blog 2 - Let the Comments be your Content!
Building a Community Around your Blog 2 - Let the Comments be your Content!Brendan Sera-Shriar
 

Viewers also liked (20)

Happy Father's Day
Happy Father's DayHappy Father's Day
Happy Father's Day
 
STEM in Motion - BASA Conference
STEM in Motion - BASA ConferenceSTEM in Motion - BASA Conference
STEM in Motion - BASA Conference
 
Awkward Family Photos Slide Show
Awkward Family Photos   Slide ShowAwkward Family Photos   Slide Show
Awkward Family Photos Slide Show
 
Linha 0i - Comparativo e opções
Linha 0i - Comparativo e opçõesLinha 0i - Comparativo e opções
Linha 0i - Comparativo e opções
 
Wastewater Treatment Systems-Public And Private
Wastewater Treatment Systems-Public And PrivateWastewater Treatment Systems-Public And Private
Wastewater Treatment Systems-Public And Private
 
Asset Management for Small Systems - AWWA Conference
Asset Management for Small Systems - AWWA ConferenceAsset Management for Small Systems - AWWA Conference
Asset Management for Small Systems - AWWA Conference
 
Rosa Et Al. 2010
Rosa Et Al. 2010Rosa Et Al. 2010
Rosa Et Al. 2010
 
Information Skills: 4. Evaluating Sources (Natural Sciences, Bangor University)
Information Skills: 4. Evaluating Sources (Natural Sciences, Bangor University)Information Skills: 4. Evaluating Sources (Natural Sciences, Bangor University)
Information Skills: 4. Evaluating Sources (Natural Sciences, Bangor University)
 
The Top 4 risks in P4P (Pay for Performance) 20120611
The Top 4 risks in P4P (Pay for Performance) 20120611The Top 4 risks in P4P (Pay for Performance) 20120611
The Top 4 risks in P4P (Pay for Performance) 20120611
 
Carpet Advantage 2
Carpet Advantage 2Carpet Advantage 2
Carpet Advantage 2
 
Adventures in freemium
Adventures in freemiumAdventures in freemium
Adventures in freemium
 
Linha Vivo - Comparativo e opções
Linha Vivo - Comparativo e opçõesLinha Vivo - Comparativo e opções
Linha Vivo - Comparativo e opções
 
Perception of Public Works - APWA Conference
Perception of Public Works - APWA Conference Perception of Public Works - APWA Conference
Perception of Public Works - APWA Conference
 
Sim House Example Dogwood
Sim House Example   DogwoodSim House Example   Dogwood
Sim House Example Dogwood
 
7 key ways to drive revenue using li updated
7 key ways to drive revenue using li updated7 key ways to drive revenue using li updated
7 key ways to drive revenue using li updated
 
AAUP Growing Sales Slides
AAUP Growing Sales SlidesAAUP Growing Sales Slides
AAUP Growing Sales Slides
 
Pró-Labore - Como aumentar o seu
Pró-Labore - Como aumentar o seuPró-Labore - Como aumentar o seu
Pró-Labore - Como aumentar o seu
 
5, 4, 3, 2, 1: The Code to Better Compensation Planning
5, 4, 3, 2, 1:  The Code to Better Compensation Planning5, 4, 3, 2, 1:  The Code to Better Compensation Planning
5, 4, 3, 2, 1: The Code to Better Compensation Planning
 
Green Stormwater: LID with GIS
Green Stormwater: LID with GISGreen Stormwater: LID with GIS
Green Stormwater: LID with GIS
 
Building a Community Around your Blog 2 - Let the Comments be your Content!
Building a Community Around your Blog 2 - Let the Comments be your Content!Building a Community Around your Blog 2 - Let the Comments be your Content!
Building a Community Around your Blog 2 - Let the Comments be your Content!
 

Similar to Next generation databases july2010

How I learned to stop worrying and love Oracle
How I learned to stop worrying and love OracleHow I learned to stop worrying and love Oracle
How I learned to stop worrying and love OracleGuy Harrison
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon RedshiftAmazon Web Services
 
Complex Analytics with NoSQL Data Store in Real Time
Complex Analytics with NoSQL Data Store in Real TimeComplex Analytics with NoSQL Data Store in Real Time
Complex Analytics with NoSQL Data Store in Real TimeNati Shalom
 
Getting Started with Big Data and HPC in the Cloud - August 2015
Getting Started with Big Data and HPC in the Cloud - August 2015Getting Started with Big Data and HPC in the Cloud - August 2015
Getting Started with Big Data and HPC in the Cloud - August 2015Amazon Web Services
 
Inroduction to Big Data
Inroduction to Big DataInroduction to Big Data
Inroduction to Big DataOmnia Safaan
 
Big Data Session 1.pptx
Big Data Session 1.pptxBig Data Session 1.pptx
Big Data Session 1.pptxElsonPaul2
 
EMC Isilon Database Converged deck
EMC Isilon Database Converged deckEMC Isilon Database Converged deck
EMC Isilon Database Converged deckKeithETD_CTO
 
Getting Started with Amazon Redshift
 Getting Started with Amazon Redshift Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftAmazon Web Services
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon RedshiftAmazon Web Services
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon RedshiftAmazon Web Services
 
Sharing Experiences in Cloud Adoption: Burlington, MA
Sharing Experiences in Cloud Adoption: Burlington, MASharing Experiences in Cloud Adoption: Burlington, MA
Sharing Experiences in Cloud Adoption: Burlington, MANuoDB
 
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...Amazon Web Services
 
Presentation architecting virtualized infrastructure for big data
Presentation   architecting virtualized infrastructure for big dataPresentation   architecting virtualized infrastructure for big data
Presentation architecting virtualized infrastructure for big datasolarisyourep
 
Presentation architecting virtualized infrastructure for big data
Presentation   architecting virtualized infrastructure for big dataPresentation   architecting virtualized infrastructure for big data
Presentation architecting virtualized infrastructure for big dataxKinAnx
 
Low-Latency Analytics with NoSQL – Introduction to Storm and Cassandra
Low-Latency Analytics with NoSQL – Introduction to Storm and CassandraLow-Latency Analytics with NoSQL – Introduction to Storm and Cassandra
Low-Latency Analytics with NoSQL – Introduction to Storm and CassandraCaserta
 
The Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
The Practice of Big Data - The Hadoop ecosystem explained with usage scenariosThe Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
The Practice of Big Data - The Hadoop ecosystem explained with usage scenarioskcmallu
 
Information processing architectures
Information processing architecturesInformation processing architectures
Information processing architecturesRaji Gogulapati
 

Similar to Next generation databases july2010 (20)

How I learned to stop worrying and love Oracle
How I learned to stop worrying and love OracleHow I learned to stop worrying and love Oracle
How I learned to stop worrying and love Oracle
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Complex Analytics with NoSQL Data Store in Real Time
Complex Analytics with NoSQL Data Store in Real TimeComplex Analytics with NoSQL Data Store in Real Time
Complex Analytics with NoSQL Data Store in Real Time
 
Getting Started with Big Data and HPC in the Cloud - August 2015
Getting Started with Big Data and HPC in the Cloud - August 2015Getting Started with Big Data and HPC in the Cloud - August 2015
Getting Started with Big Data and HPC in the Cloud - August 2015
 
Inroduction to Big Data
Inroduction to Big DataInroduction to Big Data
Inroduction to Big Data
 
Big Data Session 1.pptx
Big Data Session 1.pptxBig Data Session 1.pptx
Big Data Session 1.pptx
 
EMC Isilon Database Converged deck
EMC Isilon Database Converged deckEMC Isilon Database Converged deck
EMC Isilon Database Converged deck
 
NoSQL Basics - a quick tour
NoSQL Basics - a quick tourNoSQL Basics - a quick tour
NoSQL Basics - a quick tour
 
Getting Started with Amazon Redshift
 Getting Started with Amazon Redshift Getting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Big data business case
Big data   business caseBig data   business case
Big data business case
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Sharing Experiences in Cloud Adoption: Burlington, MA
Sharing Experiences in Cloud Adoption: Burlington, MASharing Experiences in Cloud Adoption: Burlington, MA
Sharing Experiences in Cloud Adoption: Burlington, MA
 
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
 
The CDO Agenda: how data architecture can help?
The CDO Agenda: how data architecture can help?The CDO Agenda: how data architecture can help?
The CDO Agenda: how data architecture can help?
 
Presentation architecting virtualized infrastructure for big data
Presentation   architecting virtualized infrastructure for big dataPresentation   architecting virtualized infrastructure for big data
Presentation architecting virtualized infrastructure for big data
 
Presentation architecting virtualized infrastructure for big data
Presentation   architecting virtualized infrastructure for big dataPresentation   architecting virtualized infrastructure for big data
Presentation architecting virtualized infrastructure for big data
 
Low-Latency Analytics with NoSQL – Introduction to Storm and Cassandra
Low-Latency Analytics with NoSQL – Introduction to Storm and CassandraLow-Latency Analytics with NoSQL – Introduction to Storm and Cassandra
Low-Latency Analytics with NoSQL – Introduction to Storm and Cassandra
 
The Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
The Practice of Big Data - The Hadoop ecosystem explained with usage scenariosThe Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
The Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
 
Information processing architectures
Information processing architecturesInformation processing architectures
Information processing architectures
 

More from Guy Harrison

Five database trends - updated April 2015
Five database trends - updated April 2015Five database trends - updated April 2015
Five database trends - updated April 2015Guy Harrison
 
From oracle to hadoop with Sqoop and other tools
From oracle to hadoop with Sqoop and other toolsFrom oracle to hadoop with Sqoop and other tools
From oracle to hadoop with Sqoop and other toolsGuy Harrison
 
Thriving and surviving the Big Data revolution
Thriving and surviving the Big Data revolutionThriving and surviving the Big Data revolution
Thriving and surviving the Big Data revolutionGuy Harrison
 
Mega trends in information management
Mega trends in information managementMega trends in information management
Mega trends in information managementGuy Harrison
 
Big datacamp2013 share
Big datacamp2013 shareBig datacamp2013 share
Big datacamp2013 shareGuy Harrison
 
Hadoop, Oracle and the big data revolution collaborate 2013
Hadoop, Oracle and the big data revolution collaborate 2013Hadoop, Oracle and the big data revolution collaborate 2013
Hadoop, Oracle and the big data revolution collaborate 2013Guy Harrison
 
Hadoop, oracle and the industrial revolution of data
Hadoop, oracle and the industrial revolution of data Hadoop, oracle and the industrial revolution of data
Hadoop, oracle and the industrial revolution of data Guy Harrison
 
Making the most of ssd in oracle11g
Making the most of ssd in oracle11gMaking the most of ssd in oracle11g
Making the most of ssd in oracle11gGuy Harrison
 
Oracle sql high performance tuning
Oracle sql high performance tuningOracle sql high performance tuning
Oracle sql high performance tuningGuy Harrison
 
Hadoop and rdbms with sqoop
Hadoop and rdbms with sqoop Hadoop and rdbms with sqoop
Hadoop and rdbms with sqoop Guy Harrison
 
Optimize oracle on VMware (April 2011)
Optimize oracle on VMware (April 2011)Optimize oracle on VMware (April 2011)
Optimize oracle on VMware (April 2011)Guy Harrison
 
High Performance Plsql
High Performance PlsqlHigh Performance Plsql
High Performance PlsqlGuy Harrison
 
Performance By Design
Performance By DesignPerformance By Design
Performance By DesignGuy Harrison
 
Optimize Oracle On VMware (Sep 2011)
Optimize Oracle On VMware (Sep 2011)Optimize Oracle On VMware (Sep 2011)
Optimize Oracle On VMware (Sep 2011)Guy Harrison
 
Thanks for the Memory
Thanks for the MemoryThanks for the Memory
Thanks for the MemoryGuy Harrison
 
Top 10 tips for Oracle performance
Top 10 tips for Oracle performanceTop 10 tips for Oracle performance
Top 10 tips for Oracle performanceGuy Harrison
 
Performance By Design
Performance By DesignPerformance By Design
Performance By DesignGuy Harrison
 
High Performance Plsql
High Performance PlsqlHigh Performance Plsql
High Performance PlsqlGuy Harrison
 
Top 10 tips for Oracle performance (Updated April 2015)
Top 10 tips for Oracle performance (Updated April 2015)Top 10 tips for Oracle performance (Updated April 2015)
Top 10 tips for Oracle performance (Updated April 2015)Guy Harrison
 

More from Guy Harrison (19)

Five database trends - updated April 2015
Five database trends - updated April 2015Five database trends - updated April 2015
Five database trends - updated April 2015
 
From oracle to hadoop with Sqoop and other tools
From oracle to hadoop with Sqoop and other toolsFrom oracle to hadoop with Sqoop and other tools
From oracle to hadoop with Sqoop and other tools
 
Thriving and surviving the Big Data revolution
Thriving and surviving the Big Data revolutionThriving and surviving the Big Data revolution
Thriving and surviving the Big Data revolution
 
Mega trends in information management
Mega trends in information managementMega trends in information management
Mega trends in information management
 
Big datacamp2013 share
Big datacamp2013 shareBig datacamp2013 share
Big datacamp2013 share
 
Hadoop, Oracle and the big data revolution collaborate 2013
Hadoop, Oracle and the big data revolution collaborate 2013Hadoop, Oracle and the big data revolution collaborate 2013
Hadoop, Oracle and the big data revolution collaborate 2013
 
Hadoop, oracle and the industrial revolution of data
Hadoop, oracle and the industrial revolution of data Hadoop, oracle and the industrial revolution of data
Hadoop, oracle and the industrial revolution of data
 
Making the most of ssd in oracle11g
Making the most of ssd in oracle11gMaking the most of ssd in oracle11g
Making the most of ssd in oracle11g
 
Oracle sql high performance tuning
Oracle sql high performance tuningOracle sql high performance tuning
Oracle sql high performance tuning
 
Hadoop and rdbms with sqoop
Hadoop and rdbms with sqoop Hadoop and rdbms with sqoop
Hadoop and rdbms with sqoop
 
Optimize oracle on VMware (April 2011)
Optimize oracle on VMware (April 2011)Optimize oracle on VMware (April 2011)
Optimize oracle on VMware (April 2011)
 
High Performance Plsql
High Performance PlsqlHigh Performance Plsql
High Performance Plsql
 
Performance By Design
Performance By DesignPerformance By Design
Performance By Design
 
Optimize Oracle On VMware (Sep 2011)
Optimize Oracle On VMware (Sep 2011)Optimize Oracle On VMware (Sep 2011)
Optimize Oracle On VMware (Sep 2011)
 
Thanks for the Memory
Thanks for the MemoryThanks for the Memory
Thanks for the Memory
 
Top 10 tips for Oracle performance
Top 10 tips for Oracle performanceTop 10 tips for Oracle performance
Top 10 tips for Oracle performance
 
Performance By Design
Performance By DesignPerformance By Design
Performance By Design
 
High Performance Plsql
High Performance PlsqlHigh Performance Plsql
High Performance Plsql
 
Top 10 tips for Oracle performance (Updated April 2015)
Top 10 tips for Oracle performance (Updated April 2015)Top 10 tips for Oracle performance (Updated April 2015)
Top 10 tips for Oracle performance (Updated April 2015)
 

Recently uploaded

costume and set research powerpoint presentation
costume and set research powerpoint presentationcostume and set research powerpoint presentation
costume and set research powerpoint presentationphoebematthew05
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraDeakin University
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
Bluetooth Controlled Car with Arduino.pdf
Bluetooth Controlled Car with Arduino.pdfBluetooth Controlled Car with Arduino.pdf
Bluetooth Controlled Car with Arduino.pdfngoud9212
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 

Recently uploaded (20)

costume and set research powerpoint presentation
costume and set research powerpoint presentationcostume and set research powerpoint presentation
costume and set research powerpoint presentation
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning era
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
Bluetooth Controlled Car with Arduino.pdf
Bluetooth Controlled Car with Arduino.pdfBluetooth Controlled Car with Arduino.pdf
Bluetooth Controlled Car with Arduino.pdf
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 

Next generation databases july2010

  • 1. This is Not Your Father’s Database: Everything You Need to Know Now About Cloud Computing and Emerging Database Technology  Guy Harrison Director Research and Development, Melbourne guy.harrison@quest.com www.guyharrison.net
  • 3.
  • 4.
  • 5. Mainframes After the gold rush Minicomputers Client Server Internet/Y2K Boom
  • 6. Current Day Trends Big Data Cloud computing Solid State Disk
  • 7. Big Data The Industrial Revolution of data* User generated data: Twitter, Facebook, Amazon Machine generated data: RFID, POS, cell phones, GPS Traditional RDBMS neither economic or capable * http://radar.oreilly.com/2008/11/the-commoditization-of-massive.html
  • 8. Big data 1: Google
  • 9. Map Reduce Map Map Map Map Map Map Map Map Map Map Map Start Reduce Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map
  • 10. Hadoop: Open source Map-reduce Yahoo! Hadoop cluster: 4000 nodes 16PB disk 64 TB of RAM 32,000 Cores Very Low $/TB
  • 11. Hive SQL Java Results
  • 12. Big Data 2: Web 2.0
  • 15. Web Servers Memcached Servers Database Servers Read Only Slaves Shard (G-O) Shard (P-Z) Shard (A-F)
  • 16. Clouds and Elastic provisioning Capacity / Demand Demand Hardware upgrade Under provisioned Capacity Over provisioned Time
  • 17. CAP Theorem Availability RD B M S Consistency NO GO NoSQL Partition Tolerance
  • 18. In search of the elastic database Big Web sites AND Cloud applications need servers that scale up (and down) on demand Elastic provisioning works fine for web servers, application servers, etc. However RDBMS does not scale easily: SQL Azure limited to one database <50GB on a single host Oracle’s RAC not supported in cloud environments MySQL sharding “obnoxious” Many are willing to sacrifice relational database features for scalability and operational simplicity
  • 20. NoSQL (A.K.A.) Cloud databases Generally DO NOT support SQL Transactions Immediate consistency Usually DO support: Elasticity (scale out AND in) Eventual consistency Inherent redundancy and fault tolerance
  • 22. MemcacheDB Azure Table Services Key Value Stores Redis Tokyo Cabinet SimpleDB Riak Amazon Dynamo Voldemort Google BigTable Cassandra Hbase Hypertable CouchDB Document DB JSON/XML DB MongoDB Neo4J Graph Databases FlockDB
  • 23. Not so easy to get the data out....
  • 24. Amazon AWS Cloud On-Premise (AKA private Cloud) MySQL Data Hub SQL HBase SimpleDB SQL Data Hub Microsoft Azure Cloud SQL Azure Table Services SQL Server Oracle
  • 25.
  • 26. Big Data 3: Data Warehousing
  • 29. Column Databases (Vertica, Sybase) Data is stored together in columns Very fast answers to analytic aggregate queries Better compression Not write optimized
  • 30. Disk drives and Moore’s law Transistor density doubles every 18 months Exponential growth is observed in most electronic components: CPU clock speeds RAM Hard Disk Drive storage density But not in mechanical components Service time (Seek latency) – limited by actuator arm speed and disk circumference Throughput (rotational latency) – limited by speed of rotation, circumference and data density
  • 31. Big Data vs. Fast Data Disk trends 2001-2009
  • 32. SSD to the rescue?
  • 35. Fast reads but slow writes
  • 37.
  • 38. Spool out to DW for ad-hoc analysis
  • 39.

Editor's Notes

  1. Apologies, I’m a database type.....Quest is best known for toad, but we also have enterprise monitoring across all levels of the stackIn Melbourne, SQL Navigator + the spotlights. It’s not a complete co-incidence about the star trek theme.
  2. Insanely popular – literally millions of users
  3. This is your databaseThis is your database on crack
  4. That’s a predictable linear growth curve. Gets much worse for unpredictable or cyclic demand So I think it’s real, and it excites me because it represents the realization of a more industrialized model for providing computing resources. In the early days of electricity everybody had thier own power sources and every company needed engineers as a result. Nowdays, few companies need that...
  5. NoSQL tends to be strongly coupled with the application. Everybody else is out of luck
  6. Data warehouses doubling every three years.