Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
REMINDER
Check in on the
COLLABORATE mobile app
Top 5 Trends in Database
Technology
Guy Harrison,
Executive Director, Info...
Top 5 Trends in
Database
Technology
Guy Harrison,
Executive Director, Information Mgt R&D,
Dell Software Group
Dell Software Group3
Web: guyharrison.net
Email: guy.harrison@software.dell.com
Twitter: @guyharrison
Introductions
4
Dell Software Group
5
Dell Software Group
6
Dell Software Group
7
Dell Software Group
Dell and Quest – a brief history
Dell Software Group8
But Seriously
Dell Software Group9
5 Database Technology Trends
The end of “one size fits all”
Big Data and Hadoop
NoSQL
Columnar archit...
10
Dell Software Group
Trend #1:
The end of “one size fits
all”
Dell Software Group11
History of databases
Magnetic tape
“flat” (sequential) files
Pre-computer
technologies:
Printing pre...
Dell Software Group12
Why?
• 3rd Platform drives
new demands on
the database:
– Global High
Availability
– Data volumes
– ...
Dell Software Group13
Operational
RDBMS
(Oracle, SQL
Server, …)
In-memory
Analytics
(HANA,
Exalytics …)
In-memory
processi...
Dell Software Group14
Oracle engineered systems
15
Dell Software Group
Trend #2:
Big Data and Hadoop
Dell Software Group16
The 3-4 “V”s
Volume
•Terabytes
•Petabytes
•Exabytes
•Zetabytes
Velocity
•Transaction rates
•User pop...
17
Dell Software Group
The Industrial revolution of data
18
Dell Software Group
2005
19
Dell Software Group
2009
Dell Software Group20
The instrumented human
• Bluetooth Personal
Area Network
• 3G/WiFi Wide Area
Network
• GPS
• Storage...
Dell Software Group21
Dell Software Group22
The instrumented world
Dell Software Group23
Big Data is the culmination of cloud, social and
mobile
Dell Software Group24
More Data
• Storing all data – including machine generated and
sol, Social, community, demographic d...
Dell Software Group25
More Data
• Storing all data – including machine generated
and sol, Social, community, demographic d...
26
Dell Software Group
Pioneers of big data
27
Dell Software Group
28
Dell Software Group
29
Dell Software Group
30
Dell Software Group
31
Dell Software Group
Dell Software Group32
Google File System (GFS)
Map Reduce BigTable
Google Applications
Google Software Architecture (circa...
Dell Software Group33
Start ReduceMap
Map
Map
Map
Map
Map
Map
Map
Map
Map
Map
Map
Map
Map
Map
Map
Map
Map
Map
Map
Map
Map
...
Dell Software Group34
Hadoop: 1.0: Open Source Map-Reduce Stack
Dell Software Group35
Hadoop at Yahoo
• 2010(biggest cluster):
• 4000 nodes 16PB disk
• 64 TB of RAM
• 32,000 Cores
• 2014...
Dell Software Group36
Dell Software Group37
SQOOP
(RDBMS loader)
Hive
(Query)
Pig
(Scripting)
Flume
(Log Loader)
Oozie (Workflow manager)
Hadoop...
Dell Software Group39
Economies
$4,911
$750
$0 $1,000 $2,000 $3,000 $4,000 $5,000 $6,000
Exadata
Hadoop
Exadata vs Hadoop ...
Dell Software Group40
Hadoop is the most concrete Big Data
technology
Toad: your companion in
the Big Data revolution
Dell Software Group41
More Data
• Storing all data – including machine generated and
sol, Social, community, demographic d...
Dell Software Group42
More Data
• Storing all data – including machine generated and
sol, Social, community, demographic d...
Dell Software Group43
Big Data Analytics
AKA Data Science
Machine Learning
• Programs that evolve
with “experience
Predict...
Dell Software Group44
Dell Software Group45
Collective Intelligence
From now on, I’ll call
you ‘An Ambulance’.
OK?
“Siri call me an
ambulance”
47
Dell Software Group
Trend #3:
NoSQL
Dell Software Group48
Web
Servers
Database
Servers
Memcached
Servers
Shard (G-O) Shard (P-Z)Shard (A-F)
Read Only Slaves
Dell Software Group49
CAP Theorem says something has to give
• CAP (Brewer’s) Theorem
says you can only have
two out of th...
Dell Software Group50
Major influences on non-relational
• Eventually consistent transaction model
• Consistent hashing
Am...
Dell Software Group51
Amazon Dynamo Model
Dell Software Group52
Name Site Counter
Dick Ebay 507,018
Dick Google 690,414
Jane Google 716,426
Dick Facebook 723,649
Ja...
Dell Software Group53
OODBMS -1990s
• The OODBMS Manifesto
(Atkinson/Bancilhon/DeWitt/Dittrich/Maier/Zdo
nik, '90)
• "A re...
Dell Software Group54
Revenge of the Object Nerds – Document
databases
• Structured documents – XML and
JSON (JavaScript O...
Dell Software Group55
Graph
Database
Neo4J
Infinite
Graph
FlockDB
Document
JSON based
MongoDB
CouchDB
RethinkDB
XML based
...
Dell Software Group56
It’s not a database, it’s a key value store
http://browsertoolkit.com/fault-tolerance.png
Dell Software Group57
No Means Yes!
58
Dell Software Group
Trend #4:
Column-oriented DB
Dell Software Group59
Row orientation vs column orientation
ID Name DOB Salary Sales Expenses
1001 Dick 21/12/60 67,000 78...
Dell Software Group60
Analytical Queries
Block ID Name DOB Salary Sales Expenses
1 1001 Dick 21/12/60 67,000 78980 3244
2 ...
Dell Software Group61
Compression
Block ID Name DOB Salary Sales Expenses
1 1001 Dick 21/12/60 67,000 78980 3244
2 1002 Ja...
Dell Software Group62
Inserts
Block ID Name DOB Salary Sales Expenses
1 1001 Dick 21/12/60 67,000 78980 3244
2 1002 Jane 1...
Dell Software Group63
C-Store (Vertica) Solution for inserts
Read Optimized Store
• Columnar
• Disk-based
• Highly Compres...
Dell Software Group64
Exadata Hybrid Columnar Compression (EHCC)
Compression Unit (~<1M)
Block (8K) Block Block Block
Colu...
Dell Software Group65
Exadata Hybrid Columnar Compression
• Provides high
compression ratio
• Manageable impact
on row rea...
66
Dell Software Group
Trend #5:
The End of Disk?
Dell Software Group67
5MB HDD circa 1956
Dell Software Group68
The more that things change....
Dell Software Group69
Faster or slower?
260
1,635
-630
1,013
-390
-1,000 -500 0 500 1,000 1,500 2,000
IO Rate
Disk Capacit...
Dell Software Group70
Solid state disk to the rescue
• DDR RAM Drive
• SATA flash drive
• PCI flash drive
• SSD storage Se...
Dell Software Group71
Cheaper by the IO
4,000
80
25
15
0 1,000 2,000 3,000 4,000 5,000
Magnetic Disk
SSD SATA Flash
SSD PC...
Dell Software Group72
But not by the GB
0.35 0.28 0.21 0.17 0.13
2.9
2.2
1.7
1.3
1
10
7.4
5.3
3.2
2.3
0
2
4
6
8
10
12
2011...
Dell Software Group76
Tiered storage management
Main Memory
DDR SSD
Flash SSD
Fast Disk (SAS, RAID 0+1)
Slow Disk (SATA, R...
Dell Software Group77
In-Memory databases
• Cost of RAM falling
50% each 18 months.
• Some databases can
fit entirely with...
Dell Software Group78
Oracle Times Ten
• In-memory transactional database
• Disk-based Checkpoints and disk-
based logging...
Dell Software Group79
SAP Hana
Memory
Row Store
Column store
Delta store
Persistence Layer
Savepoints
Data files
Txn logs
...
Dell Software Group80
Exalytics
Instantaneous!
Dell Software Group81
You keep using that word….
I do not think it means what
you think it meansInigo Montoya
Dell Software Group82
Exalytics
Hardware:
• 2 TB RAM
• 4 10GBe , 2
InfiniBand ports
• 6x1.2TB SAS (7.2 TB)
• 3x800GB (2.4T...
Dell Software Group83
VoltDB
• Single threaded access
to memory: no
latch/mutex waits
• Transactions in self-
contained st...
Dell Software Group84
Spark (sort of) in-memory Hadoop
• In Memory compute
• HDFS compatible
• Libraries for data processi...
Dell Software Group85
Data files
Oracle 12c in-memory
Memory (SGA)
Row store Column Store (IMCU)
OLTP Analytics
(SMU)
data...
86
Dell Software Group
What does all this
mean for me?
87
Dell Software Group
Trend #6:
shameless product
plugs will increase over
the next 120 seconds
Dell Software Group88
Toad: your
companion
in the Big
Data
revolution
Dell Software Group89
Dell Statistica
Dell Software Group90
Dell In-Memory Appliances for Cloudera Enterprise
Mid-Size Configuration
16 Node Cluster
R720- 4 Inf...
Dell Software Group91
Dell appliances for any database
• Dell provides appliances and reference
architectures specifically...
Dell Software Group92
• Success in Big Data requires
capabilities at multiple
technology levels: hardware,
software infras...
93
Dell Software Group
Thank you.
Please complete the session
evaluation
We appreciate your feedback and insight
You may complete the session evaluation eit...
Upcoming SlideShare
Loading in …5
×

Five database trends - updated April 2015

16,139 views

Published on

Presentation given at Oracle open world 2014. Five trends in database technology including big data, ssd, in-memory, NoSQL and column stores

Published in: Technology
  • Unlock Her Legs(Official) $69 | Get 90% Off + 8 Special Bonus? ▲▲▲ http://scamcb.com/unlockher/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • We called it "operation mind control" - as we discovered a simple mind game that makes a girl become obsessed with you. (Aand it works even if you're not her type or she's already dating someone else) Here's how we figured it out... ★★★ http://t.cn/AiurDrZp
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Penis Enlargement and Enhancement Techniques: What REALLY Works?!? ♣♣♣ https://tinyurl.com/yy3nfggr
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • BIG semen gains, stronger, harder erections, and one happy G.F! ▲▲▲ https://tinyurl.com/semenax101
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Natural enlargement book for free? [Download] Your natural enlargement blueprint.  http://ishbv.com/pebible/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Five database trends - updated April 2015

  1. 1. REMINDER Check in on the COLLABORATE mobile app Top 5 Trends in Database Technology Guy Harrison, Executive Director, Information Mgt R&D, Dell Software Group Session ID#: 995 @guyharrison
  2. 2. Top 5 Trends in Database Technology Guy Harrison, Executive Director, Information Mgt R&D, Dell Software Group
  3. 3. Dell Software Group3 Web: guyharrison.net Email: guy.harrison@software.dell.com Twitter: @guyharrison Introductions
  4. 4. 4 Dell Software Group
  5. 5. 5 Dell Software Group
  6. 6. 6 Dell Software Group
  7. 7. 7 Dell Software Group Dell and Quest – a brief history
  8. 8. Dell Software Group8 But Seriously
  9. 9. Dell Software Group9 5 Database Technology Trends The end of “one size fits all” Big Data and Hadoop NoSQL Columnar architectures The end of disk?
  10. 10. 10 Dell Software Group Trend #1: The end of “one size fits all”
  11. 11. Dell Software Group11 History of databases Magnetic tape “flat” (sequential) files Pre-computer technologies: Printing press Dewey decimal system Punched cards Magnetic Disk IMS Relational Model defined Indexed-Sequential Access Mechanism (ISAM) Network Model IDMS ADABAS System R Oracle V2 Ingres dBase DB2 Informix Sybase SQL Server Access Postgres MySQL Cassandra Hadoop Vertica Riak HBase Dynamo MongoDB Redis VoltDB Hana Neo4J Aerospike Hierarchical model 1960-701940-50 1950-60 1970-80 1980-90 1990-2000 2000-2010
  12. 12. Dell Software Group12 Why? • 3rd Platform drives new demands on the database: – Global High Availability – Data volumes – Unstructured data – Transaction rates – Latency • A single architecture cannot meet all those demands
  13. 13. Dell Software Group13 Operational RDBMS (Oracle, SQL Server, …) In-memory Analytics (HANA, Exalytics …) In-memory processing (Spark) Hadoop Web DBMS (MySQL, Mongo, Cassandra) ERP & in- house CRM Analytic/BI software (SAS, Tableau) Web Server Data Warehouse RDBMS (Oracle, Terradata …) It takes all sorts
  14. 14. Dell Software Group14 Oracle engineered systems
  15. 15. 15 Dell Software Group Trend #2: Big Data and Hadoop
  16. 16. Dell Software Group16 The 3-4 “V”s Volume •Terabytes •Petabytes •Exabytes •Zetabytes Velocity •Transaction rates •User populations •Machines Variety •Structured •Unstructured •Human Generated •Machine Generated Value
  17. 17. 17 Dell Software Group The Industrial revolution of data
  18. 18. 18 Dell Software Group 2005
  19. 19. 19 Dell Software Group 2009
  20. 20. Dell Software Group20 The instrumented human • Bluetooth Personal Area Network • 3G/WiFi Wide Area Network • GPS • Storage • Pulse, temp monitor • Silent alarms • Pedometer, sleep monitoring • Compass • Camera • Mike/earphones • Heads up display • Emotion/Attention monitor
  21. 21. Dell Software Group21
  22. 22. Dell Software Group22 The instrumented world
  23. 23. Dell Software Group23 Big Data is the culmination of cloud, social and mobile
  24. 24. Dell Software Group24 More Data • Storing all data – including machine generated and sol, Social, community, demographic data in original format – for ever To More Effect • Smarter use of data (data science) to achieve competitive or human benefit
  25. 25. Dell Software Group25 More Data • Storing all data – including machine generated and sol, Social, community, demographic data in original format – for ever To More Effect • Smarter use of data (data science) to achieve competitive or human benefit
  26. 26. 26 Dell Software Group Pioneers of big data
  27. 27. 27 Dell Software Group
  28. 28. 28 Dell Software Group
  29. 29. 29 Dell Software Group
  30. 30. 30 Dell Software Group
  31. 31. 31 Dell Software Group
  32. 32. Dell Software Group32 Google File System (GFS) Map Reduce BigTable Google Applications Google Software Architecture (circa 2005)
  33. 33. Dell Software Group33 Start ReduceMap Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Map Reduce
  34. 34. Dell Software Group34 Hadoop: 1.0: Open Source Map-Reduce Stack
  35. 35. Dell Software Group35 Hadoop at Yahoo • 2010(biggest cluster): • 4000 nodes 16PB disk • 64 TB of RAM • 32,000 Cores • 2014: – 16 Clusters – 32,500 nodes
  36. 36. Dell Software Group36
  37. 37. Dell Software Group37 SQOOP (RDBMS loader) Hive (Query) Pig (Scripting) Flume (Log Loader) Oozie (Workflow manager) Hadoop File System (HDFS) Map Reduce / YARN Hbase (database) Zookeeper (locking) Hadoop family
  38. 38. Dell Software Group39 Economies $4,911 $750 $0 $1,000 $2,000 $3,000 $4,000 $5,000 $6,000 Exadata Hadoop Exadata vs Hadoop $$/TB (Hardware only)
  39. 39. Dell Software Group40 Hadoop is the most concrete Big Data technology Toad: your companion in the Big Data revolution
  40. 40. Dell Software Group41 More Data • Storing all data – including machine generated and sol, Social, community, demographic data in original format – for ever To More Effect • Smarter use of data (data science) to achieve competitive or human benefit
  41. 41. Dell Software Group42 More Data • Storing all data – including machine generated and sol, Social, community, demographic data in original format – for ever To More Effect • Smarter use of data (data science) to achieve competitive or human benefit
  42. 42. Dell Software Group43 Big Data Analytics AKA Data Science Machine Learning • Programs that evolve with “experience Predictive Analytics • Programs that extrapolate from past to future Collective Intelligence • Programs that use inputs from “crowds” to simulate intelligence
  43. 43. Dell Software Group44
  44. 44. Dell Software Group45 Collective Intelligence From now on, I’ll call you ‘An Ambulance’. OK? “Siri call me an ambulance”
  45. 45. 47 Dell Software Group Trend #3: NoSQL
  46. 46. Dell Software Group48 Web Servers Database Servers Memcached Servers Shard (G-O) Shard (P-Z)Shard (A-F) Read Only Slaves
  47. 47. Dell Software Group49 CAP Theorem says something has to give • CAP (Brewer’s) Theorem says you can only have two out of three of Consistency, Partition Tolerance, Availability Consistency • Everyone always sees the same data Availability • System stays up when nodes fail Partition Tolerance • System stays up when network between nodes fail Oracle RAC lives here NO GO Most NoSQL lives here
  48. 48. Dell Software Group50 Major influences on non-relational • Eventually consistent transaction model • Consistent hashing Amazon Dynamo • Column Family model for sparse distributed columnar data Google BigTable • Paved the way for the document database OODBMS and XML DBs
  49. 49. Dell Software Group51 Amazon Dynamo Model
  50. 50. Dell Software Group52 Name Site Counter Dick Ebay 507,018 Dick Google 690,414 Jane Google 716,426 Dick Facebook 723,649 Jane Facebook 643,261 Jane ILoveLarry.com 856,767 Dick MadBillFans.com 675,230 NameId Name 1 Dick 2 Jane SiteId SiteName 1 Ebay 2 Google 3 Facebook 4 ILoveLarry.com 5 MadBillFans.com NameId SiteId Counter 1 1 507,018 1 3 690,414 2 3 716,426 1 3 723,649 2 3 643,261 2 4 856,767 1 5 675,230 Id Name Ebay Google Facebook (other columns) MadBillFans.com 1 Dick 507,018 690,414 723,649 . . . . . . . . . . . . . . 675,230 Id Name Google Facebook (other columns) ILoveLarry.com 2 Jane 716,426 643,261 . . . . . . . . . . . . . . 856,767 BigTable Data Model
  51. 51. Dell Software Group53 OODBMS -1990s • The OODBMS Manifesto (Atkinson/Bancilhon/DeWitt/Dittrich/Maier/Zdo nik, '90) • "A relational database is like a garage that forces you to take your car apart and store the pieces in little drawers“ – Also SQL is ugly • “A Object database is like a closet which requires that you hang up your suit with tie, underwear, belt socks and shoes all attached” (Dave Ensor) http://4.bp.blogspot.com/- IPgd1Tg8ByE/UkOzH- g1FmI/AAAAAAAACB0/QYg8kE Vp5_0/s1600/db4o_vs_orm.png
  52. 52. Dell Software Group54 Revenge of the Object Nerds – Document databases • Structured documents – XML and JSON (JavaScript Object Notation) become more prevalent within applications • Web programmers start storing these in BLOBS in MySQL • Emergence of XML and JSON databases
  53. 53. Dell Software Group55 Graph Database Neo4J Infinite Graph FlockDB Document JSON based MongoDB CouchDB RethinkDB XML based MarkLogic BerkeleyDB XML Key Value Memchache DB Oracle NoSQL Dynamo Voldemort DynamoDB Riak Table Based BigTable Cassandra Hbase HyperTable Accumulo
  54. 54. Dell Software Group56 It’s not a database, it’s a key value store http://browsertoolkit.com/fault-tolerance.png
  55. 55. Dell Software Group57 No Means Yes!
  56. 56. 58 Dell Software Group Trend #4: Column-oriented DB
  57. 57. Dell Software Group59 Row orientation vs column orientation ID Name DOB Salary Sales Expenses 1001 Dick 21/12/60 67,000 78980 3244 1002 Jane 12/12/55 55,000 67840 2333 1003 Robert 17/02/80 22,000 67890 6436 1004 Dan 15/03/75 65,200 98770 2345 1005 Steven 11/11/81 76,000 43240 3214 Block ID Name DOB Salary Sales Expenses 1 1001 Dick 21/12/60 67,000 78980 3244 2 1002 Jane 12/12/55 55,000 67840 2333 3 1003 Robert 17/02/80 22,000 67890 6436 4 1004 Dan 15/03/75 65,200 98770 2345 5 1005 Steven 11/11/81 76,000 43240 3214 Block 1 Dick Jane Robert Dan Steven 2 21/12/60 12/12/55 17/02/80 15/03/75 11/11/81 3 67,000 55,000 22,000 65,200 76,000 4 78980 67840 67890 98770 43240 5 3244 2333 6436 2345 3214 Row oriented database Column oriented database
  58. 58. Dell Software Group60 Analytical Queries Block ID Name DOB Salary Sales Expenses 1 1001 Dick 21/12/60 67,000 78980 3244 2 1002 Jane 12/12/55 55,000 67840 2333 3 1003 Robert 17/02/80 22,000 67890 6436 4 1004 Dan 15/03/75 65,200 98770 2345 5 1005 Steven 11/11/81 76,000 43240 3214 Block 1 Dick Jane Robert Dan Steven 2 21/12/60 12/12/55 17/02/80 15/03/75 11/11/81 3 67,000 55,000 22,000 65,200 76,000 4 78980 67840 67890 98770 43240 5 3244 2333 6436 2345 3214 Row oriented database Column oriented database SELECT SUM(salary) FROM saleperson
  59. 59. Dell Software Group61 Compression Block ID Name DOB Salary Sales Expenses 1 1001 Dick 21/12/60 67,000 78980 3244 2 1002 Jane 12/12/55 55,000 67840 2333 3 1003 Robert 17/02/80 22,000 67890 6436 4 1004 Dan 15/03/75 65,200 98770 2345 5 1005 Steven 11/11/81 76,000 43240 3214 Block 1 Dick Jane Robert Dan Steven 2 21/12/60 12/12/55 17/02/80 15/03/75 11/11/81 3 67,000 55,000 22,000 65,200 76,000 4 78980 67840 67890 98770 43240 5 3244 2333 6436 2345 3214 Row oriented database Column oriented database Poor compression ratio (low repetition) Good compression ratio (high repetition)
  60. 60. Dell Software Group62 Inserts Block ID Name DOB Salary Sales Expenses 1 1001 Dick 21/12/60 67,000 78980 3244 2 1002 Jane 12/12/55 55,000 67840 2333 3 1003 Robert 17/02/80 22,000 67890 6436 4 1004 Dan 15/03/75 65,200 98770 2345 5 1005 Steven 11/11/81 76,000 43240 3214 Block 1 Dick Jane Robert Dan Steven 2 21/12/60 12/12/55 17/02/80 15/03/75 11/11/81 3 67,000 55,000 22,000 65,200 76,000 4 78980 67840 67890 98770 43240 5 3244 2333 6436 2345 3214 Row oriented database Column oriented database INSERT INTO salesperson
  61. 61. Dell Software Group63 C-Store (Vertica) Solution for inserts Read Optimized Store • Columnar • Disk-based • Highly Compressed • Bulk loadable Write Optimized Store • Row oriented • Uncompressed • Single row inserts Asynchronous Tuple Mover Bulk sequential loads Continual Parallel inserts Merged Query
  62. 62. Dell Software Group64 Exadata Hybrid Columnar Compression (EHCC) Compression Unit (~<1M) Block (8K) Block Block Block Column 1 Column 2 Column 3 Column 4 Row Row Row
  63. 63. Dell Software Group65 Exadata Hybrid Columnar Compression • Provides high compression ratio • Manageable impact on row read/write operations • Some optimization of analytic queries SELECT SUM(Column4) FROM table
  64. 64. 66 Dell Software Group Trend #5: The End of Disk?
  65. 65. Dell Software Group67 5MB HDD circa 1956
  66. 66. Dell Software Group68 The more that things change....
  67. 67. Dell Software Group69 Faster or slower? 260 1,635 -630 1,013 -390 -1,000 -500 0 500 1,000 1,500 2,000 IO Rate Disk Capacity IO/Capacity CPU IO/CPU %age change
  68. 68. Dell Software Group70 Solid state disk to the rescue • DDR RAM Drive • SATA flash drive • PCI flash drive • SSD storage Server
  69. 69. Dell Software Group71 Cheaper by the IO 4,000 80 25 15 0 1,000 2,000 3,000 4,000 5,000 Magnetic Disk SSD SATA Flash SSD PCI flash SSD DDR-RAM Seek time (us)
  70. 70. Dell Software Group72 But not by the GB 0.35 0.28 0.21 0.17 0.13 2.9 2.2 1.7 1.3 1 10 7.4 5.3 3.2 2.3 0 2 4 6 8 10 12 2011 2012 2013 2014 2015 $$/GB HDD MLC SDD SLC SSD 0.35 0.28 0.21 0.17 0.13 2.9 2.2 1.7 1.3 1 2.3 0.1 1 10 2011 2012 2013 2014 2015 $$/GB HDD MLC SDD SLC SSD
  71. 71. Dell Software Group76 Tiered storage management Main Memory DDR SSD Flash SSD Fast Disk (SAS, RAID 0+1) Slow Disk (SATA, RAID 5) Tape, Flat Files, Hadoop $/IOP $/GB
  72. 72. Dell Software Group77 In-Memory databases • Cost of RAM falling 50% each 18 months. • Some databases can fit entirely within the RAM of a single server or cluster of servers 0.001 0.01 0.1 1 10 100 $1.00 $10.00 $100.00 $1,000.00 $10,000.00 $100,000.00 1990 1995 2000 2005 2010 2015 2020 Size(GB) Cost(US$/GB) Year US$/GB Size (GB)
  73. 73. Dell Software Group78 Oracle Times Ten • In-memory transactional database • Disk-based Checkpoints and disk- based logging • By default, COMMITs are not durable (writes to the transaction log are asynchronous). • Can configure synchronous replication or synchronous log writes to avoid data loss • Columnar compression and analytic functions in the Exalytics version Clients Memory Checkpoints Transaction Logs Commits Point in time snapshot
  74. 74. Dell Software Group79 SAP Hana Memory Row Store Column store Delta store Persistence Layer Savepoints Data files Txn logs Note: Table must be either row or column – not both
  75. 75. Dell Software Group80 Exalytics Instantaneous!
  76. 76. Dell Software Group81 You keep using that word…. I do not think it means what you think it meansInigo Montoya
  77. 77. Dell Software Group82 Exalytics Hardware: • 2 TB RAM • 4 10GBe , 2 InfiniBand ports • 6x1.2TB SAS (7.2 TB) • 3x800GB (2.4TB) SSD Software: • Oracle BI • ESSBase • Oracle R • Times-Ten • 12c In-memory
  78. 78. Dell Software Group83 VoltDB • Single threaded access to memory: no latch/mutex waits • Transactions in self- contained stored procedures: minimal locking • K-Safety for COMMIT: No sync waits CPU In-memory Partition CPU In-memory Partition CPU In-memory Partition CPU In-memory Partition CPU In-memory Partition CPU In-memory Partition Clients Clients Clients
  79. 79. Dell Software Group84 Spark (sort of) in-memory Hadoop • In Memory compute • HDFS compatible • Libraries for data processing, machine learning, streaming, SQL, etc • Python and Scala interfaces • Part of the Berkeley Data Analytic Stack • Integrating into all Hadoop distributions (and Cassandra) HDFS Tachyon – in memory File system Spark: in-memory distributed compute Spark Streaming Mlib Machine Learning SparkSQL Mesos Cluster manager
  80. 80. Dell Software Group85 Data files Oracle 12c in-memory Memory (SGA) Row store Column Store (IMCU) OLTP Analytics (SMU) database Column store Redo Logs
  81. 81. 86 Dell Software Group What does all this mean for me?
  82. 82. 87 Dell Software Group Trend #6: shameless product plugs will increase over the next 120 seconds
  83. 83. Dell Software Group88 Toad: your companion in the Big Data revolution
  84. 84. Dell Software Group89 Dell Statistica
  85. 85. Dell Software Group90 Dell In-Memory Appliances for Cloudera Enterprise Mid-Size Configuration 16 Node Cluster R720- 4 Infrastructure Nodes R720XD- 12 Data Nodes Force10- S4810P Force10- S55 ~528TB (disk raw space) ~4.5 TB (raw memory) Starter Configuration 8 Node Cluster R720- 4 Infrastructure Nodes R720XD- 4 Data Nodes Force10- S55 ~176TB (disk raw space) ~1.5TB (raw memory) Small Enterprise Configuration 24 Node Cluster R720- 4 Infrastructure Nodes R720XD- 20 Data Nodes ~880TB (disk raw space) ~7.5 TB (raw memory) Expansion Unit- R720XD-4 Data, Cloudera Enterprise Data Hub, Scale in Blocks
  86. 86. Dell Software Group91 Dell appliances for any database • Dell provides appliances and reference architectures specifically designed for: – Oracle – SQL Server – HANA – SSD database acceleration – Large memory footprints
  87. 87. Dell Software Group92 • Success in Big Data requires capabilities at multiple technology levels: hardware, software infrastructure, business intelligence and analytics • Only Dell can deliver capabilities at every technology layer • Only Dell’s solutions are designed and priced to suit mid-market initial deployments and to scale to the largest enterprise Data Integration Hadoop and database software Advanced Analytics Business Intelligence Server and Storage Boomi Boomi, Toad Intelligence Central Dell appliances for Hadoop, Oracle, etc Dell servers and storage arrays Toad Data point Statistica Systems Management Dell Foglight and TOAD Big Data for the rest of us
  88. 88. 93 Dell Software Group Thank you.
  89. 89. Please complete the session evaluation We appreciate your feedback and insight You may complete the session evaluation either on paper or online via the mobile app

×