#SGvFabric   © 2011 VMware Inc. All rights reserved
vFabric: What’s in it?                         Rich                                   Integration   Batch            Sprin...
Cloud-scale challenge…3
Challenge Managing on-line applications on a cloud-scale is hard. As number of users grows, database becomes the bottlenec...
DB Bottleneck                Scales…                Scales…  5
Cause Traditional databases were never designed to support thousands of concurrent users. 6
Traditional DB Characteristics § Designed against no            § Centralized in nature      longer relevant            ...
Traditional DB Loves IO      Buffers primarily        tuned for IO                          First write                   ...
Transaction in Traditional DB                                        Source: Research by MIT and Brown: “OLTP Under       ...
Cloud-scale solution…10
Apparent Choices Build expensive database clustering solution or lengthy re-write for “big data”? 11
Next generation optionSQLFire is different;it’s build for speed     Scale                         much?                   ...
New Approach Elastic, in-memory database designed specifically for speed and low latency accessible through a familiar SQL...
SQLFire Characteristics §  Highly concurrent data       §  Shared nothing logs on   structures resident in and      disk...
SQLFire speed…15
SQLFire v Traditional Databases SQLFire response times are faster and more consistent under increased database load.  16
Sample Comparison §  Spring Travel Application §  Similar hardware (8 vCPU, 4GB) §  Out-of-the-box configuration       ...
Response Time  R/T 200 180 160                    MySQL 140          increased with load 120 100  80                      ...
Number of Threads  R/T                                                             SQLFire scales 1200	                   ...
Distributed data…20
Why Scale Horizontally? Sub-divide system into independent data sets, eliminate distributed transactions to achieve elasti...
Horizontal Scalability – Throughput                      800000                                    1400                   ...
Horizontal Scalability – Consistency/HA §  Resiliency through replication, synchronous but in parallel §  Row updates ar...
Data management strategies…24
Data strategies – Partitioning §  Balances data across SQLFire cluster §  Delivers redundancy for high availability     ...
SQLFire Hash Partitioning §  Partition by column or primary key     •  Can specify multiple columns     •  Uses hashCode(...
SQLFire Range Partitioning §  Partition by range of column values     •  Can specify multiple ranges     •  Colocates dat...
SQLFire List Partitioning §  Partition by a set of column values     •  Can specify column value sets     •  Colocates da...
SQLFire Expression Partitioning §  Partition by a column expression     •  Expression must be valid SQL function     •  M...
SQLFire Default Partitioning §  Default hash partitioning strategy     •  Start server with table-default-partitioned pro...
Data strategies – Replication §  Copies all data across SQLFire cluster §  Appropriate for reference data               ...
SQLFire Replicated Tables §  Created by default with no PARTITION BY clause §  Created with REPLICATE clause §  Referen...
Topology strategies…33
Topology Client-server                           JVM                     JVM                           APP                ...
Topology Embedded Peer-to-peer            JVM           JVM       JVM            APP           APP       APP           SQL...
Synchronization strategies…36
Synchronous strategy In data-center or over private network            JVM                 JVM                       JVM  ...
Asynchronous strategyMulti-site over the Cloud           JVM                 JVM                                 JVM      ...
Data strategies – Server Groups                              SQLFire Cluster          JVM              JVM             JVM...
data demo…40
Summary…41
Why SQLFire?               In-memory, delivers maximum speedSpeed          and minimum latency  Scale               Horizo...
If you forgot everything else… SQLFire is better in supporting on-line applications than traditional databases.  43
SQLFire Artifacts              Sample Apps              §  Side-by-side comparison of SQLFire v MySQL                    ...
The end     http://vmware.com/go/sqlfire     @vFabricSQLFire, @_cmc45
Upcoming SlideShare
Loading in …5
×

Modernización del manejo de datos con v fabric

531 views

Published on

En esta sesión aprenderemos como podemos habilitar y administrar bases de datos en la nube por medio de vFabric.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
531
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Modernización del manejo de datos con v fabric

  1. 1. #SGvFabric © 2011 VMware Inc. All rights reserved
  2. 2. vFabric: What’s in it? Rich Integration Batch Spring Frameworks & Tools Web Social and Data Patterns Framework Tool Suite Mobile Access vFabric Application Srv Web Runtime Messaging Elastic Data Grid DBaaS Perf, Mgmt Application Services tc Server ERS vPostgres RabbitMQ Gemfire / SQLFire Hyperic / Insight EM4J Data vCops/ Director APM vSphere 5 2
  3. 3. Cloud-scale challenge…3
  4. 4. Challenge Managing on-line applications on a cloud-scale is hard. As number of users grows, database becomes the bottleneck. 4
  5. 5. DB Bottleneck Scales… Scales… 5
  6. 6. Cause Traditional databases were never designed to support thousands of concurrent users. 6
  7. 7. Traditional DB Characteristics § Designed against no § Centralized in nature longer relevant • Data change capture an constraints afterthought • Network unreliable/slow • Lacks data partitioning • RAM prices prohibitive facilities § One size fits all § Obsessed with ACID • Designed for everything, • Constant contention for optimized for nothing resources cause locks • Often incompatible with § Monolithic design modern workloads §  Requires lots of hardware to scale 7
  8. 8. Traditional DB Loves IO Buffers primarily tuned for IO First write to LOG Second write to Data files 8
  9. 9. Transaction in Traditional DB Source: Research by MIT and Brown: “OLTP Under the Looking Glass” by S. Harizopoulos, D. J. Abadi, S. Madden, M. Stonebraker, SIGMOD 2008. 12% 30% 8% Data Percentage of Btrees keys Computer cycles Logging based on 3.5M sample Locking 21% Latching 10% Buffer management 19% 9
  10. 10. Cloud-scale solution…10
  11. 11. Apparent Choices Build expensive database clustering solution or lengthy re-write for “big data”? 11
  12. 12. Next generation optionSQLFire is different;it’s build for speed Scale much? Hablo SQL?and scale.12
  13. 13. New Approach Elastic, in-memory database designed specifically for speed and low latency accessible through a familiar SQL interface. 13
  14. 14. SQLFire Characteristics §  Highly concurrent data §  Shared nothing logs on structures resident in and disk; application writes are optimized for main memory never exposed to the disk §  Rethink ACID transactions; seek latencies all state resides in §  Parallelize data access distributed memory to avoid and application behavior; any single points of dynamically “shard SQL” contention §  Dynamic rebalancing of §  Partition-aware DB design data as cluster size grows/ spreads workloads across shrinks. Most efficient way both data set and physical of managing resources/ nodes data. 14
  15. 15. SQLFire speed…15
  16. 16. SQLFire v Traditional Databases SQLFire response times are faster and more consistent under increased database load. 16
  17. 17. Sample Comparison §  Spring Travel Application §  Similar hardware (8 vCPU, 4GB) §  Out-of-the-box configuration SQLF R/T (ms) SQLF CPU % MySQL R/T (ms) MYSQL CPU % 14 9 25 1 8 32 23 19 5 61 172 76 6 77 fail fail 984 98 fail fail 17
  18. 18. Response Time R/T 200 180 160 MySQL 140 increased with load 120 100 80 SQLFire near constant 60 much lower 40 20 0 Threads 0 500 1000 1500 2000 18
  19. 19. Number of Threads R/T SQLFire scales 1200   to 7200 threads with 1 second R/T 1000   800   MySQL reaches saturation 600   at 1850 threads 400   200   0   0   1000   2000   3000   4000   5000   6000   7000   8000   Threads 19
  20. 20. Distributed data…20
  21. 21. Why Scale Horizontally? Sub-divide system into independent data sets, eliminate distributed transactions to achieve elasticity, linear scalability and predictable latency. 21
  22. 22. Horizontal Scalability – Throughput 800000 1400 700000 1200 600000 Queries per second 1000 Client threads 500000 800 400000 queriesPerSecond 600 client threads 300000 400 200000 200 100000 0 0 2 4 6 8 10 Number of servers 22
  23. 23. Horizontal Scalability – Consistency/HA §  Resiliency through replication, synchronous but in parallel §  Row updates are always atomic; no need for transactions §  Shared nothing architecture, including storage §  Instant failover at protocol level §  Apps retain their connections §  Data remains available APP SQLFire SQLFire SQLFire 23
  24. 24. Data management strategies…24
  25. 25. Data strategies – Partitioning §  Balances data across SQLFire cluster §  Delivers redundancy for high availability APP SQLFire SQLFire SQLFire Write operation (with 2 redundant copies) Read operation 25
  26. 26. SQLFire Hash Partitioning §  Partition by column or primary key •  Can specify multiple columns •  Uses hashCode() for single column or primary key •  Uses serialized bytes for multiple columns •  Creates uniform distribution of data across the cluster // Partition by column CREATE TABLE MY_TABLE ( . . . ) PARTITION BY COLUMN ( COLUMN_A) // Partition by primary key CREATE TABLE MY_TABLE ( . . . ) PARTITION BY PRIMARY KEY 26
  27. 27. SQLFire Range Partitioning §  Partition by range of column values •  Can specify multiple ranges •  Colocates data in specified ranges •  Used to ensure locality of data in a partition for range queries or cross table joins // Partition by range CREATE TABLE MY_TABLE ( . . . ) PARTITION BY RANGE ( COLUMN_A) ( VALUES BETWEEN 1 AND 10, VALUES BETWEEN 50 AND 60 ) 27
  28. 28. SQLFire List Partitioning §  Partition by a set of column values •  Can specify column value sets •  Colocates data with specified column values •  Used to ensure locality of data in a partition for sets of values or cross table joins // Partition by list CREATE TABLE MY_TABLE ( . . . ) PARTITION BY LIST ( COLUMN_A) ( VALUES (‘VALUE_A’, ‘VALUE_B’), VALUES (‘VALUE_Y’, ‘VALUE_Z’) ) 28
  29. 29. SQLFire Expression Partitioning §  Partition by a column expression •  Expression must be valid SQL function •  Must reference only columns in the table •  Hash partition with value determined by the expression // Partition by expression CREATE TABLE MY_TABLE ( . . . ) PARTITION BY ( MONTH ( MY_DATE ) ) 29
  30. 30. SQLFire Default Partitioning §  Default hash partitioning strategy •  Start server with table-default-partitioned property set to true! •  First foreign key whose referenced primary key is also a partition column •  Primary key •  First unique key •  SQLFire-generated row id // No PARTITION BY clauses CREATE TABLE MY_TABLE (COLUMN_A INT NOT NULL CONSTRAINT A_PK PRIMARY KEY, . . .) CREATE TABLE MY_OTHER_TABLE (COLUMN_B INT NOT NULL CONSTRAINT B_PK PRIMARY KEY, COLUMN_C INT CONSTRAINT A_FK REFERENCES MY_TABLE (COLUMN_A), . . .) 30
  31. 31. Data strategies – Replication §  Copies all data across SQLFire cluster §  Appropriate for reference data APP SQLFire SQLFire SQLFire Write operation (with replicated copies) Read operation 31
  32. 32. SQLFire Replicated Tables §  Created by default with no PARTITION BY clause §  Created with REPLICATE clause §  Reference data or fact tables are good candidates §  Replicates data across all peers in server group §  Replication is parallel and synchronous §  Automatic replication failure detection // Replication example CREATE TABLE MY_TABLE ( . . . ) REPLICATE 32
  33. 33. Topology strategies…33
  34. 34. Topology Client-server JVM JVM APP APP SQLFire Locator JVM JVM JVM SQLFire SQLFire SQLFire 34
  35. 35. Topology Embedded Peer-to-peer JVM JVM JVM APP APP APP SQLFire SQLFire SQLFire 35
  36. 36. Synchronization strategies…36
  37. 37. Synchronous strategy In data-center or over private network JVM JVM JVM JVM APP APP APP APP SQLFire Locator SQLFire Locator JVM JVM JVM JVM JVM JVM SQLFire SQLFire SQLFire SQLFire SQLFire SQLFire Redundancy Zone A Redundancy Zone B Site 1 Site 2 37
  38. 38. Asynchronous strategyMulti-site over the Cloud JVM JVM JVM JVM APP APP APP APP SQLFire Locator WAN SQLFire Locator Gateway JVM JVM JVM JVM JVM JVMSQLFire SQLFire SQLFire SQLFire SQLFire SQLFire Site 1 Site 238
  39. 39. Data strategies – Server Groups SQLFire Cluster JVM JVM JVM Group 1 SQLFire SQLFire SQLFire JVM JVM JVM Group 2 SQLFire SQLFire SQLFire Group 3 39
  40. 40. data demo…40
  41. 41. Summary…41
  42. 42. Why SQLFire? In-memory, delivers maximum speedSpeed and minimum latency Scale Horizontally scalable, easily adopts to changing workloads, usage patterns SQL Familiar SQL interface, accessible from Java and .NET 42
  43. 43. If you forgot everything else… SQLFire is better in supporting on-line applications than traditional databases. 43
  44. 44. SQLFire Artifacts Sample Apps §  Side-by-side comparison of SQLFire v MySQL performance - https://github.com/vFabric/sqlf-demo §  Demo call-center application, SQLFire configuration scripts https://github.com/vFabric/sqlf-cloud Demo Video §  Real-life performance comparison (YouTube, 3 min.) http://youtu.be/HV-broQHJlk 44
  45. 45. The end http://vmware.com/go/sqlfire @vFabricSQLFire, @_cmc45

×