Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

In-memory Database and MySQL Cluster


Published on

Published in: Technology, Education
  • Be the first to comment

In-memory Database and MySQL Cluster

  1. 1. In-memory Database & MySQL Cluster Grandis He ( Xiongwei He (Grandis) 2010-10-18 1
  2. 2. Personal Introduction • Before immigration to Australia – Lead Zero Downtime Upgrade Feature for Alcatel Lucent Subscriber Data Management (MySQL Application Year 2009 Award) – Lead Super Distributed Home Location Register (SDHLR) developer team which used Oracle TimesTen for 7 years Xiongwei He (Grandis) 2010-10-18 2
  3. 3. In-memory Database Development • Before Y2000 – Vendor DIY • NO SQL Support and limited Search Option • Not easy for management – Alternative choice: Berkley DB (Key-Value pair) • After Y2000 – Independent Vendors – SQL/ODBC/JDBC Support – Easy for management • Now – Major database vendors (Except Microsoft) have in memory options by purchasing or self-development • Market Value for In-memory Database: SAP acquired Sybase – One major reason mentioned in SAP PR is: Sybase In Memory Database Xiongwei He (Grandis) 2010-10-18 3
  4. 4. Different ways to be in-memory • In-memory only database (or called as diskless) – Data in Memory only • In-memory cache to database – Data will be synced to database which sync to disk • In-memory database – Data will be written to disk Note: Only for products with SQL Support 2010-10-18 Xiongwei He (Grandis) 4
  5. 5. In-memory Only Database (Diskless) • Mainstream products • Java Open Source DB – Oracle TimesTen – HyperSQL (HSQLDB) – MySQL Cluster – Apache Derby – IBM SolidDB – H2 – Sybase ASE IMDB • Typical usage – Session management – automatic generated data store such as GPS location data store of Smartphone Xiongwei He (Grandis) 2010-10-18 5
  6. 6. In-memory Cache to Database • Main products – SolidDB Universal Cache • Support DB2, Informix, Oracle, Sybase and Microsoft – Oracle In Memory Cache (Renamed from TimesTen Cache) • Only support Oracle • Advantage: No change to existing applications and optimized some applications with real time speed • Cost: Cache License + Database License • Possible motivation for Oracle to buy TimesTen and IBM to buy SoildDB Xiongwei He (Grandis) 2010-10-18 6
  7. 7. In Memory Database • Abbreviation: IMDB • Another name: Main Memory Database (MMDB) • Now it is close to disk based database for operational convenience while holding data in the memory • Real-time speed to access database (Always use 10 times faster for advertisement) • Main products – Oracle TimesTen – MySQL Cluster – IBM SolidDB – Sybase ASE RDDB Option Xiongwei He (Grandis) 2010-10-18 7
  8. 8. In Memory Database Features • FULL Database are in memory, Query will not trigger Disk IO • ACID – Non-duration for fast performance. (Some databases also provide durable option, some databases do NOT) • Low Level API to access DB beside JDBC/ODBC • Low Latency  Super Speed for Database Access – Speed in microseconds or 2-5 milliseconds – Among Select/Update/Insert, select is fastest, then update, then insert (latency might be 10 times for select) • High Throughput • High Availability (HA) Support Xiongwei He (Grandis) 2010-10-18 8
  9. 9. Features for IMDB Selection • 2 slides planned to be presented in Sydney Meetup Xiongwei He (Grandis) 2010-10-18 9
  10. 10. MySQL Cluster • 2003 Acquired Alzato – Ericsson venture • Another name: NDB Cluster • Use different version: MySQL Cluster version is different from MySQL Server version • Cost: Cheap license in comparison to TimesTen Xiongwei He (Grandis) 2010-10-18 10
  11. 11. MySQL Cluster Architecture Application Application 1. 2-Phase Commit between Data Nodes 3 2. Replication between Application Application MySQL MySQL MySQL Cluster NDB Native API NDB Native API Server Server 3. Standard MySQL Server Interface Data 1 Data 3 Management Data N-1 (MGM) Node 1 Data 2 Data 4 Data N 1 3 N-1 MySQL 2 MySQL 2 4 N Server Server Xiongwei He (Grandis) 2010-10-18 11
  12. 12. MySQL Cluster Nodes • Management Node (MGM Node): Node for Data Management (ndb_mgmd) – Multiple management node supported – Management clients (ndb_mgm): the client manage MGM node – Why there is MGM Node? The usage for MGM API • Data Node (Node Group, Replica and Partition) • SQL Node – MySQL Server Node – NDB API Node Xiongwei He (Grandis) 2010-10-18 12
  13. 13. Node Groups, Replica and Partition • Data Node (NDB Node): The node running ndbd or ndbmtd (multithread version) which stores a replica. – Each data node is sharing the load and responsible for 1/number of Replica data in data group for fast processing when all the data nodes in data group are active. • Replica: Copy of a cluster partition. The number of replicas is equal to number of nodes per group • Node Groups: A Node Group consists of 1-4 Nodes storing same set of data for reliability. • Partition: Automatically by Key and Linear Key Xiongwei He (Grandis) 2010-10-18 13
  14. 14. MySQL Cluster Replication • Replication latency is little longer than TimesTen/ SolidDB due to Distributed Architecture • Support 2-way replication but personally suggest: – Use 2-way replication when the update operations to cluster 1 and cluster 2 are using different keys (for example, odd to cluster 1, even to cluster 2) – Suggest only use 1-way replication for most applications • Conflict Resolution is NOT easy for complex scenarios • Latency due to distributed architecture Xiongwei He (Grandis) 2010-10-18 14
  15. 15. MySQL Cluster Features • Low Cost – Use commodity hardware without disk array • High reliable – Shared nothing (better than Shared Disk Array and Mirror for maintenance) in single cluster – Geo Redundancy Support by Cluster Level Replication • High performance/frequency (especially with NDB API) • Distributed for application access • Low Latency: 2-5 ms • Disk Field Support: Address the issues for memory limitation when application need support big field (Major advantage to other IMDBs) Xiongwei He (Grandis) 2010-10-18 15
  16. 16. Commit, GCP, LCP • Commit: commit to all the replicas (But in memory only until GCP happen) • Global Checkpoint (GCP): A GCP occurs every few seconds, when transactions for all nodes are synchronized and the redo-log is flush to disk • Local Checkpoint (LCP). This is a checkpoint that is specific to a single node. An LCP involves saving all of a node's data to disk, and so usually occurs every few minutes. Xiongwei He (Grandis) 2010-10-18 16
  17. 17. Commit, GCP, LCP • NDB GCP ~= Commit in Disk-based database • NDB LCP ~= Checkpoint in Disk-based database • LCP performance (full database flush) – NOT good as Checkpoint in Disk-based database or TimesTen which flush dirty pages only – Mitigation: Distributed architecture to make disk I/O reduced on single data node Xiongwei He (Grandis) 2010-10-18 17
  18. 18. GCP and LCP for NDB Recovery • NDB Recovery: – Load LCP – Load GCP • Why need Global Synchronization: Make the whole cluster data in consistence for recovery – Lose committed transaction in memory for database crash • Mitigation for data safety: Use multiple replicas “internal driving factors” for distributed architecture with multiple replica support Xiongwei He (Grandis) 2010-10-18 18
  19. 19. Programming Interface • Distributed nature using multiple MySQL Nodes and NDB API Nodes – Why: Transaction control by NDB Data Nodes • The choice – Standard MySQL Clients – NDB API (Best way for high performance) • Single Table Operation • Can not access triggers but NDB Event Xiongwei He (Grandis) 2010-10-18 19
  20. 20. Programming Interface • Java Interface – MySQL JDBC:Use Connnector/J 5.1.7+ for load balancing support – ClusterJ for Java -- Java interface based on NDBAPI – ClusterJPA – OpenJPA Implementation which take advantage of JDBC for complex query and ClusterJ for single table operation • LDAP Interface Support (Based on NDB API) – Impressive Performance – Data Store for OpenLDAP and OpenDS Xiongwei He (Grandis) 2010-10-18 20
  21. 21. Your Options • Using NDB API or NDB API originated interface (Max Performance with development Cost) • Using MySQL Interface (Best development efficiency) • LDAP (Depend on your application type) Xiongwei He (Grandis) 2010-10-18 21
  22. 22. System Architecture Input • Performance/Reliability Requirement – System Volume/Throughput/Latency – Disk Mirrored or NOT – Redundancy Model • Node Redundancy (Example: N+K (K=1 or 2) for SQL Node, 1 or 2 Data Nodes in different data group is down) • TCP/IP Redundancy (Ethernet port + WAN/LAN Network Redundancy) – CPU budget for busy hours (related to redundancy mode) • Memory/Disk per subscriber (per order) – Memory usage per subscriber (per order) – Disk usage per subscriber (per order) or Disk/Memory Rate • Disk I/O Performance/Behavior (MySQL Cluster – LCP) • Replication – WAN Budget for Geo Redundancy – Replication Daemon Throughput – Replication Latency Xiongwei He (Grandis) 2010-10-18 22
  23. 23. Your Adjustment • Hardware Key Indicator – CPU/Memory/Disk (Speed/Volume) – Switch/Router • Hardware/WAN Adjustment – CPU/Memory/Disk – NDB Node Number – Network Configuration (LAN) – Router and WAN bandwidth request • Software Adjustment – Data Model Design Adjustment – Move “INSERT/DELETE” action to non-busy hours if possible – Service/configuration data local caching (MySQL Cluster – using NDB Event) – Others such as Optimized Software System Pattern/Design Xiongwei He (Grandis) 2010-10-18 23
  24. 24. Hardware Environment • Using VMWare for Test and even for functionality demo to customers • Using Non-ATCA Blade (IBM/HP/Oracle/Dell) or ATCA for Performance Test – 1G/10Gb Ethernet @ backplane – NDB Node: Not necessary for multiple CPU but need SAS Disk if insert/update/delete take significant percentage in the transactions – MySQL Cluster Node for replication: Do not turn on Intel Hyper Thread or use Sun CMT CPU for fast replication Xiongwei He (Grandis) 2010-10-18 24
  25. 25. Upgrade • MySQL Cluster without Cluster-Level Replication • MySQL Cluster with Cluster-Level Replication Xiongwei He (Grandis) 2010-10-18 25
  26. 26. Single MySQL Cluster Upgrade (One example) • 4 Data Groups (2 replicas), 2 MGM Node – Stop front end application – Backup the cluster data – Split the node into 2 clusters • Old cluster, cluster 1: MGM1, NDB1, NDB3, NDB5, NDB7 • New cluster, cluster 2: MGM2, NDB2, NDB4, NDB6, NDB8 – Upgrade NDB and schema of cluster 2 – Let front end application connect to cluster 2 – Change cluster 1 to be part of cluster 2 Xiongwei He (Grandis) 2010-10-18 26
  27. 27. MySQL Cluster Upgrade with replication support • Database version: NDB 7.1 – NDB 7.1 Feature: attribute promotion/demotion support for replication – NDB 7.0 Feature: Default value for new adding columns in tables • Environment: Cluster 1 (Active) and Cluster 2 (Standby) • Procedures – Upgrade on cluster 2 (NO Service Interrupt) • Upgrade NDB version and apply schema changes • Cluster1Cluster2 replication verification – Switch cluster 2 to be active • Cluster2Cluster1 replication verification • If new version can NOT work, switch back cluster 1 – Upgrade on cluster 1 (Upgrade NDB version and apply application schema changes) • Carrier Upgrade Procedure Difference (Briefing in Sydney Meetup) 2010-10-18 Xiongwei He (Grandis) 27
  28. 28. Limitation of MySQL Cluster • NO Foreign Key Constraints • Transaction Limitation – NO savepoints – Read commit isolation level • NO Durable Commits – Mitigation: increase replica to 3 or 4 if you want extreme reliability Xiongwei He (Grandis) 2010-10-18 28
  29. 29. Take care in memory database (Beyond MySQL Cluster) • Possible slow startup in comparison to disk-based database (impact to MTTR for single cluster case) – Need load all the data to memory • Feature not fully deployed as disk-based database and but improving • Insure the query/update with key – Please do not perform “select count(*) from $table_name” in live machines if it is NOT tested. Find solution from database dictionary – Non-key search in transaction might make database busy which will let application hung and trigger outage Xiongwei He (Grandis) 2010-10-18 29
  30. 30. Suggestion to MySQL Cluster Release Selection • Small idea to be shared and discussed on Sydney Meetup Xiongwei He (Grandis) 2010-10-18 30