Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
ClustrixDB
@ Samsung Cloud
Kwangbock Lee
Lead Database Architect
Samsung Electronics
Agenda
1. Introduction of Samsung Cloud Platform
2. Requirements & Features
3. Samsung Cloud + ClustrixDB Journey
4. Issue...
User Benefit
Backup and restore
data and settings
Your photos on multiple
devices any time
15 GB of free storage,
Upgrade ...
Figures of Samsung Cloud
Hundreds of millions
Members
Tens of billions
Daily Request
Hundreds of PiB
Storage
ClustrixDB
Ca...
Samsung Cloud Architecture
Data Processing
Layer Backend Modules
Service Modules
Basic Modules User Modules
Data Layer
API...
User Architecture – Before Migration
Master
Slaves
Master
Slaves
Shard Info
…
Shard #1 Shard #2
Key Challenges
● RDBMS Scaling Strategy
○ Sharding Overhead
○ Migration Overhead
○ Additional Codes for both sharding & mi...
Requirements & Clustrix Features
● Scalability, No more Sharding!
● ACID Compliant
● MySQL Compatible
● Fault Tolerance, N...
Key Features of ClustrixDB
Scalability
● Scalable Architecture
○ Can scale linearly as nodes are added
○ Automatically dis...
Key Features of ClustrixDB
Fault-Tolerant
● Built-in Fault Tolerance can endure a single node failure and automatically
ma...
Key Features of ClustrixDB
Online Schema Change
● No blocking read or writes to a table
○ Requires more space to run
● Dis...
Samsung Cloud + ClustrixDB Journey
From PoC to Expansion
PoC
ClustrixDB
v7.6
2016
Pre-Launch
Workshop
ClustrixDB
v8.x
2017...
Issues & Enhancements
Replication Configuration with MySQL 5.7
● For Migraion Deployment
● MySQL 5.7(master) – ClustrixDB ...
Issues & Enhancements
Fast Backup and Restore
√ Fast Backup and Restore as a binary
backup mechanism
√ Each node sends its...
Issues & Enhancements
Replication Performance
● Write Intensive Workload
○ Replication Gap increasing
○ Binlogging Perform...
Issues & Enhancements
Enhanced Security
● SSL
○ Supports SSL Encrypted Connections
○ Requires a mysql client 5.6.38 or hig...
Issues & Enhancements
Monitoring Tools
● Built-in Monitoring tool - ClustrixGUI
● Network security policy blocks using Clu...
Current Architecture
Architecture #1
Zone 1 Zone 2 Zone 3
Master Slave
ClustrixDBClustrixDB
Architecture #2
MAX_FAILURES =...
Current Deployment & Usage
Region #2
Region #1
Region #3
M SS
M SS
230 Million
TPS
16 Billion
Rows
2 Services
3 Regions
 No Additional Resources for
Migration or Sharding
 Downsized Instance Spec.
 No Standby Replicas for HA,
Backup, Analy...
Wrap Up
● Future Work
○ BINLOG / Replication Enhancement
○ ETL Tools
● Q&A
THANK YOU!
ClustrixDB at Samsung Cloud
Upcoming SlideShare
Loading in …5
×

ClustrixDB at Samsung Cloud

309 views

Published on

In this day and age, data grows so fast it’s not uncommon for those of us using a relational database to reach the limits of its capacity. In this session, Kwangbock Lee explains how Samsung uses ClustrixDB to handle fast-growing data without manual database sharding. He highlights lessons learned, including a few hiccups along the way, and shares Samsung's experience migrating to ClustrixDB.

Published in: Software
  • Real Ways To Make Money, Most online opportunities are nothing but total scams! ■■■ http://scamcb.com/ezpayjobs/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

ClustrixDB at Samsung Cloud

  1. 1. ClustrixDB @ Samsung Cloud Kwangbock Lee Lead Database Architect Samsung Electronics
  2. 2. Agenda 1. Introduction of Samsung Cloud Platform 2. Requirements & Features 3. Samsung Cloud + ClustrixDB Journey 4. Issues & Enhancements 5. Wrap Up
  3. 3. User Benefit Backup and restore data and settings Your photos on multiple devices any time 15 GB of free storage, Upgrade for more - Home screen, App data, Contact, Messages, Device settings, Music, Documents, etc. - Sync photos, videos, notes using native applications across Samsung devices - Premium Plans . Korea, 29 countries in EU (’16, Nov) . US models (excl. VZW. ATT, ’17, Feb) . Brazil (unlock devices, ‘18, Mar) * No. 1 request from customers
  4. 4. Figures of Samsung Cloud Hundreds of millions Members Tens of billions Daily Request Hundreds of PiB Storage ClustrixDB Cassandra MySQL DynamoDB
  5. 5. Samsung Cloud Architecture Data Processing Layer Backend Modules Service Modules Basic Modules User Modules Data Layer API Gateway Application Layer Access Layer ClustrixDB Cassandra
  6. 6. User Architecture – Before Migration Master Slaves Master Slaves Shard Info … Shard #1 Shard #2
  7. 7. Key Challenges ● RDBMS Scaling Strategy ○ Sharding Overhead ○ Migration Overhead ○ Additional Codes for both sharding & migration ● High Availability ● Analytic Query ○ Need to run the query in every Shard DB and merge it. ● Online Schema Change ● Online Backup / Restore
  8. 8. Requirements & Clustrix Features ● Scalability, No more Sharding! ● ACID Compliant ● MySQL Compatible ● Fault Tolerance, No SPOF! ● OLTP and Operational Analytics ● Online Schema Change ● Online Backup / Restore ● Scalable ● High-Volume, High Concurrent OLTP ● Automatic Data Distribution ● Distributed Query Execution ● Fault-Tolerant ● Flexible Deployment Options ● MySQL Compatible ● Easy to Migrate from MySQL ● Fast Backup and Restore Requirements Clustrix Features
  9. 9. Key Features of ClustrixDB Scalability ● Scalable Architecture ○ Can scale linearly as nodes are added ○ Automatically distributes both data and query execution to scale ○ Flex Up & Flex Down ● Rebalancer ○ Automatically manage the distribution of data for the cluster ○ Read/Write imbalance across node/zones (ranking replica)
  10. 10. Key Features of ClustrixDB Fault-Tolerant ● Built-in Fault Tolerance can endure a single node failure and automatically maintain 2 copies of all data ● Replication ● Deploying Across Zones ○ AWS Availability Zones (requires 3 AZ) ● MAX_FAILURES ○ Number of failures that can occur simultaneously ○ ALTER CLUSTER SET MAX_FAILURES = number of simultaneous node failures
  11. 11. Key Features of ClustrixDB Online Schema Change ● No blocking read or writes to a table ○ Requires more space to run ● Distributed Parallel Query Execution – FANOUT option ○ query_fanout ○ query_fanout_insert_select ○ query_fanout_all_writes ● Monitoring the Process of an ALTER ○ system.alter_progress
  12. 12. Samsung Cloud + ClustrixDB Journey From PoC to Expansion PoC ClustrixDB v7.6 2016 Pre-Launch Workshop ClustrixDB v8.x 2017 Go-Live ClustrixDB v9.0 2017 Expansion ClustrixDB v9.1 2018
  13. 13. Issues & Enhancements Replication Configuration with MySQL 5.7 ● For Migraion Deployment ● MySQL 5.7(master) – ClustrixDB (slave) PoC ClustrixDB v7.6 2016 Master Slave ClustrixDB
  14. 14. Issues & Enhancements Fast Backup and Restore √ Fast Backup and Restore as a binary backup mechanism √ Each node sends its data directly to the backup target in parallel √ Provides SFTP for Backup and Restore √ Can control concurrency ClustrixDB PoC ClustrixDB v7.6 2016 FTP Server Secure FTP
  15. 15. Issues & Enhancements Replication Performance ● Write Intensive Workload ○ Replication Gap increasing ○ Binlogging Performance low Pre-Launch Workshop ClustrixDB v8.x 2017 Master ClustrixDB Slave ClustrixDB √ Zones (v9.x)
  16. 16. Issues & Enhancements Enhanced Security ● SSL ○ Supports SSL Encrypted Connections ○ Requires a mysql client 5.6.38 or higher ● SHA256 Password Plugin ○ Provides strong user password credentials than mysql_native_password plugin ● Audit (User Logging) ○ Provides audit logs of user login/logout (user.log) ○ SET GLOBAL session_log_users = true; Expansion ClustrixDB v9.1 2018
  17. 17. Issues & Enhancements Monitoring Tools ● Built-in Monitoring tool - ClustrixGUI ● Network security policy blocks using ClustrixGUI ● Need long-term historical data √ Monitoring with InfluxDB & Grafana ○ Collector script ○ Grafana dashboard √ Other tools are available Expansion ClustrixDB v9.1 2018
  18. 18. Current Architecture Architecture #1 Zone 1 Zone 2 Zone 3 Master Slave ClustrixDBClustrixDB Architecture #2 MAX_FAILURES = 2 REPLICAS = 3 MAX_FAILURES = 1 REPLICAS = 2 ClustrixDB
  19. 19. Current Deployment & Usage Region #2 Region #1 Region #3 M SS M SS 230 Million TPS 16 Billion Rows 2 Services 3 Regions
  20. 20.  No Additional Resources for Migration or Sharding  Downsized Instance Spec.  No Standby Replicas for HA, Backup, Analytics  Less Man-Month  Easy Scalability  No SPOF, Strong HA  Better Maintenance & Monitoring  Analytic Query  Tech Support  Simplified Application Architecture  No Additional Code for Migration or Sharding  Focus on Service Logic Development Benefits Operation Cost
  21. 21. Wrap Up ● Future Work ○ BINLOG / Replication Enhancement ○ ETL Tools ● Q&A
  22. 22. THANK YOU!

×