ClustrixDB at Samsung Cloud

MariaDB plc
MariaDB plcMariaDB plc
ClustrixDB
@ Samsung Cloud
Kwangbock Lee
Lead Database Architect
Samsung Electronics
Agenda
1. Introduction of Samsung Cloud Platform
2. Requirements & Features
3. Samsung Cloud + ClustrixDB Journey
4. Issues & Enhancements
5. Wrap Up
ClustrixDB at Samsung Cloud
User Benefit
Backup and restore
data and settings
Your photos on multiple
devices any time
15 GB of free storage,
Upgrade for more
- Home screen, App data, Contact, Messages,
Device settings, Music, Documents, etc.
- Sync photos, videos, notes using native
applications across Samsung devices
- Premium Plans
. Korea, 29 countries in EU (’16, Nov)
. US models (excl. VZW. ATT, ’17, Feb)
. Brazil (unlock devices, ‘18, Mar)
* No. 1 request from customers
Figures of Samsung Cloud
Hundreds of millions
Members
Tens of billions
Daily Request
Hundreds of PiB
Storage
ClustrixDB
Cassandra
MySQL
DynamoDB
Samsung Cloud Architecture
Data Processing
Layer Backend Modules
Service Modules
Basic Modules User Modules
Data Layer
API Gateway
Application Layer
Access Layer
ClustrixDB Cassandra
User Architecture – Before Migration
Master
Slaves
Master
Slaves
Shard Info
…
Shard #1 Shard #2
Key Challenges
● RDBMS Scaling Strategy
○ Sharding Overhead
○ Migration Overhead
○ Additional Codes for both sharding & migration
● High Availability
● Analytic Query
○ Need to run the query in every Shard DB and merge it.
● Online Schema Change
● Online Backup / Restore
Requirements & Clustrix Features
● Scalability, No more Sharding!
● ACID Compliant
● MySQL Compatible
● Fault Tolerance, No SPOF!
● OLTP and Operational Analytics
● Online Schema Change
● Online Backup / Restore
● Scalable
● High-Volume, High Concurrent OLTP
● Automatic Data Distribution
● Distributed Query Execution
● Fault-Tolerant
● Flexible Deployment Options
● MySQL Compatible
● Easy to Migrate from MySQL
● Fast Backup and Restore
Requirements Clustrix Features
Key Features of ClustrixDB
Scalability
● Scalable Architecture
○ Can scale linearly as nodes are added
○ Automatically distributes both data and query execution to scale
○ Flex Up & Flex Down
● Rebalancer
○ Automatically manage the distribution of data for the cluster
○ Read/Write imbalance across node/zones (ranking replica)
Key Features of ClustrixDB
Fault-Tolerant
● Built-in Fault Tolerance can endure a single node failure and automatically
maintain 2 copies of all data
● Replication
● Deploying Across Zones
○ AWS Availability Zones (requires 3 AZ)
● MAX_FAILURES
○ Number of failures that can occur simultaneously
○ ALTER CLUSTER SET MAX_FAILURES = number of simultaneous node failures
Key Features of ClustrixDB
Online Schema Change
● No blocking read or writes to a table
○ Requires more space to run
● Distributed Parallel Query Execution – FANOUT option
○ query_fanout
○ query_fanout_insert_select
○ query_fanout_all_writes
● Monitoring the Process of an ALTER
○ system.alter_progress
Samsung Cloud + ClustrixDB Journey
From PoC to Expansion
PoC
ClustrixDB
v7.6
2016
Pre-Launch
Workshop
ClustrixDB
v8.x
2017
Go-Live
ClustrixDB
v9.0
2017
Expansion
ClustrixDB
v9.1
2018
Issues & Enhancements
Replication Configuration with MySQL 5.7
● For Migraion Deployment
● MySQL 5.7(master) – ClustrixDB (slave)
PoC
ClustrixDB
v7.6
2016
Master Slave
ClustrixDB
Issues & Enhancements
Fast Backup and Restore
√ Fast Backup and Restore as a binary
backup mechanism
√ Each node sends its data directly to the
backup target in parallel
√ Provides SFTP for Backup and Restore
√ Can control concurrency
ClustrixDB
PoC
ClustrixDB
v7.6
2016
FTP Server
Secure FTP
Issues & Enhancements
Replication Performance
● Write Intensive Workload
○ Replication Gap increasing
○ Binlogging Performance low
Pre-Launch
Workshop
ClustrixDB
v8.x
2017
Master
ClustrixDB
Slave
ClustrixDB
√ Zones (v9.x)
Issues & Enhancements
Enhanced Security
● SSL
○ Supports SSL Encrypted Connections
○ Requires a mysql client 5.6.38 or higher
● SHA256 Password Plugin
○ Provides strong user password credentials than mysql_native_password plugin
● Audit (User Logging)
○ Provides audit logs of user login/logout (user.log)
○ SET GLOBAL session_log_users = true;
Expansion
ClustrixDB
v9.1
2018
Issues & Enhancements
Monitoring Tools
● Built-in Monitoring tool - ClustrixGUI
● Network security policy blocks using ClustrixGUI
● Need long-term historical data
√ Monitoring with InfluxDB & Grafana
○ Collector script
○ Grafana dashboard
√ Other tools are available
Expansion
ClustrixDB
v9.1
2018
Current Architecture
Architecture #1
Zone 1 Zone 2 Zone 3
Master Slave
ClustrixDBClustrixDB
Architecture #2
MAX_FAILURES = 2
REPLICAS = 3
MAX_FAILURES = 1
REPLICAS = 2
ClustrixDB
Current Deployment & Usage
Region #2
Region #1
Region #3
M SS
M SS
230 Million
TPS
16 Billion
Rows
2 Services
3 Regions
 No Additional Resources for
Migration or Sharding
 Downsized Instance Spec.
 No Standby Replicas for HA,
Backup, Analytics
 Less Man-Month
 Easy Scalability
 No SPOF, Strong HA
 Better Maintenance &
Monitoring
 Analytic Query
 Tech Support
 Simplified Application
Architecture
 No Additional Code for
Migration or Sharding
 Focus on Service Logic
Development
Benefits
Operation Cost
Wrap Up
● Future Work
○ BINLOG / Replication Enhancement
○ ETL Tools
● Q&A
THANK YOU!
1 of 23

Recommended

6.distributed shared memory by
6.distributed shared memory6.distributed shared memory
6.distributed shared memoryGd Goenka University
4.6K views46 slides
Google Bigtable Paper Presentation by
Google Bigtable Paper PresentationGoogle Bigtable Paper Presentation
Google Bigtable Paper Presentationvanjakom
11.2K views28 slides
Sql vs NoSQL by
Sql vs NoSQLSql vs NoSQL
Sql vs NoSQLRTigger
114.7K views17 slides
Query processing and optimization (updated) by
Query processing and optimization (updated)Query processing and optimization (updated)
Query processing and optimization (updated)Ravinder Kamboj
26K views31 slides
Introduction to Amazon Aurora by
Introduction to Amazon AuroraIntroduction to Amazon Aurora
Introduction to Amazon AuroraAmazon Web Services
5.2K views38 slides
The Google File System (GFS) by
The Google File System (GFS)The Google File System (GFS)
The Google File System (GFS)Romain Jacotin
36.4K views52 slides

More Related Content

What's hot

Database replication by
Database replicationDatabase replication
Database replicationArslan111
940 views13 slides
Data Warehousing with Amazon Redshift by
Data Warehousing with Amazon RedshiftData Warehousing with Amazon Redshift
Data Warehousing with Amazon RedshiftAmazon Web Services
4.4K views48 slides
Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb... by
Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...
Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...Amazon Web Services
911 views34 slides
Cloud Service Models by
Cloud Service ModelsCloud Service Models
Cloud Service ModelsAbhishek Pachisia
2.6K views18 slides
Building Serverless ETL Pipelines by
Building Serverless ETL PipelinesBuilding Serverless ETL Pipelines
Building Serverless ETL PipelinesAmazon Web Services
1.9K views42 slides
Caching by
CachingCaching
CachingNascenia IT
8.3K views29 slides

What's hot(20)

Database replication by Arslan111
Database replicationDatabase replication
Database replication
Arslan111940 views
Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb... by Amazon Web Services
Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...
Best practices for Data warehousing with Amazon Redshift - AWS PS Summit Canb...
Chapter 5 database security by Syaiful Ahdan
Chapter 5   database securityChapter 5   database security
Chapter 5 database security
Syaiful Ahdan2K views
Google File System by guest2cb4689
Google File SystemGoogle File System
Google File System
guest2cb468910.4K views
Relational databases vs Non-relational databases by James Serra
Relational databases vs Non-relational databasesRelational databases vs Non-relational databases
Relational databases vs Non-relational databases
James Serra22.7K views
Mongodb basics and architecture by Bishal Khanal
Mongodb basics and architectureMongodb basics and architecture
Mongodb basics and architecture
Bishal Khanal2.1K views
Salvatore Sanfilippo – How Redis Cluster works, and why - NoSQL matters Barce... by NoSQLmatters
Salvatore Sanfilippo – How Redis Cluster works, and why - NoSQL matters Barce...Salvatore Sanfilippo – How Redis Cluster works, and why - NoSQL matters Barce...
Salvatore Sanfilippo – How Redis Cluster works, and why - NoSQL matters Barce...
NoSQLmatters10.1K views

Similar to ClustrixDB at Samsung Cloud

Automating using Ansible by
Automating using AnsibleAutomating using Ansible
Automating using AnsibleAlok Patra
241 views23 slides
NetflixOSS Meetup season 3 episode 1 by
NetflixOSS Meetup season 3 episode 1NetflixOSS Meetup season 3 episode 1
NetflixOSS Meetup season 3 episode 1Ruslan Meshenberg
21.4K views90 slides
Db2 analytics accelerator on ibm integrated analytics system technical over... by
Db2 analytics accelerator on ibm integrated analytics system   technical over...Db2 analytics accelerator on ibm integrated analytics system   technical over...
Db2 analytics accelerator on ibm integrated analytics system technical over...Daniel Martin
3.4K views26 slides
Cassandra Summit 2015 - Building a multi-tenant API PaaS with DataStax Enterp... by
Cassandra Summit 2015 - Building a multi-tenant API PaaS with DataStax Enterp...Cassandra Summit 2015 - Building a multi-tenant API PaaS with DataStax Enterp...
Cassandra Summit 2015 - Building a multi-tenant API PaaS with DataStax Enterp...Restlet
3.1K views34 slides
Introducing the ultimate MariaDB cloud, SkySQL by
Introducing the ultimate MariaDB cloud, SkySQLIntroducing the ultimate MariaDB cloud, SkySQL
Introducing the ultimate MariaDB cloud, SkySQLMariaDB plc
470 views37 slides
ClustrixDB: how distributed databases scale out by
ClustrixDB: how distributed databases scale outClustrixDB: how distributed databases scale out
ClustrixDB: how distributed databases scale outMariaDB plc
1.9K views28 slides

Similar to ClustrixDB at Samsung Cloud(20)

Automating using Ansible by Alok Patra
Automating using AnsibleAutomating using Ansible
Automating using Ansible
Alok Patra241 views
NetflixOSS Meetup season 3 episode 1 by Ruslan Meshenberg
NetflixOSS Meetup season 3 episode 1NetflixOSS Meetup season 3 episode 1
NetflixOSS Meetup season 3 episode 1
Ruslan Meshenberg21.4K views
Db2 analytics accelerator on ibm integrated analytics system technical over... by Daniel Martin
Db2 analytics accelerator on ibm integrated analytics system   technical over...Db2 analytics accelerator on ibm integrated analytics system   technical over...
Db2 analytics accelerator on ibm integrated analytics system technical over...
Daniel Martin3.4K views
Cassandra Summit 2015 - Building a multi-tenant API PaaS with DataStax Enterp... by Restlet
Cassandra Summit 2015 - Building a multi-tenant API PaaS with DataStax Enterp...Cassandra Summit 2015 - Building a multi-tenant API PaaS with DataStax Enterp...
Cassandra Summit 2015 - Building a multi-tenant API PaaS with DataStax Enterp...
Restlet3.1K views
Introducing the ultimate MariaDB cloud, SkySQL by MariaDB plc
Introducing the ultimate MariaDB cloud, SkySQLIntroducing the ultimate MariaDB cloud, SkySQL
Introducing the ultimate MariaDB cloud, SkySQL
MariaDB plc470 views
ClustrixDB: how distributed databases scale out by MariaDB plc
ClustrixDB: how distributed databases scale outClustrixDB: how distributed databases scale out
ClustrixDB: how distributed databases scale out
MariaDB plc1.9K views
Introduction to ClustrixDB by I Goo Lee
Introduction to ClustrixDBIntroduction to ClustrixDB
Introduction to ClustrixDB
I Goo Lee596 views
Instaclustr Kafka Meetup Sydney Presentation by Ben Slater
Instaclustr Kafka Meetup Sydney PresentationInstaclustr Kafka Meetup Sydney Presentation
Instaclustr Kafka Meetup Sydney Presentation
Ben Slater1.3K views
Getting started with amazon redshift - Toronto by Amazon Web Services
Getting started with amazon redshift - TorontoGetting started with amazon redshift - Toronto
Getting started with amazon redshift - Toronto
Amazon Web Services3.1K views
Scalability strategies for cloud based system architecture by Brandon Kang
Scalability strategies for cloud based system architectureScalability strategies for cloud based system architecture
Scalability strategies for cloud based system architecture
Brandon Kang281 views
What to expect from MariaDB Platform X5, part 2 by MariaDB plc
What to expect from MariaDB Platform X5, part 2What to expect from MariaDB Platform X5, part 2
What to expect from MariaDB Platform X5, part 2
MariaDB plc429 views
Study Notes - Architecting for the cloud (AWS Best Practices, Feb 2016) by Rick Hwang
Study Notes - Architecting for the cloud (AWS Best Practices, Feb 2016)Study Notes - Architecting for the cloud (AWS Best Practices, Feb 2016)
Study Notes - Architecting for the cloud (AWS Best Practices, Feb 2016)
Rick Hwang771 views
Big Data on Cloud Native Platform by Sunil Govindan
Big Data on Cloud Native PlatformBig Data on Cloud Native Platform
Big Data on Cloud Native Platform
Sunil Govindan127 views
Big Data on Cloud Native Platform by Sunil Govindan
Big Data on Cloud Native PlatformBig Data on Cloud Native Platform
Big Data on Cloud Native Platform
Sunil Govindan23 views
Scaling Security on 100s of Millions of Mobile Devices Using Apache Kafka® an... by confluent
Scaling Security on 100s of Millions of Mobile Devices Using Apache Kafka® an...Scaling Security on 100s of Millions of Mobile Devices Using Apache Kafka® an...
Scaling Security on 100s of Millions of Mobile Devices Using Apache Kafka® an...
confluent803 views
Massive Lift & Shift Migrations to Microsoft Azure with the Microsoft Migrati... by Morgan Simonsen
Massive Lift & Shift Migrations to Microsoft Azure with the Microsoft Migrati...Massive Lift & Shift Migrations to Microsoft Azure with the Microsoft Migrati...
Massive Lift & Shift Migrations to Microsoft Azure with the Microsoft Migrati...
Morgan Simonsen5.6K views

More from MariaDB plc

MariaDB Paris Workshop 2023 - MaxScale 23.02.x by
MariaDB Paris Workshop 2023 - MaxScale 23.02.xMariaDB Paris Workshop 2023 - MaxScale 23.02.x
MariaDB Paris Workshop 2023 - MaxScale 23.02.xMariaDB plc
11 views17 slides
MariaDB Paris Workshop 2023 - Newpharma by
MariaDB Paris Workshop 2023 - NewpharmaMariaDB Paris Workshop 2023 - Newpharma
MariaDB Paris Workshop 2023 - NewpharmaMariaDB plc
24 views19 slides
MariaDB Paris Workshop 2023 - Cloud by
MariaDB Paris Workshop 2023 - CloudMariaDB Paris Workshop 2023 - Cloud
MariaDB Paris Workshop 2023 - CloudMariaDB plc
12 views24 slides
MariaDB Paris Workshop 2023 - MariaDB Enterprise by
MariaDB Paris Workshop 2023 - MariaDB EnterpriseMariaDB Paris Workshop 2023 - MariaDB Enterprise
MariaDB Paris Workshop 2023 - MariaDB EnterpriseMariaDB plc
40 views32 slides
MariaDB Paris Workshop 2023 - Performance Optimization by
MariaDB Paris Workshop 2023 - Performance OptimizationMariaDB Paris Workshop 2023 - Performance Optimization
MariaDB Paris Workshop 2023 - Performance OptimizationMariaDB plc
36 views33 slides
MariaDB Paris Workshop 2023 - MaxScale by
MariaDB Paris Workshop 2023 - MaxScale MariaDB Paris Workshop 2023 - MaxScale
MariaDB Paris Workshop 2023 - MaxScale MariaDB plc
32 views15 slides

More from MariaDB plc(20)

MariaDB Paris Workshop 2023 - MaxScale 23.02.x by MariaDB plc
MariaDB Paris Workshop 2023 - MaxScale 23.02.xMariaDB Paris Workshop 2023 - MaxScale 23.02.x
MariaDB Paris Workshop 2023 - MaxScale 23.02.x
MariaDB plc11 views
MariaDB Paris Workshop 2023 - Newpharma by MariaDB plc
MariaDB Paris Workshop 2023 - NewpharmaMariaDB Paris Workshop 2023 - Newpharma
MariaDB Paris Workshop 2023 - Newpharma
MariaDB plc24 views
MariaDB Paris Workshop 2023 - Cloud by MariaDB plc
MariaDB Paris Workshop 2023 - CloudMariaDB Paris Workshop 2023 - Cloud
MariaDB Paris Workshop 2023 - Cloud
MariaDB plc12 views
MariaDB Paris Workshop 2023 - MariaDB Enterprise by MariaDB plc
MariaDB Paris Workshop 2023 - MariaDB EnterpriseMariaDB Paris Workshop 2023 - MariaDB Enterprise
MariaDB Paris Workshop 2023 - MariaDB Enterprise
MariaDB plc40 views
MariaDB Paris Workshop 2023 - Performance Optimization by MariaDB plc
MariaDB Paris Workshop 2023 - Performance OptimizationMariaDB Paris Workshop 2023 - Performance Optimization
MariaDB Paris Workshop 2023 - Performance Optimization
MariaDB plc36 views
MariaDB Paris Workshop 2023 - MaxScale by MariaDB plc
MariaDB Paris Workshop 2023 - MaxScale MariaDB Paris Workshop 2023 - MaxScale
MariaDB Paris Workshop 2023 - MaxScale
MariaDB plc32 views
MariaDB Paris Workshop 2023 - novadys presentation by MariaDB plc
MariaDB Paris Workshop 2023 - novadys presentationMariaDB Paris Workshop 2023 - novadys presentation
MariaDB Paris Workshop 2023 - novadys presentation
MariaDB plc24 views
MariaDB Paris Workshop 2023 - DARVA presentation by MariaDB plc
MariaDB Paris Workshop 2023 - DARVA presentationMariaDB Paris Workshop 2023 - DARVA presentation
MariaDB Paris Workshop 2023 - DARVA presentation
MariaDB plc18 views
MariaDB Tech und Business Update Hamburg 2023 - MariaDB Enterprise Server by MariaDB plc
MariaDB Tech und Business Update Hamburg 2023 - MariaDB Enterprise Server MariaDB Tech und Business Update Hamburg 2023 - MariaDB Enterprise Server
MariaDB Tech und Business Update Hamburg 2023 - MariaDB Enterprise Server
MariaDB plc31 views
MariaDB SkySQL Autonome Skalierung, Observability, Cloud-Backup by MariaDB plc
MariaDB SkySQL Autonome Skalierung, Observability, Cloud-BackupMariaDB SkySQL Autonome Skalierung, Observability, Cloud-Backup
MariaDB SkySQL Autonome Skalierung, Observability, Cloud-Backup
MariaDB plc31 views
Einführung : MariaDB Tech und Business Update Hamburg 2023 by MariaDB plc
Einführung : MariaDB Tech und Business Update Hamburg 2023Einführung : MariaDB Tech und Business Update Hamburg 2023
Einführung : MariaDB Tech und Business Update Hamburg 2023
MariaDB plc39 views
Hochverfügbarkeitslösungen mit MariaDB by MariaDB plc
Hochverfügbarkeitslösungen mit MariaDBHochverfügbarkeitslösungen mit MariaDB
Hochverfügbarkeitslösungen mit MariaDB
MariaDB plc48 views
Die Neuheiten in MariaDB Enterprise Server by MariaDB plc
Die Neuheiten in MariaDB Enterprise ServerDie Neuheiten in MariaDB Enterprise Server
Die Neuheiten in MariaDB Enterprise Server
MariaDB plc12 views
Global Data Replication with Galera for Ansell Guardian® by MariaDB plc
Global Data Replication with Galera for Ansell Guardian®Global Data Replication with Galera for Ansell Guardian®
Global Data Replication with Galera for Ansell Guardian®
MariaDB plc2.5K views
Introducing workload analysis by MariaDB plc
Introducing workload analysisIntroducing workload analysis
Introducing workload analysis
MariaDB plc906 views
Under the hood: SkySQL monitoring by MariaDB plc
Under the hood: SkySQL monitoringUnder the hood: SkySQL monitoring
Under the hood: SkySQL monitoring
MariaDB plc558 views
Introducing the R2DBC async Java connector by MariaDB plc
Introducing the R2DBC async Java connectorIntroducing the R2DBC async Java connector
Introducing the R2DBC async Java connector
MariaDB plc766 views
MariaDB Enterprise Tools introduction by MariaDB plc
MariaDB Enterprise Tools introductionMariaDB Enterprise Tools introduction
MariaDB Enterprise Tools introduction
MariaDB plc575 views
Faster, better, stronger: The new InnoDB by MariaDB plc
Faster, better, stronger: The new InnoDBFaster, better, stronger: The new InnoDB
Faster, better, stronger: The new InnoDB
MariaDB plc651 views
The architecture of SkySQL by MariaDB plc
The architecture of SkySQLThe architecture of SkySQL
The architecture of SkySQL
MariaDB plc587 views

Recently uploaded

Page Object Model by
Page Object ModelPage Object Model
Page Object Modelartembondar5
6 views5 slides
predicting-m3-devopsconMunich-2023-v2.pptx by
predicting-m3-devopsconMunich-2023-v2.pptxpredicting-m3-devopsconMunich-2023-v2.pptx
predicting-m3-devopsconMunich-2023-v2.pptxTier1 app
12 views33 slides
How To Make Your Plans Suck Less — Maarten Dalmijn at the 57th Hands-on Agile... by
How To Make Your Plans Suck Less — Maarten Dalmijn at the 57th Hands-on Agile...How To Make Your Plans Suck Less — Maarten Dalmijn at the 57th Hands-on Agile...
How To Make Your Plans Suck Less — Maarten Dalmijn at the 57th Hands-on Agile...Stefan Wolpers
42 views38 slides
nintendo_64.pptx by
nintendo_64.pptxnintendo_64.pptx
nintendo_64.pptxpaiga02016
6 views7 slides
predicting-m3-devopsconMunich-2023.pptx by
predicting-m3-devopsconMunich-2023.pptxpredicting-m3-devopsconMunich-2023.pptx
predicting-m3-devopsconMunich-2023.pptxTier1 app
8 views24 slides
Top-5-production-devconMunich-2023.pptx by
Top-5-production-devconMunich-2023.pptxTop-5-production-devconMunich-2023.pptx
Top-5-production-devconMunich-2023.pptxTier1 app
9 views40 slides

Recently uploaded(20)

predicting-m3-devopsconMunich-2023-v2.pptx by Tier1 app
predicting-m3-devopsconMunich-2023-v2.pptxpredicting-m3-devopsconMunich-2023-v2.pptx
predicting-m3-devopsconMunich-2023-v2.pptx
Tier1 app12 views
How To Make Your Plans Suck Less — Maarten Dalmijn at the 57th Hands-on Agile... by Stefan Wolpers
How To Make Your Plans Suck Less — Maarten Dalmijn at the 57th Hands-on Agile...How To Make Your Plans Suck Less — Maarten Dalmijn at the 57th Hands-on Agile...
How To Make Your Plans Suck Less — Maarten Dalmijn at the 57th Hands-on Agile...
Stefan Wolpers42 views
predicting-m3-devopsconMunich-2023.pptx by Tier1 app
predicting-m3-devopsconMunich-2023.pptxpredicting-m3-devopsconMunich-2023.pptx
predicting-m3-devopsconMunich-2023.pptx
Tier1 app8 views
Top-5-production-devconMunich-2023.pptx by Tier1 app
Top-5-production-devconMunich-2023.pptxTop-5-production-devconMunich-2023.pptx
Top-5-production-devconMunich-2023.pptx
Tier1 app9 views
Understanding HTML terminology by artembondar5
Understanding HTML terminologyUnderstanding HTML terminology
Understanding HTML terminology
artembondar57 views
Ports-and-Adapters Architecture for Embedded HMI by Burkhard Stubert
Ports-and-Adapters Architecture for Embedded HMIPorts-and-Adapters Architecture for Embedded HMI
Ports-and-Adapters Architecture for Embedded HMI
Burkhard Stubert33 views
How Workforce Management Software Empowers SMEs | TraQSuite by TraQSuite
How Workforce Management Software Empowers SMEs | TraQSuiteHow Workforce Management Software Empowers SMEs | TraQSuite
How Workforce Management Software Empowers SMEs | TraQSuite
TraQSuite6 views
Dapr Unleashed: Accelerating Microservice Development by Miroslav Janeski
Dapr Unleashed: Accelerating Microservice DevelopmentDapr Unleashed: Accelerating Microservice Development
Dapr Unleashed: Accelerating Microservice Development
Miroslav Janeski15 views
Automated Testing of Microsoft Power BI Reports by RTTS
Automated Testing of Microsoft Power BI ReportsAutomated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI Reports
RTTS10 views
Mobile App Development Company by Richestsoft
Mobile App Development CompanyMobile App Development Company
Mobile App Development Company
Richestsoft 5 views
tecnologia18.docx by nosi6702
tecnologia18.docxtecnologia18.docx
tecnologia18.docx
nosi67025 views
Supercharging your Python Development Environment with VS Code and Dev Contai... by Dawn Wages
Supercharging your Python Development Environment with VS Code and Dev Contai...Supercharging your Python Development Environment with VS Code and Dev Contai...
Supercharging your Python Development Environment with VS Code and Dev Contai...
Dawn Wages5 views
How to build dyanmic dashboards and ensure they always work by Wiiisdom
How to build dyanmic dashboards and ensure they always workHow to build dyanmic dashboards and ensure they always work
How to build dyanmic dashboards and ensure they always work
Wiiisdom14 views
ADDO_2022_CICID_Tom_Halpin.pdf by TomHalpin9
ADDO_2022_CICID_Tom_Halpin.pdfADDO_2022_CICID_Tom_Halpin.pdf
ADDO_2022_CICID_Tom_Halpin.pdf
TomHalpin95 views

ClustrixDB at Samsung Cloud

  • 1. ClustrixDB @ Samsung Cloud Kwangbock Lee Lead Database Architect Samsung Electronics
  • 2. Agenda 1. Introduction of Samsung Cloud Platform 2. Requirements & Features 3. Samsung Cloud + ClustrixDB Journey 4. Issues & Enhancements 5. Wrap Up
  • 4. User Benefit Backup and restore data and settings Your photos on multiple devices any time 15 GB of free storage, Upgrade for more - Home screen, App data, Contact, Messages, Device settings, Music, Documents, etc. - Sync photos, videos, notes using native applications across Samsung devices - Premium Plans . Korea, 29 countries in EU (’16, Nov) . US models (excl. VZW. ATT, ’17, Feb) . Brazil (unlock devices, ‘18, Mar) * No. 1 request from customers
  • 5. Figures of Samsung Cloud Hundreds of millions Members Tens of billions Daily Request Hundreds of PiB Storage ClustrixDB Cassandra MySQL DynamoDB
  • 6. Samsung Cloud Architecture Data Processing Layer Backend Modules Service Modules Basic Modules User Modules Data Layer API Gateway Application Layer Access Layer ClustrixDB Cassandra
  • 7. User Architecture – Before Migration Master Slaves Master Slaves Shard Info … Shard #1 Shard #2
  • 8. Key Challenges ● RDBMS Scaling Strategy ○ Sharding Overhead ○ Migration Overhead ○ Additional Codes for both sharding & migration ● High Availability ● Analytic Query ○ Need to run the query in every Shard DB and merge it. ● Online Schema Change ● Online Backup / Restore
  • 9. Requirements & Clustrix Features ● Scalability, No more Sharding! ● ACID Compliant ● MySQL Compatible ● Fault Tolerance, No SPOF! ● OLTP and Operational Analytics ● Online Schema Change ● Online Backup / Restore ● Scalable ● High-Volume, High Concurrent OLTP ● Automatic Data Distribution ● Distributed Query Execution ● Fault-Tolerant ● Flexible Deployment Options ● MySQL Compatible ● Easy to Migrate from MySQL ● Fast Backup and Restore Requirements Clustrix Features
  • 10. Key Features of ClustrixDB Scalability ● Scalable Architecture ○ Can scale linearly as nodes are added ○ Automatically distributes both data and query execution to scale ○ Flex Up & Flex Down ● Rebalancer ○ Automatically manage the distribution of data for the cluster ○ Read/Write imbalance across node/zones (ranking replica)
  • 11. Key Features of ClustrixDB Fault-Tolerant ● Built-in Fault Tolerance can endure a single node failure and automatically maintain 2 copies of all data ● Replication ● Deploying Across Zones ○ AWS Availability Zones (requires 3 AZ) ● MAX_FAILURES ○ Number of failures that can occur simultaneously ○ ALTER CLUSTER SET MAX_FAILURES = number of simultaneous node failures
  • 12. Key Features of ClustrixDB Online Schema Change ● No blocking read or writes to a table ○ Requires more space to run ● Distributed Parallel Query Execution – FANOUT option ○ query_fanout ○ query_fanout_insert_select ○ query_fanout_all_writes ● Monitoring the Process of an ALTER ○ system.alter_progress
  • 13. Samsung Cloud + ClustrixDB Journey From PoC to Expansion PoC ClustrixDB v7.6 2016 Pre-Launch Workshop ClustrixDB v8.x 2017 Go-Live ClustrixDB v9.0 2017 Expansion ClustrixDB v9.1 2018
  • 14. Issues & Enhancements Replication Configuration with MySQL 5.7 ● For Migraion Deployment ● MySQL 5.7(master) – ClustrixDB (slave) PoC ClustrixDB v7.6 2016 Master Slave ClustrixDB
  • 15. Issues & Enhancements Fast Backup and Restore √ Fast Backup and Restore as a binary backup mechanism √ Each node sends its data directly to the backup target in parallel √ Provides SFTP for Backup and Restore √ Can control concurrency ClustrixDB PoC ClustrixDB v7.6 2016 FTP Server Secure FTP
  • 16. Issues & Enhancements Replication Performance ● Write Intensive Workload ○ Replication Gap increasing ○ Binlogging Performance low Pre-Launch Workshop ClustrixDB v8.x 2017 Master ClustrixDB Slave ClustrixDB √ Zones (v9.x)
  • 17. Issues & Enhancements Enhanced Security ● SSL ○ Supports SSL Encrypted Connections ○ Requires a mysql client 5.6.38 or higher ● SHA256 Password Plugin ○ Provides strong user password credentials than mysql_native_password plugin ● Audit (User Logging) ○ Provides audit logs of user login/logout (user.log) ○ SET GLOBAL session_log_users = true; Expansion ClustrixDB v9.1 2018
  • 18. Issues & Enhancements Monitoring Tools ● Built-in Monitoring tool - ClustrixGUI ● Network security policy blocks using ClustrixGUI ● Need long-term historical data √ Monitoring with InfluxDB & Grafana ○ Collector script ○ Grafana dashboard √ Other tools are available Expansion ClustrixDB v9.1 2018
  • 19. Current Architecture Architecture #1 Zone 1 Zone 2 Zone 3 Master Slave ClustrixDBClustrixDB Architecture #2 MAX_FAILURES = 2 REPLICAS = 3 MAX_FAILURES = 1 REPLICAS = 2 ClustrixDB
  • 20. Current Deployment & Usage Region #2 Region #1 Region #3 M SS M SS 230 Million TPS 16 Billion Rows 2 Services 3 Regions
  • 21.  No Additional Resources for Migration or Sharding  Downsized Instance Spec.  No Standby Replicas for HA, Backup, Analytics  Less Man-Month  Easy Scalability  No SPOF, Strong HA  Better Maintenance & Monitoring  Analytic Query  Tech Support  Simplified Application Architecture  No Additional Code for Migration or Sharding  Focus on Service Logic Development Benefits Operation Cost
  • 22. Wrap Up ● Future Work ○ BINLOG / Replication Enhancement ○ ETL Tools ● Q&A

Editor's Notes

  1. Title Slide for OpenWorks
  2. OpenWorks End Slide