This document discusses the setup and architecture of a MySQL Spider configuration. It describes installing multiple MySQL instances on a single server, including 4 MariaDB 10.1 nodes and 6 MySQL 5.7 nodes. It then configures the Spider storage engine to partition tables across these nodes for both data sharding and high availability. Examples are provided of creating servers, tables, and inserting/selecting data to demonstrate the federated and sharded architecture.
OSMC 2019 | Use Cloud services & features in your redundant Icinga2 Environme...NETWAYS
This talk will start with a quick walk through the setup of all required components for a cloud based icinga2, icingaweb2 & icingaweb2-director environment. Focus will be on the configuration and monitoring of keepalived, HAProxy and Galera. Keepalived for example is used to interact with DigitalOcean and manage floating IPs. Examples will show how to use DigitalOcean loadbalancer instead of HAProxy. The talk will end with a summary of experienced limitations and pitfalls.
Thousands of running services on hundreds of machines ask the admins to pay attention. At all times, on all channels, with all available software packages that openSUSE has to offer.
Lars does not only keep an eye on all devices inside SUSE R&D, he is also maintaining some of the biggest packages in the server:monitoring repository like Nagios, Icinga, Shinken, check_mk, mod_gearman, Naemon, PNP4Nagios or the monitoring-plugins (and others).
The integration of some of those packages into a complex, secure and high available monitoring setup together with some tips and tricks and insights into the SUSE R&D infrastructure will be demonstrated.
Setting up a HADOOP 2.2 cluster on CentOS 6Manish Chopra
Create your own Hadoop distributed cluster using 3 virtual machines. Linux (CentOS 6 or RHEL 6) can be used, along with Java and Hadoop binary distributions.
Apresentação na Pós-Graduação em Segurança da Informação:
- Sniffer de senhas em plain text;
- Ataque de brute-force no SSH;
- Proteção: Firewall, IPS e/ou TCP Wrappers;
- Segurança básica no sshd_config;
- Chaves RSA/DSA para acesso remoto;
- SSH buscando chaves no LDAP;
- Porque previnir o acesso: Fork Bomb
OSMC 2019 | Use Cloud services & features in your redundant Icinga2 Environme...NETWAYS
This talk will start with a quick walk through the setup of all required components for a cloud based icinga2, icingaweb2 & icingaweb2-director environment. Focus will be on the configuration and monitoring of keepalived, HAProxy and Galera. Keepalived for example is used to interact with DigitalOcean and manage floating IPs. Examples will show how to use DigitalOcean loadbalancer instead of HAProxy. The talk will end with a summary of experienced limitations and pitfalls.
Thousands of running services on hundreds of machines ask the admins to pay attention. At all times, on all channels, with all available software packages that openSUSE has to offer.
Lars does not only keep an eye on all devices inside SUSE R&D, he is also maintaining some of the biggest packages in the server:monitoring repository like Nagios, Icinga, Shinken, check_mk, mod_gearman, Naemon, PNP4Nagios or the monitoring-plugins (and others).
The integration of some of those packages into a complex, secure and high available monitoring setup together with some tips and tricks and insights into the SUSE R&D infrastructure will be demonstrated.
Setting up a HADOOP 2.2 cluster on CentOS 6Manish Chopra
Create your own Hadoop distributed cluster using 3 virtual machines. Linux (CentOS 6 or RHEL 6) can be used, along with Java and Hadoop binary distributions.
Apresentação na Pós-Graduação em Segurança da Informação:
- Sniffer de senhas em plain text;
- Ataque de brute-force no SSH;
- Proteção: Firewall, IPS e/ou TCP Wrappers;
- Segurança básica no sshd_config;
- Chaves RSA/DSA para acesso remoto;
- SSH buscando chaves no LDAP;
- Porque previnir o acesso: Fork Bomb
Presentation at March 2019 Dutch Postgres User Group Meetup on lessons learnt while migrating from Oracle to Postgres, demo'ed via vagrant test environments and using generic pgbench datasets.
[CB20] Vulnerabilities of Machine Learning Infrastructure by Sergey GordeychikCODE BLUE
The boom of AI brought to the market a set of impressive solutions both on the hardware and software side. On the other hand, massive implementation of AI in various areas brings about problems, and security is one of the greatest concerns.
In this talk we will present results of hands-on vulnerability research of different components of AI infrastructure including NVIDIA DGX GPU servers, ML frameworks such as Pytorch, Keras and Tensorflow, data processing pipelines and specific applications, including Medical Imaging and face recognition powered CCTV. Updated Internet Census toolkit based on the Grinder framework will be introduced.
Salesforce at Stacki Atlanta Meetup February 2016StackIQ
Dave Peterson's presentation on how Salesforce uses Stacki and Chef to provision and manage thousands of servers. Stacki Atlanta kickoff Meetup on 2/23/16 at the Microsoft Innovation Center. Dave is a Lead Systems Engineer at Salesforce.
Code testing and Continuous Integration are just the first step in a source code to production process. Combined with infrastructure-as-code tools such as Puppet the whole process can be automated, and tested!
An Express Guide ~ Cacti for IT Infrastructure Monitoring & GraphingAbhishek Kumar
It's an Express Guide to "Setup of Cacti Server with purpose of IT Infrastructure Monitoring & Service Graphs" ~~~~~ its aimed at monitoring of various IT services and brilliant graphing of statistics
OpenStack Tokyo Meeup - Gluster Storage DayDan Radez
November 2012 Tokyo OpenStack meetup was dedicated to using Gluster storage. This presentation showed the fuse mount method to integrating gluster into OpenStack. There are new drivers that have been developed that make mounting gluster volumes to instances more efficient. This presentation doesn't show how to use them.
This tutorial will guide you through the many considerations when deploying a sharded cluster. We will cover the services that make up a sharded cluster, configuration recommendations for these services, shard key selection, use cases, and how data is managed within a sharded cluster. Maintaining a sharded cluster also has its challenges. We will review these challenges and how you can prevent them with proper design or ways to resolve them if they exist today. There will be lab sessions at the end of some chapters so please have your laptops with you.
6th International Conference on Machine Learning & Applications (CMLA 2024)ClaraZara1
6th International Conference on Machine Learning & Applications (CMLA 2024) will provide an excellent international forum for sharing knowledge and results in theory, methodology and applications of on Machine Learning & Applications.
A review on techniques and modelling methodologies used for checking electrom...nooriasukmaningtyas
The proper function of the integrated circuit (IC) in an inhibiting electromagnetic environment has always been a serious concern throughout the decades of revolution in the world of electronics, from disjunct devices to today’s integrated circuit technology, where billions of transistors are combined on a single chip. The automotive industry and smart vehicles in particular, are confronting design issues such as being prone to electromagnetic interference (EMI). Electronic control devices calculate incorrect outputs because of EMI and sensors give misleading values which can prove fatal in case of automotives. In this paper, the authors have non exhaustively tried to review research work concerned with the investigation of EMI in ICs and prediction of this EMI using various modelling methodologies and measurement setups.
TOP 10 B TECH COLLEGES IN JAIPUR 2024.pptxnikitacareer3
Looking for the best engineering colleges in Jaipur for 2024?
Check out our list of the top 10 B.Tech colleges to help you make the right choice for your future career!
1) MNIT
2) MANIPAL UNIV
3) LNMIIT
4) NIMS UNIV
5) JECRC
6) VIVEKANANDA GLOBAL UNIV
7) BIT JAIPUR
8) APEX UNIV
9) AMITY UNIV.
10) JNU
TO KNOW MORE ABOUT COLLEGES, FEES AND PLACEMENT, WATCH THE FULL VIDEO GIVEN BELOW ON "TOP 10 B TECH COLLEGES IN JAIPUR"
https://www.youtube.com/watch?v=vSNje0MBh7g
VISIT CAREER MANTRA PORTAL TO KNOW MORE ABOUT COLLEGES/UNIVERSITITES in Jaipur:
https://careermantra.net/colleges/3378/Jaipur/b-tech
Get all the information you need to plan your next steps in your medical career with Career Mantra!
https://careermantra.net/
Harnessing WebAssembly for Real-time Stateless Streaming PipelinesChristina Lin
Traditionally, dealing with real-time data pipelines has involved significant overhead, even for straightforward tasks like data transformation or masking. However, in this talk, we’ll venture into the dynamic realm of WebAssembly (WASM) and discover how it can revolutionize the creation of stateless streaming pipelines within a Kafka (Redpanda) broker. These pipelines are adept at managing low-latency, high-data-volume scenarios.
Hierarchical Digital Twin of a Naval Power SystemKerry Sado
A hierarchical digital twin of a Naval DC power system has been developed and experimentally verified. Similar to other state-of-the-art digital twins, this technology creates a digital replica of the physical system executed in real-time or faster, which can modify hardware controls. However, its advantage stems from distributing computational efforts by utilizing a hierarchical structure composed of lower-level digital twin blocks and a higher-level system digital twin. Each digital twin block is associated with a physical subsystem of the hardware and communicates with a singular system digital twin, which creates a system-level response. By extracting information from each level of the hierarchy, power system controls of the hardware were reconfigured autonomously. This hierarchical digital twin development offers several advantages over other digital twins, particularly in the field of naval power systems. The hierarchical structure allows for greater computational efficiency and scalability while the ability to autonomously reconfigure hardware controls offers increased flexibility and responsiveness. The hierarchical decomposition and models utilized were well aligned with the physical twin, as indicated by the maximum deviations between the developed digital twin hierarchy and the hardware.
Using recycled concrete aggregates (RCA) for pavements is crucial to achieving sustainability. Implementing RCA for new pavement can minimize carbon footprint, conserve natural resources, reduce harmful emissions, and lower life cycle costs. Compared to natural aggregate (NA), RCA pavement has fewer comprehensive studies and sustainability assessments.
HEAP SORT ILLUSTRATED WITH HEAPIFY, BUILD HEAP FOR DYNAMIC ARRAYS.
Heap sort is a comparison-based sorting technique based on Binary Heap data structure. It is similar to the selection sort where we first find the minimum element and place the minimum element at the beginning. Repeat the same process for the remaining elements.
Literature Review Basics and Understanding Reference Management.pptxDr Ramhari Poudyal
Three-day training on academic research focuses on analytical tools at United Technical College, supported by the University Grant Commission, Nepal. 24-26 May 2024
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...Amil Baba Dawood bangali
Contact with Dawood Bhai Just call on +92322-6382012 and we'll help you. We'll solve all your problems within 12 to 24 hours and with 101% guarantee and with astrology systematic. If you want to take any personal or professional advice then also you can call us on +92322-6382012 , ONLINE LOVE PROBLEM & Other all types of Daily Life Problem's.Then CALL or WHATSAPP us on +92322-6382012 and Get all these problems solutions here by Amil Baba DAWOOD BANGALI
#vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore#blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #blackmagicforlove #blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #Amilbabainuk #amilbabainspain #amilbabaindubai #Amilbabainnorway #amilbabainkrachi #amilbabainlahore #amilbabaingujranwalan #amilbabainislamabad
8. café.naver.com/mysqlpg
Setup
■ MySQL Install
Step 1. 사전 요구 프로그램 설치
sudo su -
yum install -y gcc automake autoconf libtool make cpan libstdc*
yum install -y perl-CPAN
yum install -y --skip-broken perl*
cpan YAML
9. café.naver.com/mysqlpg
Setup
■ MySQL Install
Step 2. 그룹(mysql)/계정(maria) 생성
sudo su -
mkdir -p /usr1/program
mkdir -p /usr1/mysql/dba/tools
groupadd -g 900 mysql
cat /etc/group | grep -E 'mysql|maria'
useradd maria -g mysql -d /home/maria -u 901 -c mysql_user
echo 'maria:oracle' | chpasswd
cat /etc/passwd | grep -E 'mysql|maria'
chown -R maria:mysql /usr1
ls -l /usr1
10. café.naver.com/mysqlpg
Setup
■ MySQL Install
Step 3. Sandbox Download & Compile
sudo su -
cd /usr1/program
wget http://search.cpan.org/CPAN/authors/id/G/GM/GMAX/MySQL-Sandbox-3.1.05.tar.gz
ls -l MySQL-Sandbox*
tar xvfz MySQL-Sandbox-3.1.05.tar.gz
cd MySQL-Sandbox-3.1.05
perl Makefile.PL
make && make test
make install
11. café.naver.com/mysqlpg
Setup
■ MySQL Install
Step 4. MariaDB 10.1 (SpiderNode) / MySQL 5.7 (DataNode) Binary Download
su - maria
cd /usr1/program/
wget http://ftp.kaist.ac.kr/mariadb//mariadb-10.1.12/bintar-linux-glibc_214-x86_64/mariadb-10.1.12-linux-glibc_214-x86_64.tar.gz
tar xvfz mariadb-10.1.12-linux-glibc_214-x86_64.tar.gz
mv mariadb-10.1.12-linux-glibc_214-x86_64 /usr1/mysql/10.1.12
wget https://downloads.mariadb.com/archives/mysql-5.7/mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz
tar zxvf mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz
mv mysql-5.7.11-linux-glibc2.5-x86_64 /usr1/mysql/5.7.11
ls -l /usr1/mysql
chown -R maria:mysql /usr1
12. café.naver.com/mysqlpg
Setup
■ MySQL Install
Step 5. Multi Instance Install (Spider : 4, Data : 6)
su - maria
cd /usr1/mysql
mkdir -p /usr1/mysql/sandboxes
make_multiple_sandbox -r spider_node 10.1.12 --upper_directory=/usr1/mysql/sandboxes --how_many_nodes=4
make_multiple_sandbox -r data_node 5.7.11 --upper_directory=/usr1/mysql/sandboxes --how_many_nodes=6
ps -ef | awk -F ' ' '{print $2, $9;}' | grep mysqld_safe
6479 /usr1/mysql/10.1.12/bin/mysqld_safe
…
7604 /usr1/mysql/5.7.11/bin/mysqld_safe
…
netstat -nap | grep LISTEN | grep mysql
13. café.naver.com/mysqlpg
Setup
■ MySQL Install
Step 5. Multi Instance Install
# Port 변경 (Spider Node)
cd /usr1/mysql/sandboxes/spider_node && ./stop_all
cd /usr1/mysql/sandboxes/spider_node/node1 && ./change_ports 3306
cd /usr1/mysql/sandboxes/spider_node/node2 && ./change_ports 3307
cd /usr1/mysql/sandboxes/spider_node/node3 && ./change_ports 9906
cd /usr1/mysql/sandboxes/spider_node/node4 && ./change_ports 9907
cd /usr1/mysql/sandboxes/spider_node && ./start_all
cd /usr1/mysql/sandboxes/spider_node && ./status_all
cd /usr1/mysql/sandboxes/spider_node && ./use_all -e " select version()"
netstat -nap | grep LISTEN | grep mysql
3306
10001
10002
10003
14. café.naver.com/mysqlpg
Setup
■ MySQL Install
Step 5. Multi Instance Install
# Port 변경 (Data Node)
cd /usr1/mysql/sandboxes/data_node && ./stop_all
cd /usr1/mysql/sandboxes/data_node/node1 && ./change_ports 10001
cd /usr1/mysql/sandboxes/data_node/node2 && ./change_ports 10002
cd /usr1/mysql/sandboxes/data_node/node3 && ./change_ports 10003
cd /usr1/mysql/sandboxes/data_node/node4 && ./change_ports 20001
cd /usr1/mysql/sandboxes/data_node/node5 && ./change_ports 20002
cd /usr1/mysql/sandboxes/data_node/node6 && ./change_ports 20003
cd /usr1/mysql/sandboxes/data_node && ./start_all
cd /usr1/mysql/sandboxes/data_node && ./status_all
cd /usr1/mysql/sandboxes/data_node && ./use_all -e " select version()"
netstat -nap | grep LISTEN | grep mysql
3306
10001
10002
10003
15. café.naver.com/mysqlpg
Setup
■ MySQL Install
Step 6. Define an easy way to access the nodes from the MariaDB or MySQL client
su - maria
alias spider1='/usr1/mysql/10.1.12/bin/mysql --user=msandbox --password=msandbox --host=localhost --port=3306 --socket=/tmp/mysql_sandbox3306.sock'
alias spider2='/usr1/mysql/10.1.12/bin/mysql --user=msandbox --password=msandbox --host=localhost --port=3307 --socket=/tmp/mysql_sandbox3307.sock'
alias spider3='/usr1/mysql/10.1.12/bin/mysql --user=msandbox --password=msandbox --host=localhost --port=9906 --socket=/tmp/mysql_sandbox9906.sock'
alias spider4='/usr1/mysql/10.1.12/bin/mysql --user=msandbox --password=msandbox --host=localhost --port=9907 --socket=/tmp/mysql_sandbox9907.sock'
alias backend1='/usr1/mysql/5.7.11/bin/mysql --user=msandbox --password=msandbox --host=localhost --port=10001 --socket=/tmp/mysql_sandbox10001.sock
alias backend2='/usr1/mysql/5.7.11/bin/mysql --user=msandbox --password=msandbox --host=localhost --port=10002 --socket=/tmp/mysql_sandbox10002.sock
alias backend3='/usr1/mysql/5.7.11/bin/mysql --user=msandbox --password=msandbox --host=localhost --port=10003 --socket=/tmp/mysql_sandbox10003.sock
alias backend11='/usr1/mysql/5.7.11/bin/mysql --user=msandbox --password=msandbox --host=localhost --port=20001 --socket=/tmp/mysql_sandbox20001.sock'
alias backend12='/usr1/mysql/5.7.11/bin/mysql --user=msandbox --password=msandbox --host=localhost --port=20002 --socket=/tmp/mysql_sandbox20002.sock'
alias backend13='/usr1/mysql/5.7.11/bin/mysql --user=msandbox --password=msandbox --host=localhost --port=20003 --socket=/tmp/mysql_sandbox20003.sock'
$ spider1 -e "select version()"
+-----------------+
| version() |
+-----------------+
| 10.1.12-MariaDB |
+-----------------+
$ backend1 -e "select version()"
+------------+
| version() |
+------------+
| 5.7.11-log |
+------------+
21. café.naver.com/mysqlpg
Architecture
■ Federation
Without connection pool or MariaDB thread pool, HaProxy and Spider have been protecting the tcp socket overflow without specific TCP tuning.
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
backend1 << EOF
DROP DATABASE IF EXISTS backend ;
CREATE DATABASE backend;
CREATE TABLE backend.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=InnoDB;
DESC backend.sbtest;
EOF
①
22. café.naver.com/mysqlpg
Architecture
■ Federation
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
spider1 << EOF
DROP SERVER IF EXISTS backend1;
CREATE SERVER backend1 FOREIGN DATA WRAPPER mysql
OPTIONS( HOST '127.0.0.1‘
, DATABASE 'backend‘
, USER 'msandbox‘
, PASSWORD 'msandbox‘
, PORT 10001);
EOF②
23. café.naver.com/mysqlpg
Architecture
■ Federation
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
spider1 << EOF
CREATE DATABASE IF NOT EXISTS test;
DROP TABLE IF EXISTS test.sbtest;
CREATE TABLE test.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=spider COMMENT='wrapper "mysql",srv "backend1"';
SELECT * FROM test.sbtest LIMIT 10;
EOF
③
24. café.naver.com/mysqlpg
Architecture
■ Federation & HA
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
backend1 << EOF
CREATE TABLE backend.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=InnoDB;
EOF
backend2 << EOF
…
EOF
backend3 << EOF
…
EOF
①①
26. café.naver.com/mysqlpg
Architecture
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
spider1 << EOF
CREATE DATABASE IF NOT EXISTS test;
DROP TABLE IF EXISTS test.sbtest;
CREATE TABLE test.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=spider COMMENT='wrapper "mysql",srv "backend1 backend2 backend3"';
SELECT * FROM test.sbtest LIMIT 10;
EOF
■ Federation & HA
③
27. café.naver.com/mysqlpg
Architecture
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
spider1 << EOF
TRUNCATE TABLE test.sbtest;
Insert into test.sbtest values (1, 100, 'A', 'A1');
Insert into test.sbtest values (2, 200, 'A', 'A1');
Insert into test.sbtest values (3, 300, 'A', 'A1');
Insert into test.sbtest values (4, 400, 'A', 'A1');
Insert into test.sbtest values (5, 500, 'A', 'A1');
SELECT * FROM test.sbtest;
EOF
backend1 -e"select count(*) from backend.sbtest;"
backend2 -e"select count(*) from backend.sbtest;"
backend3 -e"select count(*) from backend.sbtest;"
+----------+
| count(*) |
+----------+
| 5 |
+----------+
■ Federation & HA
④
동일한 데이터를 확인할 수 있음 !
28. café.naver.com/mysqlpg
Architecture
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
cd /usr1/mysql/sandboxes/data_node/node2 && ./stop
spider1 -e "Insert into test.sbtest values (11, 1100, 'A', 'A1');"
-- ERROR 1429 (HY000) at line 1:
Unable to connect to foreign data source: backend2
backend1 -e"select count(*) from backend.sbtest;"
-- 5
backend2 -e"select count(*) from backend.sbtest;"
-- ERROR 2002 (HY000):
Can't connect to local MySQL server through socket '/tmp/mysql_sandbox100
backend3 -e"select count(*) from backend.sbtest;"
-- 5
■ Federation & HA
Datanode2 장애유발!
29. café.naver.com/mysqlpg
Architecture
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
■ Federation & HA
spider1 << EOF
DROP TABLE IF EXISTS test.sbtest;
CREATE TABLE test.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=spider COMMENT='wrapper "mysql",srv "backend1 backend3"';
EOF
spider1 -e "Insert into test.sbtest values (11, 1100, 'A', 'A1');"
backend1 -e"select count(*) from backend.sbtest;"
backend3 -e"select count(*) from backend.sbtest;"
-- 6
-- 6
복구방법 :
테이블의 정의를 변경함
으로써 빠르게 복구 가능
30. café.naver.com/mysqlpg
Architecture
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
backend1 << EOF
CREATE TABLE backend.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=InnoDB;
EOF
backend2 << EOF
CREATE TABLE backend.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=InnoDB;
EOF
■ Sharding
Data Sharding
②①
31. café.naver.com/mysqlpg
Architecture
■ Sharding
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
spider1 << EOF
CREATE SERVER backend1 FOREIGN DATA WRAPPER mysql
OPTIONS( HOST '127.0.0.1‘
, DATABASE 'backend‘
, USER 'msandbox‘
, PASSWORD 'msandbox‘
, PORT 10001);
CREATE SERVER backend2 FOREIGN DATA WRAPPER mysql
OPTIONS( HOST '127.0.0.1‘
, DATABASE 'backend‘
, USER 'msandbox‘
, PASSWORD 'msandbox‘
, PORT 10002);
EOF
③ ④
32. café.naver.com/mysqlpg
Architecture
■ Sharding
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
spider1 << EOF
CREATE DATABASE IF NOT EXISTS test;
CREATE TABLE test.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=spider COMMENT='wrapper "mysql", table "sbtest"'
PARTITION BY KEY (id)
(
PARTITION pt1 COMMENT = 'srv "backend1"'
, PARTITION pt2 COMMENT = 'srv "backend2"'
);
SELECT count(*) FROM test.sbtest LIMIT 10;
EOF
⑤
33. café.naver.com/mysqlpg
Architecture
■ Sharding
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
spider1 << EOF
TRUNCATE TABLE test.sbtest;
Insert into test.sbtest values (1, 100, 'A', 'A1');
Insert into test.sbtest values (2, 200, 'A', 'A1');
Insert into test.sbtest values (3, 300, 'A', 'A1');
Insert into test.sbtest values (4, 400, 'A', 'A1');
Insert into test.sbtest values (5, 500, 'A', 'A1');
SELECT * FROM test.sbtest;
EOF
backend1 -e"select count(*) from backend.sbtest;"
-- 3
backend2 -e"select count(*) from backend.sbtest;"
-- 2
⑥
34. café.naver.com/mysqlpg
Architecture
■ High Availability
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
backend1 << EOF
CREATE TABLE backend.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=InnoDB;
CREATE TABLE backend_rpl.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=InnoDB;
EOF
backend2 << EOF
CREATE TABLE backend.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=InnoDB;
CREATE TABLE backend_rpl.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=InnoDB;
EOF
②①
back1 back1_rpl back2 back2_rpl
35. café.naver.com/mysqlpg
Architecture
■ High Availability
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
spider1 << EOF
CREATE SERVER backend1 FOREIGN DATA WRAPPER mysql
OPTIONS( HOST '127.0.0.1‘, DATABASE 'backend‘ , … , PORT 10001);
CREATE SERVER backend1_rpl FOREIGN DATA WRAPPER mysql
OPTIONS( HOST '127.0.0.1‘, DATABASE 'backend_rpl‘, … , PORT 10001);
CREATE SERVER backend2 FOREIGN DATA WRAPPER mysql
OPTIONS( HOST '127.0.0.1‘, DATABASE 'backend‘ , … , PORT 10002);
CREATE SERVER backend2_rpl FOREIGN DATA WRAPPER mysql
OPTIONS( HOST '127.0.0.1‘, DATABASE 'backend_rpl‘, … , PORT 10002);
EOF
back1 back1_rpl back2 back2_rpl
④③
36. café.naver.com/mysqlpg
Architecture
■ High Availability
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
back1 back1_rpl back2 back2_rpl
spider1 << EOF
CREATE DATABASE IF NOT EXISTS test;
CREATE TABLE test.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
, k int unsigned NOT NULL DEFAULT '0'
, c char(100) NOT NULL DEFAULT ''
, pad char(60) NOT NULL DEFAULT ''
, PRIMARY KEY (id)
, KEY k (k)
)
ENGINE=spider COMMENT='wrapper "mysql", table "sbtest"'
PARTITION BY KEY (id)
(
PARTITION pt1 COMMENT = 'srv "backend1 backend2_rpl"'
, PARTITION pt2 COMMENT = 'srv "backend2 backend1_rpl"'
);
INSERT INTO test.sbtest select 10000001, 0, '' ,'replicas test';
EOF
⑤
37. café.naver.com/mysqlpg
Architecture
■ High Availability
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
back1 back1_rpl back2 back2_rpl
backend1 -e "SELECT * FROM backend.sbtest WHERE id=10000001";
+----------+---+---+---------------+
| id | k | c | pad |
+----------+---+---+---------------+
| 10000001 | 0 | | replicas test |
+----------+---+---+---------------+
backend2 -e "SELECT * FROM backend.sbtest where id=10000001";
backend2 -e "SELECT * FROM backend_rpl.sbtest where id=10000001";
+----------+---+---+---------------+
| id | k | c | pad |
+----------+---+---+---------------+
| 10000001 | 0 | | replicas test |
+----------+---+---+---------------+
⑥
38. café.naver.com/mysqlpg
Architecture
■ High Availability
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
back1 back1_rpl back2 back2_rpl
spider1 << EOF
insert into test.sbtest values (1, 100, 'A', 'A1');
insert into test.sbtest values (2, 200, 'A', 'A1');
insert into test.sbtest values (3, 300, 'A', 'A1');
insert into test.sbtest values (4, 400, 'A', 'A1');
insert into test.sbtest values (5, 500, 'A', 'A1');
EOF
backend1 -e "select (select count(*) from backend.sbtest) as cnt1
, (select count(*) from backend_rpl.sbtest) as cnt2"
+------+------+
| cnt1 | cnt2 |
+------+------+
| 4 | 2 |
+------+------+
backend2 -e "select (select count(*) from backend.sbtest) as cnt1
, (select count(*) from backend_rpl.sbtest) as cnt2"
+------+------+
| cnt1 | cnt2 |
+------+------+
| 2 | 4 |
+------+------+
⑦
⑧
4 42 2
39. café.naver.com/mysqlpg
Architecture
■ High Availability
What is happening if we stop one backend?
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
back1 back1_rpl back2 back2_rpl
spider1 -e "SELECT * FROM test.sbtest where id=10000001";
+----------+---+---+---------------+
| id | k | c | pad |
+----------+---+---+---------------+
| 10000001 | 0 | | replicas test |
+----------+---+---+---------------+
cd /usr1/mysql/sandboxes/data_node/node1 && ./stop
backend1 -e "SELECT * FROM backend.sbtest where id=10000001"
-- ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql_sandbox10001.sock'
spider1 -e "SELECT * FROM test.sbtest where id=10000001";
-- ERROR 1429 (HY000): Unable to connect to foreign data source: backend1_rpl
spider1 -e "SELECT * from mysql.spider_tables order by link_id, server";
cd /usr1/mysql/sandboxes/data_node/node1 && ./start
spider1 -e "SELECT * FROM test.sbtest where id=10000001";
+----------+---+---+---------------+
| id | k | c | pad |
+----------+---+---+---------------+
| 10000001 | 0 | | replicas test |
+----------+---+---+---------------+
⑨
⑩
4 42 2
40. café.naver.com/mysqlpg
Architecture
■ High Availability
What is happening if we stop one backend?
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
back1 back1_rpl back2 back2_rpl
-- Data Node Replica 모니터링 등록
spider1 << EOF
DROP TABLE IF EXISTS test.sbtest;
CREATE TABLE test.sbtest (
id int unsigned NOT NULL AUTO_INCREMENT
…
)
ENGINE=spider COMMENT='wrapper "mysql", table "sbtest"'
PARTITION BY KEY (id)
(
PARTITION pt1 COMMENT = 'srv "backend1 backend2_rpl", mbk "2", mkd "2", msi "1", link_status "0 0"'
, PARTITION pt2 COMMENT = 'srv "backend2 backend1_rpl", mbk "2", mkd "2", msi "1", link_status "0 0"'
);
CREATE SERVER mon FOREIGN DATA WRAPPER mysql
OPTIONS ( HOST '127.0.0.1', DATABASE 'test', … , PORT 3306);
INSERT INTO mysql.spider_link_mon_servers VALUES ('%','%','%',1,'mon‘
,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL);
SELECT * FROM mysql.spider_link_mon_servers;
SELECT spider_flush_table_mon_cache();
EOF
HA 모니터링을 관장할
서버의 @@Service_ID
- Spider_Node : 1, 2
- Data_Node : 101, 102
Node Status
1 : 정상, 3 : 중지
2 : 관리 작업 중
@@server_id = 1
@@server_id = 101
@@server_id = 102
⑪
41. café.naver.com/mysqlpg
Architecture
■ High Availability
What is happening if we stop one backend?
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
back1 back1_rpl back2 back2_rpl
spider1 -e "SELECT * FROM test.sbtest where id=10000001";
+----------+---+---+---------------+
| id | k | c | pad |
+----------+---+---+---------------+
| 10000001 | 0 | | replicas test |
+----------+---+---+---------------+
cd /usr1/mysql/sandboxes/data_node/node1 && ./stop
backend1 -e "SELECT * FROM backend.sbtest where id=10000001"
-- ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql_sandbox10001.sock'
spider1 -e "SELECT * FROM test.sbtest where id=10000001";
+----------+---+---+---------------+
| id | k | c | pad |
+----------+---+---+---------------+
| 10000001 | 0 | | replicas test |
+----------+---+---+---------------+
spider1 -e "SELECT * from mysql.spider_tables order by link_id, server";
+---------+--------------+--------------+-------------+
| link_id | server | table_name | link_status |
+---------+--------------+--------------+-------------+
| 0 | backend1 | sbtest#P#pt1 | 3 |
| 1 | backend1_rpl | sbtest#P#pt2 | 3 |
| 0 | backend2 | sbtest#P#pt2 | 1 |
| 1 | backend2_rpl | sbtest#P#pt1 | 1 |
+---------+--------------+--------------+-------------+
데이터 조회는 되었지만,
실제는 fail 상태로 존재함
⑫
42. café.naver.com/mysqlpg
Architecture
■ High Availability
What is happening if we stop one backend?
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
back1 back1_rpl back2 back2_rpl
cd /usr1/mysql/sandboxes/data_node/node1 && ./start
spider1 << EOF
select * from mysql.spider_tables order by link_id, server, table_name;
-- link status 변경 : fail -> maintenance
ALTER TABLE test.sbtest
ENGINE=spider COMMENT='wrapper "mysql", table "sbtest"'
PARTITION BY KEY (id)
(
PARTITION pt1 COMMENT = 'srv "backend1 backend2_rpl", mbk "2", mkd "2", msi "1", link_status "2 0"'
, PARTITION pt2 COMMENT = 'srv "backend2 backend1_rpl", mbk "2", mkd "2", msi "1", link_status "0 2"'
);
select * from mysql.spider_tables order by link_id, server, table_name;
+---------+--------------+--------------+-------------+
| link_id | server | table_name | link_status |
+---------+--------------+--------------+-------------+
| 0 | backend1 | sbtest#P#pt1 | 2 |
| 1 | backend1_rpl | sbtest#P#pt2 | 2 |
| 0 | backend2 | sbtest#P#pt2 | 1 |
| 1 | backend2_rpl | sbtest#P#pt1 | 1 |
+---------+--------------+--------------+-------------+
⑬
43. café.naver.com/mysqlpg
Architecture
■ High Availability
What is happening if we stop one backend?
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
back1 back1_rpl back2 back2_rpl
select * from mysql.spider_tables order by link_id, server, table_name;
+---------+--------------+--------------+-------------+
| link_id | server | table_name | link_status |
+---------+--------------+--------------+-------------+
| 0 | backend1 | sbtest#P#pt1 | 2 |
| 1 | backend1_rpl | sbtest#P#pt2 | 2 |
| 0 | backend2 | sbtest#P#pt2 | 1 |
| 1 | backend2_rpl | sbtest#P#pt1 | 1 |
+---------+--------------+--------------+-------------+
select spider_copy_tables('test.sbtest#P#pt1','1','0');
-- 1
select spider_copy_tables('test.sbtest#P#pt2','0','1');
-- 1
⑭
44. café.naver.com/mysqlpg
Architecture
■ High Availability
What is happening if we stop one backend?
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
back1 back1_rpl back2 back2_rpl
-- link status 변경 : maintenance -> live
ALTER TABLE test.sbtest
ENGINE=spider COMMENT='wrapper "mysql", table "sbtest"'
PARTITION BY KEY (id)
(
PARTITION pt1 COMMENT = 'srv "backend1 backend2_rpl", mbk "2", mkd "2", msi "1", link_status "1 0"'
, PARTITION pt2 COMMENT = 'srv "backend2 backend1_rpl", mbk "2", mkd "2", msi "1", link_status "0 1"'
);
select * from mysql.spider_tables order by link_id, server, table_name;
+---------+--------------+--------------+-------------+
| link_id | server | table_name | link_status |
+---------+--------------+--------------+-------------+
| 0 | backend1 | sbtest#P#pt1 | 1 |
| 1 | backend1_rpl | sbtest#P#pt2 | 1 |
| 0 | backend2 | sbtest#P#pt2 | 1 |
| 1 | backend2_rpl | sbtest#P#pt1 | 1 |
+---------+--------------+--------------+-------------+
⑮
45. café.naver.com/mysqlpg
Architecture
■ High Availability
Reference : https://mariadb.com/kb/en/mariadb/spider-storage-engine-overview/
back1 back1_rpl back2 back2_rpl
spider1 << EOF
insert into test.sbtest values (1, 100, 'A', 'A1');
insert into test.sbtest values (2, 200, 'A', 'A1');
insert into test.sbtest values (3, 300, 'A', 'A1');
insert into test.sbtest values (4, 400, 'A', 'A1');
insert into test.sbtest values (5, 500, 'A', 'A1');
EOF
backend1 -e "select (select count(*) from backend.sbtest) as cnt1
, (select count(*) from backend_rpl.sbtest) as cnt2"
+------+------+
| cnt1 | cnt2 |
+------+------+
| 4 | 2 |
+------+------+
backend2 -e "select (select count(*) from backend.sbtest) as cnt1
, (select count(*) from backend_rpl.sbtest) as cnt2"
+------+------+
| cnt1 | cnt2 |
+------+------+
| 2 | 4 |
+------+------+