SlideShare a Scribd company logo
1 of 12
Download to read offline
11th
Central and Eastern European Software
Engineering Conference in Russia - CEE-SECR
2015
October 22 - 24, Moscow
Timofey Turenko
Automatic tool for multi-configuration
environment creation for database server
and database proxy application testing
MariaDB Corporation AB
Kirill Krinkin
St-Petersburg Electrotechnical University
MariaDB & Maxscale
Test environment problem
 Server and proxy application testing:
– backend is needed
– lots of combination of machine configurations and
version of backend servers
 Maxscale example:
– 12 major linux distributions
– MariaDB 5.5, 10.0, 10.1, 10.2
– MySQL 5.1, 5.5, 5.6, 5.7
– different topologies
– Maxscale itself from CI
Requirements
 arbitrary number of backend servers;
 any version of server software can be installed on every
backend server;
 different virtual machine providers have to be supported
(Virtual Box, Qemu, different cloud providers);
 deployment of test environment have to be automated;
 several test round can be executed in parallel;
 test environment description have to be clear and human
readable.
Configuration example
VM managers, SW manager
 Vagrant, Terraform, Ansible
 Chef, Puppet
works? try to
write/read it:
### Import AWS Provider access config ###
require 'yaml'
aws_config = YAML.load_file("../aws-config.yml")['aws']
## of import AWS Provider access config
### Vagrant configuration block ###
####################################
Vagrant.configure(2) do |config|
### AWS Provider config block ###
#############################################
config.vm.box = "dummy"
config.vm.provider :aws do |aws, override|
aws.access_key_id = aws_config["access_key_id"]
aws.secret_access_key = aws_config["secret_access_key"]
aws.keypair_name = aws_config["keypair_name"]
aws.region = aws_config["region"]
aws.security_groups = aws_config["security_groups"]
aws.user_data = aws_config["user_data"]
override.ssh.private_key_path = aws_config["pemfile"]
override.nfs.functional = false
end ## of AWS Provider config block
config.vm.synced_folder "/home/vagrant/build-scripts/test-setup-...
config.vm.define :node0 do |node0|
node0.vm.provider :aws do |aws,override|
aws.ami = "ami-1c2e8b6b"
aws.instance_type = "t1.micro"
override.ssh.username = "ec2-user"
end
##--- Chef binding ---
node0.vm.provision "chef_solo" do |chef|
chef.cookbooks_path = "../recipes/cookbooks/"
chef.roles_path = "."
chef.add_role "node0"
chef.synced_folder_type = "rsync"
end #<-- end of chef binding
end # -> End definition for machine: node0
config.vm.synced_folder "/home/vagrant/build-scripts/test-setup-...
# -> Begin definition for machine: node1
config.vm.define :node1 do |node1|
node1.vm.provider :aws do |aws,override|
aws.ami = "ami-1c2e8b6b"
aws.instance_type = "t1.micro"
override.ssh.username = "ec2-user"
end
##--- Chef binding ---
Make it readable
"node0" :
{
"hostname" : "node0",
"box" : "centos7",
"product" : {
"name": "mariadb",
"version": "10.0",
"cnf_template" : "server1.cnf",
"cnf_template_path": "/home/vagrant/cnf"
}
},
"node1" :
{
"hostname" : "node1",
"box" : "rhel7",
"product" : {
"name": "mysql",
"version": "5.6",
"cnf_template" : "server2.cnf",
"cnf_template_path": "/home/vagrant//cnf"
}
}
"node2" :
{
"hostname" : "node2",
"box" : "centos7",
"product" : {
"name": "galera",
"version": "5.5",
"cnf_template" : "server-galera3.cnf",
"cnf_template_path": "/home/vagrant//cnf"
}
}
boxes.json
{
"ubuntu_precise": {
"provider": "virtualbox",
"box": "chef/ubuntu-12.04",
"platform": "ubuntu",
"platform_version": "precise"
},
"centos7" : {
"provider": "aws",
"ami": "ami-1c2e8b6b",
"user": "ec2-user",
"default_instance_type": "t1.micro",
"platform": "centos",
"platform_version": "7"
}
}
Repository description
{ "product": "mariadb",
"version": "10.0.17",
"repo": "http://yum.mariadb.org/10.0.17/centos5-amd64",
"repo_key": "https://yum.mariadb.org/RPM-GPG-KEY-MariaDB",
"platform": "centos",
"platform_version": 5}
{ "product": "mysql",
"version": "5.6",
"repo": "http://repo.mysql.com/yum/mysql-5.6-community/sles/12/x86_64",
"repo_key": "http://repo.mysql.com/RPM-GPG-KEY-mysql",
"platform": "sles",
"platform_version": 12}
We have a generator for
MariaDB, MariaDB enterprise, MariaDB Galera, Maxscale, MySQL
{ "product": "mariadb",
"version": "10.0",
"repo": "http://yum.mariadb.org/10.0/centos5-amd64",
"repo_key": "https://yum.mariadb.org/RPM-GPG-KEY-MariaDB",
"platform": "centos",
"platform_version": 5}
Workflow
 write template
 './mdbci generate <name>'
 './mdbci up <name>'
 Get all needed info: './mdbci show …'
 Access machines: './mdbci ssh' or directly via 'ssh'
 './mdbci destory <name>'
Does it work?
 9 and 25 machines configurations
 40 test runs every day
 MariaDB, Maxscale, MariaDB enterprise, MariaDB-Galera
 VirtualBox, Amazon EC2, QEMU (plans: Docker, DigitalOcean)
 Developers like it: “use ./mdbci ssh and see all log, states,
core dumps, use my env for debugging”
Source
 https://github.com/OSLL/mdbci
 https://github.com:mariadb-corporation/mdbci-repository-
config
https://github.com/mariadb-corporation/MaxScale
https://mariadb.com/products/mariadb-maxscale
Maxscale

More Related Content

What's hot

OpenNebula and SaltStack - OpenNebulaConf 2013
OpenNebula and SaltStack - OpenNebulaConf 2013OpenNebula and SaltStack - OpenNebulaConf 2013
OpenNebula and SaltStack - OpenNebulaConf 2013
databus.pro
 
Introduction to selenium_grid_workshop
Introduction to selenium_grid_workshopIntroduction to selenium_grid_workshop
Introduction to selenium_grid_workshop
seleniumconf
 
Free enterpriseloadbalancer
Free enterpriseloadbalancerFree enterpriseloadbalancer
Free enterpriseloadbalancer
mikesoule
 

What's hot (20)

Chef solo the beginning
Chef solo the beginning Chef solo the beginning
Chef solo the beginning
 
DevOpsDaysCPT Ansible Infrastrucutre as Code 2017
DevOpsDaysCPT Ansible Infrastrucutre as Code 2017DevOpsDaysCPT Ansible Infrastrucutre as Code 2017
DevOpsDaysCPT Ansible Infrastrucutre as Code 2017
 
Orchestration? You Don't Need Orchestration. What You Want is Choreography.
Orchestration? You Don't Need Orchestration. What You Want is Choreography.Orchestration? You Don't Need Orchestration. What You Want is Choreography.
Orchestration? You Don't Need Orchestration. What You Want is Choreography.
 
Chef training - Day3
Chef training - Day3Chef training - Day3
Chef training - Day3
 
What Makes a Good Chef Cookbook? (May 2014 Edition)
What Makes a Good Chef Cookbook? (May 2014 Edition)What Makes a Good Chef Cookbook? (May 2014 Edition)
What Makes a Good Chef Cookbook? (May 2014 Edition)
 
Introduction to Ansible
Introduction to AnsibleIntroduction to Ansible
Introduction to Ansible
 
Chef training Day5
Chef training Day5Chef training Day5
Chef training Day5
 
Selenium grid workshop london 2016
Selenium grid workshop london 2016Selenium grid workshop london 2016
Selenium grid workshop london 2016
 
Jenkins and ansible reference
Jenkins and ansible referenceJenkins and ansible reference
Jenkins and ansible reference
 
IT Automation with Ansible
IT Automation with AnsibleIT Automation with Ansible
IT Automation with Ansible
 
Chef training Day4
Chef training Day4Chef training Day4
Chef training Day4
 
Ansible intro
Ansible introAnsible intro
Ansible intro
 
Test driven infrastructure
Test driven infrastructureTest driven infrastructure
Test driven infrastructure
 
Configuration management and orchestration with Salt
Configuration management and orchestration with SaltConfiguration management and orchestration with Salt
Configuration management and orchestration with Salt
 
OpenNebula and SaltStack - OpenNebulaConf 2013
OpenNebula and SaltStack - OpenNebulaConf 2013OpenNebula and SaltStack - OpenNebulaConf 2013
OpenNebula and SaltStack - OpenNebulaConf 2013
 
Introduction to selenium_grid_workshop
Introduction to selenium_grid_workshopIntroduction to selenium_grid_workshop
Introduction to selenium_grid_workshop
 
Free enterpriseloadbalancer
Free enterpriseloadbalancerFree enterpriseloadbalancer
Free enterpriseloadbalancer
 
Cook Infrastructure with chef -- Justeat.IN
Cook Infrastructure with chef  -- Justeat.INCook Infrastructure with chef  -- Justeat.IN
Cook Infrastructure with chef -- Justeat.IN
 
Learn basic ansible using docker
Learn basic ansible using dockerLearn basic ansible using docker
Learn basic ansible using docker
 
#OktoCampus - Workshop : An introduction to Ansible
#OktoCampus - Workshop : An introduction to Ansible#OktoCampus - Workshop : An introduction to Ansible
#OktoCampus - Workshop : An introduction to Ansible
 

Viewers also liked

OLPC Mesh networking improvements
OLPC Mesh networking improvementsOLPC Mesh networking improvements
OLPC Mesh networking improvements
OSLL
 
N8xx olpc connectivity
N8xx olpc connectivityN8xx olpc connectivity
N8xx olpc connectivity
OSLL
 
SVG-player plugin for ns2 simulations
SVG-player plugin for ns2 simulationsSVG-player plugin for ns2 simulations
SVG-player plugin for ns2 simulations
OSLL
 
Fruct4 n8xx olpc-connectivity
Fruct4 n8xx olpc-connectivityFruct4 n8xx olpc-connectivity
Fruct4 n8xx olpc-connectivity
OSLL
 
Open Source implementation of ZigBee
Open Source implementation of ZigBeeOpen Source implementation of ZigBee
Open Source implementation of ZigBee
OSLL
 

Viewers also liked (20)

Implementation of the new REST API for Open Source LBS-platform Geo2Tag
Implementation of the new REST API for Open Source LBS-platform Geo2TagImplementation of the new REST API for Open Source LBS-platform Geo2Tag
Implementation of the new REST API for Open Source LBS-platform Geo2Tag
 
Testing with Selenium
Testing with SeleniumTesting with Selenium
Testing with Selenium
 
Обзор Btrfs
Обзор BtrfsОбзор Btrfs
Обзор Btrfs
 
OLPC Mesh networking improvements
OLPC Mesh networking improvementsOLPC Mesh networking improvements
OLPC Mesh networking improvements
 
N8xx olpc connectivity
N8xx olpc connectivityN8xx olpc connectivity
N8xx olpc connectivity
 
MOOCs Virtual Lab in Modern Education
MOOCs Virtual Lab in Modern EducationMOOCs Virtual Lab in Modern Education
MOOCs Virtual Lab in Modern Education
 
Обзор файловой системы GlusterFS
Обзор файловой системы GlusterFSОбзор файловой системы GlusterFS
Обзор файловой системы GlusterFS
 
SVG-player plugin for ns2 simulations
SVG-player plugin for ns2 simulationsSVG-player plugin for ns2 simulations
SVG-player plugin for ns2 simulations
 
E.Kalishenko, K.Krinkin, S.P.Shiva Prakash. Process Mining Approach for Traff...
E.Kalishenko, K.Krinkin, S.P.Shiva Prakash. Process Mining Approach for Traff...E.Kalishenko, K.Krinkin, S.P.Shiva Prakash. Process Mining Approach for Traff...
E.Kalishenko, K.Krinkin, S.P.Shiva Prakash. Process Mining Approach for Traff...
 
Fruct4 n8xx olpc-connectivity
Fruct4 n8xx olpc-connectivityFruct4 n8xx olpc-connectivity
Fruct4 n8xx olpc-connectivity
 
Full Automated Continuous Integration and Testing Infrastructure for Maxscale...
Full Automated Continuous Integration and Testing Infrastructure for Maxscale...Full Automated Continuous Integration and Testing Infrastructure for Maxscale...
Full Automated Continuous Integration and Testing Infrastructure for Maxscale...
 
Обзор архитектуры [файловой] системы Ceph
Обзор архитектуры [файловой] системы CephОбзор архитектуры [файловой] системы Ceph
Обзор архитектуры [файловой] системы Ceph
 
Работа с геоданными в MongoDb
Работа с геоданными в MongoDbРабота с геоданными в MongoDb
Работа с геоданными в MongoDb
 
Обзор Linux Control Groups
Обзор Linux Control GroupsОбзор Linux Control Groups
Обзор Linux Control Groups
 
Open Source implementation of ZigBee
Open Source implementation of ZigBeeOpen Source implementation of ZigBee
Open Source implementation of ZigBee
 
Пространства имен Linux (linux namespaces)
Пространства имен Linux (linux namespaces)Пространства имен Linux (linux namespaces)
Пространства имен Linux (linux namespaces)
 
Virtual-HSM: Virtualization of Hardware Security Modules in Linux Containers
Virtual-HSM: Virtualization of Hardware Security Modules in Linux ContainersVirtual-HSM: Virtualization of Hardware Security Modules in Linux Containers
Virtual-HSM: Virtualization of Hardware Security Modules in Linux Containers
 
Кратчайшее введение в docker по-русски
Кратчайшее введение в docker по-русскиКратчайшее введение в docker по-русски
Кратчайшее введение в docker по-русски
 
Block-level compression in Linux. Pro et contra
Block-level compression in Linux. Pro et contraBlock-level compression in Linux. Pro et contra
Block-level compression in Linux. Pro et contra
 
Студентам и не только. Как выступить с докладом по своей научной работе
Студентам и не только. Как выступить с докладом по своей научной работеСтудентам и не только. Как выступить с докладом по своей научной работе
Студентам и не только. Как выступить с докладом по своей научной работе
 

Similar to [MDBCI] Mariadb continuous integration tool

Similar to [MDBCI] Mariadb continuous integration tool (20)

DB proxy server test: run tests on tens of virtual machines with Jenkins, Vag...
DB proxy server test: run tests on tens of virtual machines with Jenkins, Vag...DB proxy server test: run tests on tens of virtual machines with Jenkins, Vag...
DB proxy server test: run tests on tens of virtual machines with Jenkins, Vag...
 
Philly security shell meetup
Philly security shell meetupPhilly security shell meetup
Philly security shell meetup
 
Network Automation Tools
Network Automation ToolsNetwork Automation Tools
Network Automation Tools
 
Automation day red hat ansible
   Automation day red hat ansible    Automation day red hat ansible
Automation day red hat ansible
 
Cooking with Chef
Cooking with ChefCooking with Chef
Cooking with Chef
 
Bare Metal to OpenStack with Razor and Chef
Bare Metal to OpenStack with Razor and ChefBare Metal to OpenStack with Razor and Chef
Bare Metal to OpenStack with Razor and Chef
 
Deploying windows containers with kubernetes
Deploying windows containers with kubernetesDeploying windows containers with kubernetes
Deploying windows containers with kubernetes
 
Postgres the hardway
Postgres the hardwayPostgres the hardway
Postgres the hardway
 
Software Defined Datacenter
Software Defined DatacenterSoftware Defined Datacenter
Software Defined Datacenter
 
Automação do físico ao NetSecDevOps
Automação do físico ao NetSecDevOpsAutomação do físico ao NetSecDevOps
Automação do físico ao NetSecDevOps
 
Building cloud stack at scale
Building cloud stack at scaleBuilding cloud stack at scale
Building cloud stack at scale
 
Test-Driven Infrastructure with Chef
Test-Driven Infrastructure with ChefTest-Driven Infrastructure with Chef
Test-Driven Infrastructure with Chef
 
Fullstack conf 2017 - Basic dev pipeline end-to-end
Fullstack conf 2017 - Basic dev pipeline end-to-endFullstack conf 2017 - Basic dev pipeline end-to-end
Fullstack conf 2017 - Basic dev pipeline end-to-end
 
Automating Compliance with InSpec - Chef Singapore Meetup
Automating Compliance with InSpec - Chef Singapore MeetupAutomating Compliance with InSpec - Chef Singapore Meetup
Automating Compliance with InSpec - Chef Singapore Meetup
 
Artem Zhurbila - docker clusters (solit 2015)
Artem Zhurbila - docker clusters (solit 2015)Artem Zhurbila - docker clusters (solit 2015)
Artem Zhurbila - docker clusters (solit 2015)
 
Automating Container Deployments on Virtualization with Ansible: OpenShift on...
Automating Container Deployments on Virtualization with Ansible: OpenShift on...Automating Container Deployments on Virtualization with Ansible: OpenShift on...
Automating Container Deployments on Virtualization with Ansible: OpenShift on...
 
AWS Lambda with Serverless Framework and Java
AWS Lambda with Serverless Framework and JavaAWS Lambda with Serverless Framework and Java
AWS Lambda with Serverless Framework and Java
 
Deploying your web application with AWS ElasticBeanstalk
Deploying your web application with AWS ElasticBeanstalkDeploying your web application with AWS ElasticBeanstalk
Deploying your web application with AWS ElasticBeanstalk
 
EC2 AMI Factory with Chef, Berkshelf, and Packer
EC2 AMI Factory with Chef, Berkshelf, and PackerEC2 AMI Factory with Chef, Berkshelf, and Packer
EC2 AMI Factory with Chef, Berkshelf, and Packer
 
Antons Kranga Building Agile Infrastructures
Antons Kranga   Building Agile InfrastructuresAntons Kranga   Building Agile Infrastructures
Antons Kranga Building Agile Infrastructures
 

More from OSLL

Fruct14 sholokhova
Fruct14 sholokhovaFruct14 sholokhova
Fruct14 sholokhova
OSLL
 
SECR'13 Lightweight linux shared libraries profiling
SECR'13 Lightweight linux shared libraries profilingSECR'13 Lightweight linux shared libraries profiling
SECR'13 Lightweight linux shared libraries profiling
OSLL
 
Smart-M3 and Geo2Tag integration
Smart-M3 and Geo2Tag integrationSmart-M3 and Geo2Tag integration
Smart-M3 and Geo2Tag integration
OSLL
 
HTML5 Intro and Tizen Web API
HTML5 Intro and Tizen Web APIHTML5 Intro and Tizen Web API
HTML5 Intro and Tizen Web API
OSLL
 
Json protocol, Geo2tag REST API fundamentals
Json protocol, Geo2tag REST API fundamentalsJson protocol, Geo2tag REST API fundamentals
Json protocol, Geo2tag REST API fundamentals
OSLL
 
Introduction to geo-tagging and geo2tag platform
Introduction to geo-tagging and geo2tag platformIntroduction to geo-tagging and geo2tag platform
Introduction to geo-tagging and geo2tag platform
OSLL
 
Detection pulse by video
Detection pulse by video Detection pulse by video
Detection pulse by video
OSLL
 
Using Intel NAS-PT for testing NAS disks
Using Intel NAS-PT for testing NAS disksUsing Intel NAS-PT for testing NAS disks
Using Intel NAS-PT for testing NAS disks
OSLL
 
Geo2Tag LBS Platform Architecture Overview
Geo2Tag LBS Platform Architecture OverviewGeo2Tag LBS Platform Architecture Overview
Geo2Tag LBS Platform Architecture Overview
OSLL
 

More from OSLL (16)

SLAM Constructor Framework for ROS
SLAM Constructor Framework for ROSSLAM Constructor Framework for ROS
SLAM Constructor Framework for ROS
 
Microservice architecture for Geo2Tag
Microservice architecture for Geo2TagMicroservice architecture for Geo2Tag
Microservice architecture for Geo2Tag
 
Fast Artificial Landmark Detection for indoor mobile robots AIMAVIG'2015
Fast Artificial Landmark Detection for indoor mobile robots AIMAVIG'2015Fast Artificial Landmark Detection for indoor mobile robots AIMAVIG'2015
Fast Artificial Landmark Detection for indoor mobile robots AIMAVIG'2015
 
Linuxvirt seminar-csc-2015
Linuxvirt seminar-csc-2015Linuxvirt seminar-csc-2015
Linuxvirt seminar-csc-2015
 
Raspberry Pi robot with ROS
Raspberry Pi robot with ROSRaspberry Pi robot with ROS
Raspberry Pi robot with ROS
 
Source code analyzer
Source code analyzer Source code analyzer
Source code analyzer
 
Fruct14 sholokhova
Fruct14 sholokhovaFruct14 sholokhova
Fruct14 sholokhova
 
SECR'13 Lightweight linux shared libraries profiling
SECR'13 Lightweight linux shared libraries profilingSECR'13 Lightweight linux shared libraries profiling
SECR'13 Lightweight linux shared libraries profiling
 
Smart-M3 and Geo2Tag integration
Smart-M3 and Geo2Tag integrationSmart-M3 and Geo2Tag integration
Smart-M3 and Geo2Tag integration
 
HTML5 Intro and Tizen Web API
HTML5 Intro and Tizen Web APIHTML5 Intro and Tizen Web API
HTML5 Intro and Tizen Web API
 
Fruct13 geo2tag-training
Fruct13 geo2tag-trainingFruct13 geo2tag-training
Fruct13 geo2tag-training
 
Json protocol, Geo2tag REST API fundamentals
Json protocol, Geo2tag REST API fundamentalsJson protocol, Geo2tag REST API fundamentals
Json protocol, Geo2tag REST API fundamentals
 
Introduction to geo-tagging and geo2tag platform
Introduction to geo-tagging and geo2tag platformIntroduction to geo-tagging and geo2tag platform
Introduction to geo-tagging and geo2tag platform
 
Detection pulse by video
Detection pulse by video Detection pulse by video
Detection pulse by video
 
Using Intel NAS-PT for testing NAS disks
Using Intel NAS-PT for testing NAS disksUsing Intel NAS-PT for testing NAS disks
Using Intel NAS-PT for testing NAS disks
 
Geo2Tag LBS Platform Architecture Overview
Geo2Tag LBS Platform Architecture OverviewGeo2Tag LBS Platform Architecture Overview
Geo2Tag LBS Platform Architecture Overview
 

Recently uploaded

Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 

Recently uploaded (20)

Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
Agentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdfAgentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdf
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024
 
Buy Epson EcoTank L3210 Colour Printer Online.pdf
Buy Epson EcoTank L3210 Colour Printer Online.pdfBuy Epson EcoTank L3210 Colour Printer Online.pdf
Buy Epson EcoTank L3210 Colour Printer Online.pdf
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
 

[MDBCI] Mariadb continuous integration tool

  • 1. 11th Central and Eastern European Software Engineering Conference in Russia - CEE-SECR 2015 October 22 - 24, Moscow Timofey Turenko Automatic tool for multi-configuration environment creation for database server and database proxy application testing MariaDB Corporation AB Kirill Krinkin St-Petersburg Electrotechnical University
  • 3. Test environment problem  Server and proxy application testing: – backend is needed – lots of combination of machine configurations and version of backend servers  Maxscale example: – 12 major linux distributions – MariaDB 5.5, 10.0, 10.1, 10.2 – MySQL 5.1, 5.5, 5.6, 5.7 – different topologies – Maxscale itself from CI
  • 4. Requirements  arbitrary number of backend servers;  any version of server software can be installed on every backend server;  different virtual machine providers have to be supported (Virtual Box, Qemu, different cloud providers);  deployment of test environment have to be automated;  several test round can be executed in parallel;  test environment description have to be clear and human readable.
  • 6. VM managers, SW manager  Vagrant, Terraform, Ansible  Chef, Puppet works? try to write/read it: ### Import AWS Provider access config ### require 'yaml' aws_config = YAML.load_file("../aws-config.yml")['aws'] ## of import AWS Provider access config ### Vagrant configuration block ### #################################### Vagrant.configure(2) do |config| ### AWS Provider config block ### ############################################# config.vm.box = "dummy" config.vm.provider :aws do |aws, override| aws.access_key_id = aws_config["access_key_id"] aws.secret_access_key = aws_config["secret_access_key"] aws.keypair_name = aws_config["keypair_name"] aws.region = aws_config["region"] aws.security_groups = aws_config["security_groups"] aws.user_data = aws_config["user_data"] override.ssh.private_key_path = aws_config["pemfile"] override.nfs.functional = false end ## of AWS Provider config block config.vm.synced_folder "/home/vagrant/build-scripts/test-setup-... config.vm.define :node0 do |node0| node0.vm.provider :aws do |aws,override| aws.ami = "ami-1c2e8b6b" aws.instance_type = "t1.micro" override.ssh.username = "ec2-user" end ##--- Chef binding --- node0.vm.provision "chef_solo" do |chef| chef.cookbooks_path = "../recipes/cookbooks/" chef.roles_path = "." chef.add_role "node0" chef.synced_folder_type = "rsync" end #<-- end of chef binding end # -> End definition for machine: node0 config.vm.synced_folder "/home/vagrant/build-scripts/test-setup-... # -> Begin definition for machine: node1 config.vm.define :node1 do |node1| node1.vm.provider :aws do |aws,override| aws.ami = "ami-1c2e8b6b" aws.instance_type = "t1.micro" override.ssh.username = "ec2-user" end ##--- Chef binding ---
  • 7. Make it readable "node0" : { "hostname" : "node0", "box" : "centos7", "product" : { "name": "mariadb", "version": "10.0", "cnf_template" : "server1.cnf", "cnf_template_path": "/home/vagrant/cnf" } }, "node1" : { "hostname" : "node1", "box" : "rhel7", "product" : { "name": "mysql", "version": "5.6", "cnf_template" : "server2.cnf", "cnf_template_path": "/home/vagrant//cnf" } } "node2" : { "hostname" : "node2", "box" : "centos7", "product" : { "name": "galera", "version": "5.5", "cnf_template" : "server-galera3.cnf", "cnf_template_path": "/home/vagrant//cnf" } }
  • 8. boxes.json { "ubuntu_precise": { "provider": "virtualbox", "box": "chef/ubuntu-12.04", "platform": "ubuntu", "platform_version": "precise" }, "centos7" : { "provider": "aws", "ami": "ami-1c2e8b6b", "user": "ec2-user", "default_instance_type": "t1.micro", "platform": "centos", "platform_version": "7" } }
  • 9. Repository description { "product": "mariadb", "version": "10.0.17", "repo": "http://yum.mariadb.org/10.0.17/centos5-amd64", "repo_key": "https://yum.mariadb.org/RPM-GPG-KEY-MariaDB", "platform": "centos", "platform_version": 5} { "product": "mysql", "version": "5.6", "repo": "http://repo.mysql.com/yum/mysql-5.6-community/sles/12/x86_64", "repo_key": "http://repo.mysql.com/RPM-GPG-KEY-mysql", "platform": "sles", "platform_version": 12} We have a generator for MariaDB, MariaDB enterprise, MariaDB Galera, Maxscale, MySQL { "product": "mariadb", "version": "10.0", "repo": "http://yum.mariadb.org/10.0/centos5-amd64", "repo_key": "https://yum.mariadb.org/RPM-GPG-KEY-MariaDB", "platform": "centos", "platform_version": 5}
  • 10. Workflow  write template  './mdbci generate <name>'  './mdbci up <name>'  Get all needed info: './mdbci show …'  Access machines: './mdbci ssh' or directly via 'ssh'  './mdbci destory <name>'
  • 11. Does it work?  9 and 25 machines configurations  40 test runs every day  MariaDB, Maxscale, MariaDB enterprise, MariaDB-Galera  VirtualBox, Amazon EC2, QEMU (plans: Docker, DigitalOcean)  Developers like it: “use ./mdbci ssh and see all log, states, core dumps, use my env for debugging”