Me and lxc-containers
2011 Hyves (now part of TMG)
chat service - python service, single threaded (GIL)
Solution:
lxc-cont...
2013 SpilGames
a global network of online gaming platforms
(180 million active users each month) and
growing
Technology us...
Openstack at SpilGames

Swift in production since Diablo 2011
Swift - highly available, distributed, eventually
consistent...
Goal: IaaS, migrate most of production
systems to private cloud

Ultimate Goal: PaaS, Autoscaling
SpilGames infrastructure today
Two locations: Amsterdam, Toronto
Two openstack availability zones
Both sites serve live tr...
Puppet Intro
$puppet apply test.pp

Manifest file test.pp:
file {'testfile':
path => '/tmp/testfile',
ensure => present,
m...
Puppet in SpilGames
Every server has puppet role registered and assigned in
CMDB (Configuration Management Databse):
hadoo...
Role classes are used as containers for the rest of puppet
classes which provide actual configuration:
class roles::gerrit...
Role classes include "roles::spil_base" class
class roles::gerrit_standalone inherits
roles::spil_base {
...
}

This "spil...
Code Verification
We use Gerrit to run: syntax check (puppetlint), parser validation (puppet parser)
Syntax checks, code v...
Functional testing
Launch host -> Confgure host by applying specific puppet
module -> Verify result
Functional testing to ...
Available tools
rspec-system - framework for creating system tests using the rspec
testing library
launch tests nodes
● co...
Why Docker
lxc-containers – light, fast, resource efficient, possible to
bind external directories, support snapshots, goo...
Prepare
Initial image
(ssh, puppet)

Test Baseline image
(roles::spil_base)
Snapshot
Baseline image

Test Module
(mysql_ma...
Jenkins build results
https://github.com/lruslan/puppet_test
Orchestration script has the following features:
●

●

●

●

●

●

‘full’ mode: fin...
Future plans
Our orchestration script is still proof of concept but already
serves us very well.
Next step would be bring ...
Docker + Stackato = PaaS ?
●

Stackato started in 2012

●

Stackato 3.0 switched to Docker containers

●

Uses Cloud Found...
Spil Games are hiring
Docker meetup
Upcoming SlideShare
Loading in …5
×

Docker meetup

971 views
811 views

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
971
On SlideShare
0
From Embeds
0
Number of Embeds
305
Actions
Shares
0
Downloads
8
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Docker meetup

  1. 1. Me and lxc-containers 2011 Hyves (now part of TMG) chat service - python service, single threaded (GIL) Solution: lxc-containers (since 2010!!!): 100+ instances AUFS: advanced multi layered unification filesystem Gentoo: patched kernel
  2. 2. 2013 SpilGames a global network of online gaming platforms (180 million active users each month) and growing Technology used: Production: PHP, Erlang Internally: Python Platform: Scientific Linux Private cloud: Openstack
  3. 3. Openstack at SpilGames Swift in production since Diablo 2011 Swift - highly available, distributed, eventually consistent object/blob store Compute in production since Folsom end of 2012 Compute - cloud computing fabric controller
  4. 4. Goal: IaaS, migrate most of production systems to private cloud Ultimate Goal: PaaS, Autoscaling
  5. 5. SpilGames infrastructure today Two locations: Amsterdam, Toronto Two openstack availability zones Both sites serve live traffic
  6. 6. Puppet Intro $puppet apply test.pp Manifest file test.pp: file {'testfile': path => '/tmp/testfile', ensure => present, mode => 0640, content => "I'm a test file.", } service { 'ntp': ensure => running, enable => true, }
  7. 7. Puppet in SpilGames Every server has puppet role registered and assigned in CMDB (Configuration Management Databse): hadoop_datanode gerrit_standalone During puppet run, role gets resolved and mapped to the puppet class with the same name: class roles::hadoop_datanode class roles::gerrit_standalone
  8. 8. Role classes are used as containers for the rest of puppet classes which provide actual configuration: class roles::gerrit_standalone inherits roles::spil_base { class{ '::nginx': port => 80, } class{ '::gerrit': mysql_host mysql_user mysql_password mysql_db } => '127.0.0.1', => 'gerrit2', => 'gerrit2', => 'reviewdb', class { '::mysql::server':} mysql::db { 'reviewdb':} Class['::nginx'] → Mysql::Db['reviewdb'] → Class['::Gerrit'] }
  9. 9. Role classes include "roles::spil_base" class class roles::gerrit_standalone inherits roles::spil_base { ... } This "spil_base" class provide minimal necessary baseline configuration: ntp, accounts, sysctl, security etc
  10. 10. Code Verification We use Gerrit to run: syntax check (puppetlint), parser validation (puppet parser) Syntax checks, code verification is very fast (< 10 seconds) and initiated by Gerrit commit hooks
  11. 11. Functional testing Launch host -> Confgure host by applying specific puppet module -> Verify result Functional testing to be useful require: 1) Every single test has to be run inside isolated virtual environments 2) Each module has to be tested on freshly provisioned host 3) Time to run tests should be minimal: < 5-7 min 4) Everything should be automated Ideally we'd like to test modified puppet modules during Gerrit verification.
  12. 12. Available tools rspec-system - framework for creating system tests using the rspec testing library launch tests nodes ● copy test content and execute commands on such nodes to be tested with standard rspec assertions within the standard rspec test format ● Supports: Vagrant + VirtualBox/VSphere, Openstack Problems: ● both VirtualBox/VSphere and Openstack are slow and require resource intensive virtaulziation ● functionality is limited
  13. 13. Why Docker lxc-containers – light, fast, resource efficient, possible to bind external directories, support snapshots, good python library Puppet roles share same baseline configuration! Main benefits: ● ● Having SNAPSHOT of baseline system we can use it later to spawn containers and use them to test the rest of the modules. Because of low resource overhead containers with tests can be run in parallel
  14. 14. Prepare Initial image (ssh, puppet) Test Baseline image (roles::spil_base) Snapshot Baseline image Test Module (mysql_master) Test Module (nginx) Analyze results Test Module (hadoop)
  15. 15. Jenkins build results
  16. 16. https://github.com/lruslan/puppet_test Orchestration script has the following features: ● ● ● ● ● ● ‘full’ mode: find puppet modules, build base docker image using base module and run tests for the rest of the modules ‘quick’ mode: reuse previously created base image and run tests for the modules parallel mode: it’s possible to specify the number of workers (so multiple tests run in a parallel) Jenkins integration: detect which Puppet modules have been changed since last jenkins build Results publishing: generates html report with results and ability to see details (stdout/stderr) of every test ability to set timer and stop containers if the test takes longer than expected
  17. 17. Future plans Our orchestration script is still proof of concept but already serves us very well. Next step would be bring similar functionality to one of existing rspec tools used by community. Ooops … rspec-system retired 10 days ago :( Now official tool for the rspec tool is: Beaker (beaker-rspec) https://github.com/puppetlabs/beaker
  18. 18. Docker + Stackato = PaaS ? ● Stackato started in 2012 ● Stackato 3.0 switched to Docker containers ● Uses Cloud Foundry v2 - open source PaaS, developed by Vmware Evaluating … to be continued
  19. 19. Spil Games are hiring

×