Treat your servers like your Ruby
App: Infrastructure as Code
2015 March 27-28
Allan Espinosa
Platform as a Service Team
Global Operations Department
Rakuten, Inc.
http://www.rakuten.co.jp/
About me
 Allan Espinosa
 Technology Background
 Language Polyglot: Ruby, Node.js, Java, .NET
 Systems Polyglot: Docker, VMWare
 Automation Junkie: Chef, Jenkins
 Job Background
 “Private Heroku”
 Senior Systems Developer
Platform as a Service Team
Global Operations Department
@AllanEspinosa
Rakuten, Inc.
Agenda
 Audience Demographics
 Typical App TDD
 Introduction to Configuration Management
 Infrastructure as Code
 Evolved App and Infrastructure TDD
 Wrap-up
Audience Demographics
 Web application developers?
 Infrastructure engineers?
 Fullstack engineers?
 Unit Tests?
 Test Driven Development?
 Automated/ Push-button deployments?
 Post-deployment end-to-end tests?
Infrastructure as Code
 Reproducable servers
 Bring up a lot of servers
 Programmatically add servers to monitor/ load balancer
Server as cattle not pets
Reference: Test Driven Infrastructure with Chef
Unmaintained Infrastructure Code
 Where did I install ruby-2.0.0-p589 to?
 Which machines upgraded to ruby-2.1.1-p589?
 Have I upgraded libxml2 for nokogiri?
 Why is database.yml pointing to staging????
 Test and software engineering practices still relevant!
# FIXME:
…
# TODO:
…
Typical TDD Cycle
Make the test pass
Refactor
Write a failing test
Application
Ref: K. Beck, Test Driven Development by Example, 2003.
Extended TDD with Acceptance Tests
Pass the
unit test
Refactor
Write a failing
unit test
Write Failing
Acceptance Test
Application
Capybara.current_driver = :selenium
Capybara.app_host = ‘http://localhost:8000’
G. Adzic, Specification by Example, 2011.
Apply to your Infrastructure
Pass the
unit test
Refactor
Write a failing
unit test
Write Failing
Acceptance Test
Application
+
Servers
Capybara.current_driver = :selenium
Capybara.app_host = ‘http://actual-production.domain
Build our infrastructure!
.kitchen.yml
---
provisioner: chef_zero
platforms:
- freebsd-10.1
suites:
- name: app
 Write a failing test
 Test-kitchen
 Plain Minitest
 Make the test pass
 Chef Recipe
 Refactor
 Repeat!
 Roll out to servers
 Pass acceptance test!
test/integration/app/minitest/test_app.rb
class TestApp < Minitest::Unit::TestCase
def test_ruby_is_installed
assert File.exists
‘/usr/local/bin/ruby’, ‘Ruby should be
installed’
end
end
Failing Test!
-----> Running minitest test suite
/opt/chef/embedded/bin/ruby -I"/opt/chef/embedded/lib/ruby/2.1.0"
"/opt/chef/embedded/lib/ruby/2.1.0/rake/rake_test_loader.rb"
"/tmp/busser/suites/minitest/test_app.rb"
Run options: --seed 48374
# Running tests:
F
Finished tests in 0.001660s, 602.4866 tests/s, 602.4866 assertions/s.
1) Failure:
AppTest#test_ruby_should_be_installed [/tmp/busser/suites/minitest/test_app.rb:5]:
ruby is not installed
1 tests, 1 assertions, 1 failures, 0 errors, 0 skips
/opt/chef/embedded/lib/ruby/2.1.0/rake/testtask.rb:106:in `block (3 levels) in define':
Command failed with status (1): [ruby -I"/opt/chef/embedded/lib/ruby/2.1.0" "/opt/che
f/embedded/lib/ruby/2.1.0/rake/rake_test_loader.rb" "/tmp/busser/suites/minitest/test_app.rb" ]
(RuntimeError)
Build our infrastructure!
.kitchen.yml
---
provisioner: chef_zero
platforms:
- freebsd-10.1
suites:
- name: app
run_list:
- recipe[todo::app]
 Write a failing test
 Test-kitchen
 Plain Minitest
 Make the test pass
 Chef Recipe
 Refactor
 Repeat!
 Roll out to servers
 Pass acceptance test!
Build our infrastructure!
recipes/app.rb
execute ‘pkg update’
package ‘ruby’ do
action :install
end
metadata.rb
name ‘todo’
 Write a failing test
 Test-kitchen
 Plain Minitest
 Make the test pass
 Chef Recipe
 Refactor
 Repeat!
 Roll out to servers
 Pass acceptance test!
Make the test pass
resolving cookbooks for run list: ["todo::app"]
Synchronizing Cookbooks:
- todo
Compiling Cookbooks...
Converging 2 resources
Recipe: todo::app
* execute[pkg update] action run
- execute pkg update
* package[ruby] action install
- install version 2.0.0.598_2,1 of package ruby
Running handlers:
Running handlers complete
Chef Client finished, 2/2 resources updated in 47.595913304
seconds
Finished converging <app-freebsd-101> (0m55.78s).
GREEN! Make the test pass
Run options: --seed 4256
# Running tests:
.
Finished tests in 0.001526s, 655.4072 tests/s, 655.4072
assertions/s.
1 tests, 1 assertions, 0 failures, 0 errors, 0 skips
Finished verifying <app-freebsd-101> (0m2.59s).
Conclusion
 Infrastructure express in code
 Leapfrog on software engineering disciplines
We’re Hiring!
Search for “Platform as a Service”
http://global.rakuten.com/corp/careers/engineering/
Thank you! Questions?

Treat your servers like your Ruby App: Infrastructure as Code

  • 1.
    Treat your serverslike your Ruby App: Infrastructure as Code 2015 March 27-28 Allan Espinosa Platform as a Service Team Global Operations Department Rakuten, Inc. http://www.rakuten.co.jp/
  • 2.
    About me  AllanEspinosa  Technology Background  Language Polyglot: Ruby, Node.js, Java, .NET  Systems Polyglot: Docker, VMWare  Automation Junkie: Chef, Jenkins  Job Background  “Private Heroku”  Senior Systems Developer Platform as a Service Team Global Operations Department @AllanEspinosa
  • 3.
  • 4.
    Agenda  Audience Demographics Typical App TDD  Introduction to Configuration Management  Infrastructure as Code  Evolved App and Infrastructure TDD  Wrap-up
  • 5.
    Audience Demographics  Webapplication developers?  Infrastructure engineers?  Fullstack engineers?  Unit Tests?  Test Driven Development?  Automated/ Push-button deployments?  Post-deployment end-to-end tests?
  • 6.
    Infrastructure as Code Reproducable servers  Bring up a lot of servers  Programmatically add servers to monitor/ load balancer Server as cattle not pets Reference: Test Driven Infrastructure with Chef
  • 7.
    Unmaintained Infrastructure Code Where did I install ruby-2.0.0-p589 to?  Which machines upgraded to ruby-2.1.1-p589?  Have I upgraded libxml2 for nokogiri?  Why is database.yml pointing to staging????  Test and software engineering practices still relevant! # FIXME: … # TODO: …
  • 8.
    Typical TDD Cycle Makethe test pass Refactor Write a failing test Application Ref: K. Beck, Test Driven Development by Example, 2003.
  • 9.
    Extended TDD withAcceptance Tests Pass the unit test Refactor Write a failing unit test Write Failing Acceptance Test Application Capybara.current_driver = :selenium Capybara.app_host = ‘http://localhost:8000’ G. Adzic, Specification by Example, 2011.
  • 10.
    Apply to yourInfrastructure Pass the unit test Refactor Write a failing unit test Write Failing Acceptance Test Application + Servers Capybara.current_driver = :selenium Capybara.app_host = ‘http://actual-production.domain
  • 11.
    Build our infrastructure! .kitchen.yml --- provisioner:chef_zero platforms: - freebsd-10.1 suites: - name: app  Write a failing test  Test-kitchen  Plain Minitest  Make the test pass  Chef Recipe  Refactor  Repeat!  Roll out to servers  Pass acceptance test! test/integration/app/minitest/test_app.rb class TestApp < Minitest::Unit::TestCase def test_ruby_is_installed assert File.exists ‘/usr/local/bin/ruby’, ‘Ruby should be installed’ end end
  • 12.
    Failing Test! -----> Runningminitest test suite /opt/chef/embedded/bin/ruby -I"/opt/chef/embedded/lib/ruby/2.1.0" "/opt/chef/embedded/lib/ruby/2.1.0/rake/rake_test_loader.rb" "/tmp/busser/suites/minitest/test_app.rb" Run options: --seed 48374 # Running tests: F Finished tests in 0.001660s, 602.4866 tests/s, 602.4866 assertions/s. 1) Failure: AppTest#test_ruby_should_be_installed [/tmp/busser/suites/minitest/test_app.rb:5]: ruby is not installed 1 tests, 1 assertions, 1 failures, 0 errors, 0 skips /opt/chef/embedded/lib/ruby/2.1.0/rake/testtask.rb:106:in `block (3 levels) in define': Command failed with status (1): [ruby -I"/opt/chef/embedded/lib/ruby/2.1.0" "/opt/che f/embedded/lib/ruby/2.1.0/rake/rake_test_loader.rb" "/tmp/busser/suites/minitest/test_app.rb" ] (RuntimeError)
  • 13.
    Build our infrastructure! .kitchen.yml --- provisioner:chef_zero platforms: - freebsd-10.1 suites: - name: app run_list: - recipe[todo::app]  Write a failing test  Test-kitchen  Plain Minitest  Make the test pass  Chef Recipe  Refactor  Repeat!  Roll out to servers  Pass acceptance test!
  • 14.
    Build our infrastructure! recipes/app.rb execute‘pkg update’ package ‘ruby’ do action :install end metadata.rb name ‘todo’  Write a failing test  Test-kitchen  Plain Minitest  Make the test pass  Chef Recipe  Refactor  Repeat!  Roll out to servers  Pass acceptance test!
  • 15.
    Make the testpass resolving cookbooks for run list: ["todo::app"] Synchronizing Cookbooks: - todo Compiling Cookbooks... Converging 2 resources Recipe: todo::app * execute[pkg update] action run - execute pkg update * package[ruby] action install - install version 2.0.0.598_2,1 of package ruby Running handlers: Running handlers complete Chef Client finished, 2/2 resources updated in 47.595913304 seconds Finished converging <app-freebsd-101> (0m55.78s).
  • 16.
    GREEN! Make thetest pass Run options: --seed 4256 # Running tests: . Finished tests in 0.001526s, 655.4072 tests/s, 655.4072 assertions/s. 1 tests, 1 assertions, 0 failures, 0 errors, 0 skips Finished verifying <app-freebsd-101> (0m2.59s).
  • 17.
    Conclusion  Infrastructure expressin code  Leapfrog on software engineering disciplines
  • 18.
    We’re Hiring! Search for“Platform as a Service” http://global.rakuten.com/corp/careers/engineering/
  • 19.