Practical introduction to dev ops with chef
Upcoming SlideShare
Loading in...5
×
 

Practical introduction to dev ops with chef

on

  • 9,325 views

 

Statistics

Views

Total Views
9,325
Slideshare-icon Views on SlideShare
2,527
Embed Views
6,798

Actions

Likes
2
Downloads
32
Comments
0

5 Embeds 6,798

http://www.leandog.com 3546
http://leandog.com 2866
http://localhost 351
http://leandogsoftware.com 26
http://www.leandogsoftware.com 9

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • Agile Software Studio & Coaching\nWeb, Mobile, Embedded Desktop\nCommunity Meetups, GiveCamp\n
  • What is DevOps?\n\n
  • traditionally separate teams\nDevelopment - creates software\nOperations - puts it intro production and keeps it running\nfigurative (sometimes literal) wall between \n
  • Development - creates software\nOperations - puts it intro production and keeps it running\n
  • portmanteau word capturing the idea of integrated development AND operations\nmates with Agile/Lean principles\nfrequent releases, shorter cycle times\nautomation\n“holistic” view of development and production\n
  • \n
  • \n
  • To make this discussion concrete and practical, we’ll be using a small web application for the examples. \n
  • using Apache to serve our static content and mod_wsgi to serve our Python Django app\nthere are lots of different deployment options here (nginx, gunicorn, etc.); this is pretty common\nusing postgresql for the Django persistence\n
  • \n
  • in our configuration runs as a daemon under Apache\n
  • we’re using postgresql\n
  • \n
  • But often necessary\n
  • \n
  • \n
  • \n
  • \n
  • package name == package version\ndjango - the web framework we’re using\npsycopg2 is a Python client interface for postgresql\n\n
  • \n
  • \n
  • editing lots of text files...\n
  • \n
  • Now we’re almost ready to actually do something\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • very simple Dept/Course/Student/Instructor Roster web application\n
  • \n
  • can always use your instructions to create a master image that can be “frozen” or bundled\n
  • \n
  • \n
  • actually, a great way to start\n
  • \n
  • \n
  • install the needed packages\nthe script is useful and goes a long way to documenting process\n
  • \n
  • \n
  • sure, you do some horrible shell scripts with sed and awk and such, but really?\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • we’d like to declare the state of the resource, like a database username, password, and database access, and let someone else figure out how to “make it so!”\n\nthat’s essentially what Chef is\n
  • very little Ruby experience is required\nothers to be aware of are: cfengine, puppet, and now salt\n
  • Chef Server (opscode or your own)\n“knife” is a Ruby command-line program\nchef-client runs on your “cluster”\n
  • really, it looks like this...\nCouchDB\nsolr\nRabbitMQ\n
  • \n
  • \n
  • \n
  • \n
  • an instance of a particular machine\nMostly Linux centric, but some Windows support\n
  • to name just a few\n
  • \n
  • resource type of “package”\n
  • resource name of “libpq-dev”\n
  • Chef calls yum, apt-get, etc. as appropriate for the platform\n
  • \n
  • \n
  • \n
  • “executed” top to bottom\n
  • \n
  • Yes, they have Django support, but we wouldn’t learn as much from using that\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Test Driven Infrastructure book by O’Reilly\n\n
  • \n
  • Today’s talk intentionally not a tutorial - lots of fine Chef tutorials out there\n

Practical introduction to dev ops with chef Practical introduction to dev ops with chef Presentation Transcript

  • A PRACTICALINTRODUCTION TO DEVOPS WITH CHEF Nick Barendt - LeanDog CWRU ACM Link-State 2012
  • WHO AM I• CWRU Alumnus - Wayward Electrical Engineer• LeanDog - Director of Labs• CWRU EECS Adjunct Faculty• Embedded Linux (Bootloaders, Kernel Drivers, Daemons)• Web/Cloud (AWS, Python Django)
  • LEANDOG http://leandog.comhttp://leandog.com/dogtreats/ Copyright 2012, Nick Barendt
  • DEVOPS
  • DEVELOPMENT + OPERATIONS
  • DEVELOPMENT + OPERATIONS
  • DEVOPS
  • • Agile/Lean Development• Frequent Releases (Minimize WIP)• Continuous Integration → Continuous Delivery• Reduced Stress: Release is Non-Event; TDD/BDD• Holistic - code/infrastructure are inextricably linked
  • TOPICS FOR TODAY• Configuration Management• Deployment Automation
  • SAMPLE APPLICATION• Ubuntu 12.04 LTS• Apache2• Python Django• PostgreSQL
  • Apache2static content mod_wsgi Postgresql
  • STATIC CONTENT• aka “files”• HTML, CSS, JavaScript, images, etc.
  • MOD_WSGI• WSGI - Web Server Gateway Interface • Python Universal HTTP interface• mod_wsgi Apache plugin• executes Python/Django code in response to HTTP requests
  • POSTGRESQL• Relational Database Management System• Django’s Object Relational Mapper (ORM) supports: • Postgresql • MySQL • SQLite • Oracle
  • HOW DO WE DEPLOY?
  • PART 1: THE HARD WAY
  • INSTALL DISTRO PACKAGES
  • CREATE THE VIRTUALENV
  • Virtualenv• Python “environment” isolated from rest of system• Analogous to Ruby’s rvm• Python Interpreter + 3rd party libraries• pip - Python Package Manager• requirements.txt - 3rd party library manifest, with versions
  • requirements.txt django==1.4.2 psycopg2==2.4.5
  • CONFIGURE POSTGRESQL
  • • Configure Postgresql to listen on TCP Port 5432• Configure Postgresql Authentication• Create User “roster”• Create Database “roster”
  • SOURCE!
  • THAT WASN’T PRETTY
  • OR EASILY REPEATABLE
  • PAUSE
  • WHAT DOES THE APP DO?
  • BACK TO THE STORY
  • CONFIGURATION• Repeatable and works on a minimal, standard install image• TIP: Use virtual machines for development/testing: • Snapshot Virtual Machine • Test Configuration Process • Verify and/or Fix Configuration Process • Rollback
  • SNAPSHOT TREE VMware Fusion Copyright 2012, Nick Barendt
  • HOW TO IMPROVE?
  • TAKE A FEW NOTES?
  • WE CAN DO SLIGHTLY BETTER
  • SCRIPTS!
  • “prepare_ubuntu”#!/bin/bash# must be run as root!apt-get install -y apache2 libapache2-mod-wsgiapt-get install -y python-dev python-virtualenvapt-get install -y gitapt-get install -y postgresql-9.1 libpq-devapt-get install -y vim
  • “create_virtualenv”#!/bin/bashENV_DIR=envrm -rf $ENV_DIRvirtualenv $ENV_DIR$ENV_DIR/bin/pip install -r requirements.txt
  • HOW ABOUT CONFIG FILES...
  • YEAH....ABOUT THAT
  • FINALLY, WE CAN TALK ABOUT CHEF
  • PART 2: CHEF!
  • LET’S WAIT JUST A LITTLE BIT LONGER
  • IN AN IDEAL WORLD, WHAT WOULD WE WANT?
  • Configuration IS Source• Configuration in Version Control• Text Files• Diff ’able• Tie Infrastructure Configuration and Production Releases
  • A DECLARATIVE DSLdatabase_user roster do    password spartans    database_name rosterend
  • CHEF• Opscode - Private company• 2009• Open Source - Apache License 2.0• Ruby DSL for configuration “recipes” or “cookbooks”
  • CHEF ARCHITECTURE Chef Server chef client knife
  • CHEF ARCHITECTURE
  • BUT I JUST WANT TOMANAGE A FEW MACHINES...
  • CHEF-SOLO
  • chef-solo• Essentially chef-client, but operates without chef server• Only uses local resources• Simpler to start learning• “Recipes” are inherently the same as “full blown” chef
  • CHEF CONCEPTS• NODES - “servers”, “computers”, “machines”• RESOURCES - abstraction of the “thing” you’re configuring• RECIPES - collection of Resource descriptions• COOKBOOKS - collections of Recipes and Attributes
  • NODES• Ubuntu • Scientific Linux• Debian • OS X• Red Hat • Windows• Fedora• CentOS• Oracle Copyright 2012, Nick Barendt
  • RESOURCES• File • User and Group• Template • Git• Execute • Subversion• Cron• Link• Directory Copyright 2012, Nick Barendt
  • RECIPESpackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
  • RECIPESpackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
  • RECIPESpackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
  • RECIPES Install Distro Packagepackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
  • RECIPES create virtualenv @package "libpq-dev" specified pathpython_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
  • RECIPES some variablespackage "libpq-dev" and Ruby codepython_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
  • RECIPESpackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend execute command-line
  • RECIPESpackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
  • COOKBOOKS• Collections of • Attributes • Recipes • Templates Copyright 2012, Nick Barendt
  • TONS OF COOKBOOKS HTTPS://GITHUB.COM/OPSCODE-COOKBOOKS• Java • Erlang• Python • Django• Nagios • Rsyslog• openssh • Nginx• Apache2 • Perl• Postfix • Many, Many More Copyright 2012, Nick Barendt
  • CONVERGENCE• Chef attempts to converge a machine’s configuration to the desired state• Withtemporary errors, repeated runs will drive towards convergence• Resources are (generally) idempotent• Robust• Try, try again
  • HOW DOES THIS ACTUALLY WORK?
  • • Merge Defaults and Node Attributes• Recipe Compilation• Recipe Execution
  • sudo chef-solo -c solo.rb -j solo-configs/standalone.jsonsolo.rb# minimal configuration file for chef-solocookbook_path File.expand_path File.dirname(__FILE__) + "/cookbooks"standalone.json{ "roster": { "app_server": { "load_default_data": true } }, "postgresql": { "password":{ "postgres": "roster_123" } }, "run_list": [ "recipe[roster::database]", "recipe[roster::app_server]" ]}
  • app_server.rbinclude_recipe "apache2"include_recipe "apache2::mod_wsgi"include_recipe "python"node[:roster]["DEPLOY_DIR"] = File.expand_path( File.join( File.expand_path( File.dirname(__FILE__) ), "..","..","..",".."))node[:roster]["VIRTUALENV_DIR"] = File.expand_path( File.join(node[:roster]["DEPLOY_DIR"], "env"))# disable default, regardless of how it is symlinkedapache_site "default" do    enable falseendapache_site "000-default" do    enable falseend# we need a few packagespackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createend
  • continuedpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin","pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"],"requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doendweb_app "roster" do    template "roster_apache.conf.erb"enddjango_admin_cmd = File.join(node[:roster]["DEPLOY_DIR"],"cwru_acm", "manage.py")python_exec = File.join(node[:roster]["VIRTUALENV_DIR"], "bin","python")execute "#{python_exec} #{django_admin_cmd} syncdb --noinput"doendexecute "#{python_exec} #{django_admin_cmd} collectstatic --noinput" doendexecute "#{python_exec} #{django_admin_cmd} loaddata rostersome_initial_data" do    only_if {node[:roster][:app_server][:load_default_data]}end
  • database.rbinclude_recipe "postgresql::client"include_recipe "postgresql::server"r = package "ruby-pg" do  package_name "libpgsql-ruby"  action :nothingendr.run_action(:upgrade)# Snippet from opscode to reload gemsrequire rubygemsGem.clear_pathsrequire "pg"include_recipe "database"postgresql_connection_info = {:host => "127.0.0.1", :port => 5432, :username =>postgres, :password => node[postgresql][password][postgres]}postgresql_database "roster" do    connection postgresql_connection_info    action :createenddatabase_user roster do    connection postgresql_connection_info    password spartans    database_name roster    provider Chef::Provider::Database::PostgresqlUser    action :createenddatabase_user roster do    connection postgresql_connection_info    password spartans    provider Chef::Provider::Database::PostgresqlUser    action :grantend
  • But Wait, There’s More• Parametric Configs. (environments): Dev, Staging, Production• Roles - collect recipes based on how server is used (e.g., Apache2 + Memcache for “front-end”)• Test Driven Infrastructure - Configuration is just code!• Bootstrapping in AWS EC2 and other Clouds• Chef Server, Knife
  • Questions?nick@barendt.com Twitter: @nickbarendt Code From Today: https://github.com/nbarendt/cwru_acm_2012
  • FOR MORE INFO• http://wiki.opscode.com• Fast Start Linux Guide: http://wiki.opscode.com/display/chef/ Fast+Start+Guide• AWSEC2 Bootstrapping: http://wiki.opscode.com/display/chef/ EC2+Bootstrap+Fast+Start+Guide• O’Reilly: Test-Driven Infrastructure with Chef