Your SlideShare is downloading. ×
Practical introduction to dev ops with chef
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Practical introduction to dev ops with chef

11,287

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
11,287
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
43
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • \n
  • \n
  • Agile Software Studio & Coaching\nWeb, Mobile, Embedded Desktop\nCommunity Meetups, GiveCamp\n
  • What is DevOps?\n\n
  • traditionally separate teams\nDevelopment - creates software\nOperations - puts it intro production and keeps it running\nfigurative (sometimes literal) wall between \n
  • Development - creates software\nOperations - puts it intro production and keeps it running\n
  • portmanteau word capturing the idea of integrated development AND operations\nmates with Agile/Lean principles\nfrequent releases, shorter cycle times\nautomation\n“holistic” view of development and production\n
  • \n
  • \n
  • To make this discussion concrete and practical, we’ll be using a small web application for the examples. \n
  • using Apache to serve our static content and mod_wsgi to serve our Python Django app\nthere are lots of different deployment options here (nginx, gunicorn, etc.); this is pretty common\nusing postgresql for the Django persistence\n
  • \n
  • in our configuration runs as a daemon under Apache\n
  • we’re using postgresql\n
  • \n
  • But often necessary\n
  • \n
  • \n
  • \n
  • \n
  • package name == package version\ndjango - the web framework we’re using\npsycopg2 is a Python client interface for postgresql\n\n
  • \n
  • \n
  • editing lots of text files...\n
  • \n
  • Now we’re almost ready to actually do something\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • very simple Dept/Course/Student/Instructor Roster web application\n
  • \n
  • can always use your instructions to create a master image that can be “frozen” or bundled\n
  • \n
  • \n
  • actually, a great way to start\n
  • \n
  • \n
  • install the needed packages\nthe script is useful and goes a long way to documenting process\n
  • \n
  • \n
  • sure, you do some horrible shell scripts with sed and awk and such, but really?\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • we’d like to declare the state of the resource, like a database username, password, and database access, and let someone else figure out how to “make it so!”\n\nthat’s essentially what Chef is\n
  • very little Ruby experience is required\nothers to be aware of are: cfengine, puppet, and now salt\n
  • Chef Server (opscode or your own)\n“knife” is a Ruby command-line program\nchef-client runs on your “cluster”\n
  • really, it looks like this...\nCouchDB\nsolr\nRabbitMQ\n
  • \n
  • \n
  • \n
  • \n
  • an instance of a particular machine\nMostly Linux centric, but some Windows support\n
  • to name just a few\n
  • \n
  • resource type of “package”\n
  • resource name of “libpq-dev”\n
  • Chef calls yum, apt-get, etc. as appropriate for the platform\n
  • \n
  • \n
  • \n
  • “executed” top to bottom\n
  • \n
  • Yes, they have Django support, but we wouldn’t learn as much from using that\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Test Driven Infrastructure book by O’Reilly\n\n
  • \n
  • Today’s talk intentionally not a tutorial - lots of fine Chef tutorials out there\n
  • Transcript

    • 1. A PRACTICALINTRODUCTION TO DEVOPS WITH CHEF Nick Barendt - LeanDog CWRU ACM Link-State 2012
    • 2. WHO AM I• CWRU Alumnus - Wayward Electrical Engineer• LeanDog - Director of Labs• CWRU EECS Adjunct Faculty• Embedded Linux (Bootloaders, Kernel Drivers, Daemons)• Web/Cloud (AWS, Python Django)
    • 3. LEANDOG http://leandog.comhttp://leandog.com/dogtreats/ Copyright 2012, Nick Barendt
    • 4. DEVOPS
    • 5. DEVELOPMENT + OPERATIONS
    • 6. DEVELOPMENT + OPERATIONS
    • 7. DEVOPS
    • 8. • Agile/Lean Development• Frequent Releases (Minimize WIP)• Continuous Integration → Continuous Delivery• Reduced Stress: Release is Non-Event; TDD/BDD• Holistic - code/infrastructure are inextricably linked
    • 9. TOPICS FOR TODAY• Configuration Management• Deployment Automation
    • 10. SAMPLE APPLICATION• Ubuntu 12.04 LTS• Apache2• Python Django• PostgreSQL
    • 11. Apache2static content mod_wsgi Postgresql
    • 12. STATIC CONTENT• aka “files”• HTML, CSS, JavaScript, images, etc.
    • 13. MOD_WSGI• WSGI - Web Server Gateway Interface • Python Universal HTTP interface• mod_wsgi Apache plugin• executes Python/Django code in response to HTTP requests
    • 14. POSTGRESQL• Relational Database Management System• Django’s Object Relational Mapper (ORM) supports: • Postgresql • MySQL • SQLite • Oracle
    • 15. HOW DO WE DEPLOY?
    • 16. PART 1: THE HARD WAY
    • 17. INSTALL DISTRO PACKAGES
    • 18. CREATE THE VIRTUALENV
    • 19. Virtualenv• Python “environment” isolated from rest of system• Analogous to Ruby’s rvm• Python Interpreter + 3rd party libraries• pip - Python Package Manager• requirements.txt - 3rd party library manifest, with versions
    • 20. requirements.txt django==1.4.2 psycopg2==2.4.5
    • 21. CONFIGURE POSTGRESQL
    • 22. • Configure Postgresql to listen on TCP Port 5432• Configure Postgresql Authentication• Create User “roster”• Create Database “roster”
    • 23. SOURCE!
    • 24. THAT WASN’T PRETTY
    • 25. OR EASILY REPEATABLE
    • 26. PAUSE
    • 27. WHAT DOES THE APP DO?
    • 28. BACK TO THE STORY
    • 29. CONFIGURATION• Repeatable and works on a minimal, standard install image• TIP: Use virtual machines for development/testing: • Snapshot Virtual Machine • Test Configuration Process • Verify and/or Fix Configuration Process • Rollback
    • 30. SNAPSHOT TREE VMware Fusion Copyright 2012, Nick Barendt
    • 31. HOW TO IMPROVE?
    • 32. TAKE A FEW NOTES?
    • 33. WE CAN DO SLIGHTLY BETTER
    • 34. SCRIPTS!
    • 35. “prepare_ubuntu”#!/bin/bash# must be run as root!apt-get install -y apache2 libapache2-mod-wsgiapt-get install -y python-dev python-virtualenvapt-get install -y gitapt-get install -y postgresql-9.1 libpq-devapt-get install -y vim
    • 36. “create_virtualenv”#!/bin/bashENV_DIR=envrm -rf $ENV_DIRvirtualenv $ENV_DIR$ENV_DIR/bin/pip install -r requirements.txt
    • 37. HOW ABOUT CONFIG FILES...
    • 38. YEAH....ABOUT THAT
    • 39. FINALLY, WE CAN TALK ABOUT CHEF
    • 40. PART 2: CHEF!
    • 41. LET’S WAIT JUST A LITTLE BIT LONGER
    • 42. IN AN IDEAL WORLD, WHAT WOULD WE WANT?
    • 43. Configuration IS Source• Configuration in Version Control• Text Files• Diff ’able• Tie Infrastructure Configuration and Production Releases
    • 44. A DECLARATIVE DSLdatabase_user roster do    password spartans    database_name rosterend
    • 45. CHEF• Opscode - Private company• 2009• Open Source - Apache License 2.0• Ruby DSL for configuration “recipes” or “cookbooks”
    • 46. CHEF ARCHITECTURE Chef Server chef client knife
    • 47. CHEF ARCHITECTURE
    • 48. BUT I JUST WANT TOMANAGE A FEW MACHINES...
    • 49. CHEF-SOLO
    • 50. chef-solo• Essentially chef-client, but operates without chef server• Only uses local resources• Simpler to start learning• “Recipes” are inherently the same as “full blown” chef
    • 51. CHEF CONCEPTS• NODES - “servers”, “computers”, “machines”• RESOURCES - abstraction of the “thing” you’re configuring• RECIPES - collection of Resource descriptions• COOKBOOKS - collections of Recipes and Attributes
    • 52. NODES• Ubuntu • Scientific Linux• Debian • OS X• Red Hat • Windows• Fedora• CentOS• Oracle Copyright 2012, Nick Barendt
    • 53. RESOURCES• File • User and Group• Template • Git• Execute • Subversion• Cron• Link• Directory Copyright 2012, Nick Barendt
    • 54. RECIPESpackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
    • 55. RECIPESpackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
    • 56. RECIPESpackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
    • 57. RECIPES Install Distro Packagepackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
    • 58. RECIPES create virtualenv @package "libpq-dev" specified pathpython_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
    • 59. RECIPES some variablespackage "libpq-dev" and Ruby codepython_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
    • 60. RECIPESpackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend execute command-line
    • 61. RECIPESpackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createendpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doend
    • 62. COOKBOOKS• Collections of • Attributes • Recipes • Templates Copyright 2012, Nick Barendt
    • 63. TONS OF COOKBOOKS HTTPS://GITHUB.COM/OPSCODE-COOKBOOKS• Java • Erlang• Python • Django• Nagios • Rsyslog• openssh • Nginx• Apache2 • Perl• Postfix • Many, Many More Copyright 2012, Nick Barendt
    • 64. CONVERGENCE• Chef attempts to converge a machine’s configuration to the desired state• Withtemporary errors, repeated runs will drive towards convergence• Resources are (generally) idempotent• Robust• Try, try again
    • 65. HOW DOES THIS ACTUALLY WORK?
    • 66. • Merge Defaults and Node Attributes• Recipe Compilation• Recipe Execution
    • 67. sudo chef-solo -c solo.rb -j solo-configs/standalone.jsonsolo.rb# minimal configuration file for chef-solocookbook_path File.expand_path File.dirname(__FILE__) + "/cookbooks"standalone.json{ "roster": { "app_server": { "load_default_data": true } }, "postgresql": { "password":{ "postgres": "roster_123" } }, "run_list": [ "recipe[roster::database]", "recipe[roster::app_server]" ]}
    • 68. app_server.rbinclude_recipe "apache2"include_recipe "apache2::mod_wsgi"include_recipe "python"node[:roster]["DEPLOY_DIR"] = File.expand_path( File.join( File.expand_path( File.dirname(__FILE__) ), "..","..","..",".."))node[:roster]["VIRTUALENV_DIR"] = File.expand_path( File.join(node[:roster]["DEPLOY_DIR"], "env"))# disable default, regardless of how it is symlinkedapache_site "default" do    enable falseendapache_site "000-default" do    enable falseend# we need a few packagespackage "libpq-dev"python_virtualenv node[:roster]["VIRTUALENV_DIR"] do    interpreter "python2.7"    action :createend
    • 69. continuedpip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin","pip")requirements_txt = File.join(node[:roster]["DEPLOY_DIR"],"requirements.txt")execute "#{pip_cmd} install -r #{requirements_txt}" doendweb_app "roster" do    template "roster_apache.conf.erb"enddjango_admin_cmd = File.join(node[:roster]["DEPLOY_DIR"],"cwru_acm", "manage.py")python_exec = File.join(node[:roster]["VIRTUALENV_DIR"], "bin","python")execute "#{python_exec} #{django_admin_cmd} syncdb --noinput"doendexecute "#{python_exec} #{django_admin_cmd} collectstatic --noinput" doendexecute "#{python_exec} #{django_admin_cmd} loaddata rostersome_initial_data" do    only_if {node[:roster][:app_server][:load_default_data]}end
    • 70. database.rbinclude_recipe "postgresql::client"include_recipe "postgresql::server"r = package "ruby-pg" do  package_name "libpgsql-ruby"  action :nothingendr.run_action(:upgrade)# Snippet from opscode to reload gemsrequire rubygemsGem.clear_pathsrequire "pg"include_recipe "database"postgresql_connection_info = {:host => "127.0.0.1", :port => 5432, :username =>postgres, :password => node[postgresql][password][postgres]}postgresql_database "roster" do    connection postgresql_connection_info    action :createenddatabase_user roster do    connection postgresql_connection_info    password spartans    database_name roster    provider Chef::Provider::Database::PostgresqlUser    action :createenddatabase_user roster do    connection postgresql_connection_info    password spartans    provider Chef::Provider::Database::PostgresqlUser    action :grantend
    • 71. But Wait, There’s More• Parametric Configs. (environments): Dev, Staging, Production• Roles - collect recipes based on how server is used (e.g., Apache2 + Memcache for “front-end”)• Test Driven Infrastructure - Configuration is just code!• Bootstrapping in AWS EC2 and other Clouds• Chef Server, Knife
    • 72. Questions?nick@barendt.com Twitter: @nickbarendt Code From Today: https://github.com/nbarendt/cwru_acm_2012
    • 73. FOR MORE INFO• http://wiki.opscode.com• Fast Start Linux Guide: http://wiki.opscode.com/display/chef/ Fast+Start+Guide• AWSEC2 Bootstrapping: http://wiki.opscode.com/display/chef/ EC2+Bootstrap+Fast+Start+Guide• O’Reilly: Test-Driven Infrastructure with Chef

    ×