Infrastructure as
Code
Mike McGarr, Director of Learn DevOps
Blackboard, Inc.
ABOUT ME
Mike McGarr
Director of Learn DevOps
Blackboard Inc.
michael.mcgarr@blackboard.com
I have been at Blackboard for 8 months.
Software Engineer for 12+ years
Founder of the DC Continuous Integration,
Delivery and Deployment Meetup
LEARN DEVOPS TEAM
Purpose: Provide development support to the rest of
Learn‟s Product Development (PD) team.
Includes:
• Build/Release Engineering
• Manage Enterprise Applications
• Manage Data Center
• Help desk for PD
ENTERPRISE
APPLICATIONS
Problems:
• Manually managed
• Little/No Version
Control
• No history/context
DATA CENTER
Problems:
• Few knew how to run
CM
• Fewer knew how to fix
CM tool
• Lots of firefighting to
resolve CM issues
FIREFIGHTING
INFRASTRUCTURE AS
CODE
Automate the creation and maintenance of
servers (to the farthest extent possible)
• Build from source control
• Utilize open source tools
• Ensure testability
A practice popularized by the DevOps
movement.
TOOLS
CONFIGURATION
SYNCHRONIZATION
http://martinfowler.com/bliki/ConfigurationSynchronization.html
CONFIGURATION
DRIFT
http://kief.com/configuration-drift.html
NO MANUAL
CHANGES
http://www.agileweboperations.com/devops-protocol-no-manual-changes
PHOENIX SERVERS
http://martinfowler.com/bliki/PhoenixServer.html
IMMUTABLE SERVERS
http://martinfowler.com/bliki/ImmutableServer.html
REMINDER:
CULTURE FIRST
Simple
Example
Infrastructure as Code
BUILD A
JENKINS SERVER
CHEF‟S
ARCHITECTURE
CREATE COOKBOOK
http://youtu.be/Yu3--aljnJ0
VAGRANT
$> vagrant box add
$> vagrant init
$> vagrant up
$> vagrant ssh
$> vagrant provision
$> vagrant suspend
$> vagrant destroy
VAGRANT UP
http://youtu.be/I0Kd7mzEOD4
USE CHEF
http://youtu.be/UhXWrQvwsyE
INSTALLING JENKINS
INSTALL JDK VIA
CHEF
1. Include Java default recipe (recipe/default.rb)
2. Add cookbook dependency (metadata.rb)
3. Download Java cookbook
4. Run „vagrant provision‟
INSTALL JDK
http://youtu.be/Pt7cg3oa4Hs
0%
http://youtu.be/Pt7cg3oa4Hs
ADD JENKINS USER
http://youtu.be/2VhvHXPKR6M
REUSE LIBRARIES
SECURITY
templates/defaults/port_jenkins.erb
recipes/default.rb
YUM REPOSITORY
recipes/default.rb
INSTALL PACKAGE
SERVICE IS RUNNING
LEARN CHEF
Software Server
Craftsmen
CONFIGURATION
WORKFLOW
VERSION CONTROL
https://www.youtube.com/watch?v=ZDR433b0HJY
$> git init
$> git add .
$> git commit –m “new commit”
$> git remote add git:git@...
$> git push origin master
Chef/Puppet Server
is not Version Control!!
DEPENDENCY
MANAGEMENT
Tools:
Berkshelf
librarian-chef
UNIT TESTING
chefspec
(ruby gem)
https://github.com/acrmp/chefspec
Test a cookbook
fast and from
anywhere
spec/default_spec.rb
CONTINUOUS
TESTING
guard
(ruby gem)
https://github.com/guard/guard
When a local file
changes, then
run unit tests.
Guardfile
INTEGRATION
TESTING
test kitchen
(ruby gem)
https://github.com/opscode/test-kitchen
Run and test
cookbook in
Vagrant or LXC
container.
.kitchen.yml
STATIC ANALYSIS
foodcritic
(ruby gem)
http://acrmp.github.io/foodcritic/
Style guide and
find defects in
cookbook source
code.
CONTINUOUS
INTEGRATION
Jenkins
(Java app)
When a change
is checked in,
then run tests
and deploy to
Chef Server.
Best Practices
Infrastructure as Code
FORWARD ONLY
AVOID SCRIPTING
SEPARATE CONFIG
FROM DATA
USE & BUILD
PACKAGES
https://github.com/jordansissel/fpm
LEARN TO PROGRAM
Ops acting like developers…
Learn the language!
Learn the environment:
• rvm
• rbenv
• gems
• bundler
THANK YOU!
Mike McGarr
Director of Learn DevOps
Blackboard Inc.
michael.mcgarr@blackboard.com
@jmichaelmcgarr
http://earlyandoften.wordpress.com

Infrastructure as Code (BBWorld/DevCon13)

Editor's Notes

  • #6 2000+ virtual machinesHundreds of learn instancesHundreds of test agentsIn-house CM tool
  • #7 Constantly fighting firesResulted from high technical debtImpact to team:High WIPContext switchingLarge backlogImage: http://www.ctif.org/CTIF-Commission-Rescue-and-Fire
  • #9 If your question is Puppet vs. Chef, the answer is yes.Advantages:Provide DSL (easier to read)Community for helpOpen source, corporate backingHandles errors, etc.
  • #10 ConvergenceIdempotence
  • #12 Manual changes lead to driftGoal: Avoid Configuration Drift
  • #13 Fully capable of rebuilding a server from scratch at any timeEasier than troubleshooting unique issuesImage: http://coverlaydown.com/2013/02/the-phoenix-rises-on-coming-back-and-moving-forward/
  • #14 The logical evolution of Phoenix Servers
  • #15 DevOps is not about tools, it’s about culture.…however, tools can change behavior, …and behavior change leads to culture change.
  • #21 Ruby Wrapper to VirtualBox (and others)Simplifies Local VM integrationUnderstands CM ToolsPlugin API
  • #22 Chef client installed on machine<Vagrant boots machine, passes runlist to client>Describe Chef Server briefly
  • #33 Discuss how some developer best practices can/should apply to ops.
  • #34 Depict how changes are made:Change locally, in VagrantUnit test changes/regressionsRun Integration TestsCommit to Git (pre-commit hook to ensure incremented version)Jenkins runs testsIf successful, publish to Chef Server (staging environment)Staging environment updates (daemon or push)Manual testing…if all goodPromote change in Jenkins to production environmentServers update (daemon/push)
  • #35 Communication mechanism
  • #37 Requires:Updates to GemfileTo Run: bundle install && bundle exec rspec --color --format documentationUnit Tests: FIRSTFastIndependentRepeatableSelf-VerifyingTimely
  • #39 Runs
  • #43 Common gotcha…Removing a resource from
  • #44 Script has explicit OS dependencies
  • #45 Don’t forget to backup data prior to upgrade.