Puppet for Production in WebEx - PuppetConf 2013
Upcoming SlideShare
Loading in...5
×
 

Puppet for Production in WebEx - PuppetConf 2013

on

  • 21,190 views

"Puppet for Production in WebEx" by Reinhardt Quelle, Cloud Services Architect, Cisco. ...

"Puppet for Production in WebEx" by Reinhardt Quelle, Cloud Services Architect, Cisco.

Presentation Overview: Getting started with Puppet configuring an individual machine is straightforward. Managing a cluster of machines across multiple data centers, supporting upgrades while running a 7x24 service, and building for collaboration is significantly more challenging. The WebEx team will discuss the problems and some strategies they are using to manage this complexity.

Speaker Bio: Reinhardt Quelle is a Cloud Services Architect in the Cloud Collaboration Applications group at Cisco, where he’s responsible for defining infrastructure architecture and deployment automation . His group manages thousands of servers across multiple data centers around the world serving multiple applications, including WebEx conferencing, to tens of millions of users. In prior roles, he’s worked extensively in SaaS operations, delivering diverse applications from email security through social media applications.

Statistics

Views

Total Views
21,190
Views on SlideShare
19,785
Embed Views
1,405

Actions

Likes
0
Downloads
66
Comments
0

4 Embeds 1,405

http://puppetlabs.com 1401
https://www.google.co.uk 2
http://analyst.test.ciradar.com 1
http://richards-mbp.corp.puppetlabs.net 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • We don’t rely upon OS package management dependencies; these should be explicitly listed in Puppet manifests.Nothing precludes installing RPM containing Puppet config onto a puppet master; use “Environments” and yum --installrootModules are designed for transparency, simplicity: “4AM-proofing”Composition usually trumps inheritanceTim Bell and the CERN folks talk of “Pets” and “Cattle”You can only shoot a system in the head if you can create another at will“Fried” or “Baked”? YES. Even for systems which we launch from snapshots, the system has to come from version control.

Puppet for Production in WebEx - PuppetConf 2013 Puppet for Production in WebEx - PuppetConf 2013 Presentation Transcript

  • Cisco Confidential© 2011 Cisco and/or its affiliates. All rights reserved. 1 Puppet at Cisco CCATG Aug 23, 2013 Reinhardt Quelle, Cloud Services Architect
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2 Millions of Meetings for 10s of Millions of Users totaling Billions of Minutes each month 7x24x365 Cisco Social WebEx Connect
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3 U S U K Indi a Australi a China Hong Kong Amsterdam Japa n ~ 7K Hosts ~ 8 Data Centers > 12 iPOPs Private Backbone
  • Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 4Cisco Confidential 4© 2011 Cisco and/or its affiliates. All rights reserved.
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9 Files Packages = Users Services … Etc.
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10 Manifests - nodes.pp - site.pp Classes, Modules =
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13 BaseOS_Hardening v1.1 ElasticSearch v0.20.6 JRE v1.7.0_25 BaseOS_Hardening v1.1 ElasticSearch v0.90.2-1 JRE v1.7.0_25 • Some systems can simply be knocked over the head and recreated with fresh versions • Others – notably most database servers – cannot; updates are performed in-place • “Big Bang” upgrades don’t often happen; we step methodically through groups of machines
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15 DC1 DC2 Multiple DC Pairs Multiple Clusters of each Service type • By Customer Class • By Lifecycle Stage • By Special Needs
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16 By DC Or by Node v1 v1 v2 v2 v1 v1 v2v2
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17 “Blueprints” or “Models” • JSON/YAML • TOSCA • CMDB * Orchestration • Fabric • SLiM • Mcollective
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 18 Puppet Master Manifests & Modules DC1 DC2
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19 Puppet Master Manifests & Modules DC1 DC2 Guess when you’ll need to push infrastructure changes the most!
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20 Puppet Master Manifests & Modules DC1 DC2 Puppet Master Manifests & Modules
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21 Puppet Master Manifests & Modules DC1 DC2 Puppet Master Manifests & Modules
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22 WebEx Meetings WebEx Connect
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23 puppet apply -–modulepath=/opt/puppet_local --execute “include servertype::front-end” Manifests & Modules copy [/etc/puppet/*] to each node
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24 Manifests & Modules .rpm or .deb yum install app_pp_v1 && puppet apply …‟ private package repository
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 25 ssh node81 „yum install app_pp_v1 && puppet apply …‟ fab dfw-frontends pp_apply:latest‟ Fabric Mcollective Salt Ansible
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 26 • Application Stacks/Deployment are NOT Homogenous • The “right” solution for one stack not always right for another • Share as much as possible, but don’t force it • Tightly coupled systems are often rigid, brittle • Solving big, general problems is hard; small bites are easily digested “A foolish consistency is the hobgoblin of little minds” – Emerson
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27 • Every artifact (module, manifest, Hiera file) is checked into version control • Versions are packaged and released and should go through same promotion process as application code • All good coding practices apply Modular Well defined interfaces Tested Shared
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 28 • Modules are the atomic packages of configuration • “Profiles” bundle modules into commonly used sets for ease of consumption: BaseOS JavaApp Tomcat App • A given machine has exactly one “ServerType” • Inspired by Chef’s “roles”, and similar to Craig Dunn’s Role/Profile/Modules • At the code level, these are actually all just modules
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 29 • Build loosely coupled modules that can work together if installed together, but that can stand on own, too • Example: standard monit config includes /etc/monit/conf.d/* Application that wants to be monitored just drops file in this location • logrotate, collectd, apache, nginx, etc all support class elasticsarch { … if $monit::include_dir != undef { validate_absolute_path($monit::include_dir) file { "${monit::include_dir}/${monit_config_file_name}": ensure => present, content => template("elasticsearch/${monit_config_file_name}.erb"), notify => Service['monit'], } } }
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 30 • The singleton ServerType defines which profiles and modules are included; structure and order • Puppet Librarian and its Puppetfile describe which version of a module is used, and where it comes from • After Puppet Librarian has run and downloaded all required assets, FPM is called upon to build the package • Dev cycle includes doing local builds/tests against Vagrant in both develpers personal machines as well as the Jenkin’s build farm • Upon successful build, packages are uploaded to repositories
  • © 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 31 • Dependencies in Puppet, not RPM/DEB • Packaging works for deploying to Puppetmasters, too. • Modules are designed for transparency, simplicity: “4AM-proofing” • Composition usually trumps inheritance • Tim Bell and the CERN folks talk of “Pets” and “Cattle” You can only shoot a system in the head if you can create another at will • “Fried” or “Baked”? YES.
  • Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 36Cisco Confidential© 2011 Cisco and/or its affiliates. All rights reserved. 36