Functional Hostnames and Why they are Bad
Upcoming SlideShare
Loading in...5
×
 

Functional Hostnames and Why they are Bad

on

  • 8,807 views

"Functional Hostnames and Why they are Bad" by Andrew Fong and Gary Josack of Dropbox at Puppet Camp SF 2013. Find a Puppet Camp near you and learn more about configuration management: ...

"Functional Hostnames and Why they are Bad" by Andrew Fong and Gary Josack of Dropbox at Puppet Camp SF 2013. Find a Puppet Camp near you and learn more about configuration management: puppetlabs.com/community/puppet-camp/

Statistics

Views

Total Views
8,807
Views on SlideShare
8,798
Embed Views
9

Actions

Likes
16
Downloads
45
Comments
0

2 Embeds 9

https://twitter.com 6
https://puppetlabs.com 3

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Functional Hostnames and Why they are Bad Functional Hostnames and Why they are Bad Presentation Transcript

  • Puppet Camp April 9th 2013
  • What’s in a name?Andrew Fong and Gary Josack fong@dropbox.com gary@dropbox.com
  • About Dropbox•  Thousands of instances/servers•  Mostly Python Stack•  EC2 and Large Self Datacenters•  Over a billion file syncs per day•  Thousands of MySQL Shards•  4 SREs and 1 DBA
  • A story of a startup... Hostapuppet.com
  • Chapter One•  1 or 2 teams•  Couple of hosts•  Webserver and a database•  Maybe one ops guy
  • What Ops People Like•  Simplicity•  Repeatability•  Assurances that things remain consistent•  Puppet / Configuration management
  • Config Managementnode www1.example.com { include common include apache include squid }node db1.example.com { include common include mysql }
  • My First Puppet ConfigNode ‘mickey.hostapuppet.com’ { include common include webserver include sudoers}Node ‘donald.hostapuppet.com’{ include common include db include sudoers}
  • Sudoers Module…file { "/etc/sudoers": owner => root, group => root, mode => "440", source => "puppet:///modules/sudo/sudoers", }…
  • Sudoers File itselfHost_Alias DONALD=donald.hostapuppet.comHost_Alias MICKEY=mickey.hostapuppet.comdb_guy DONALD=(all) NOPASSWD: ALLops_guy MICKEY=(all) NOPASSWD: ALL
  • Chapter Two: A growing service•  A few teams•  2 or 3 services•  multiple types of hosts –  Web –  API –  DB
  • Hostnames•  sjc-web[1-N]•  sjc-db[1-N]•  sjc-api[1-N]
  • Host Regex$hosttype = inline_template(<%=hostname.sub(/w+-([a-z]+)d*/){$1} %>’)
  • Hosttypes $hosttype = inline_template(<%= hostname.sub(/w+-([a-z]+)d*/){$1} %>’)if $hosttype == ‘web’ { if $hosttype == ‘api’ { include sudoers include sudoers include web include api }}If $hosttype == ‘db’ { include sudoers include db}
  • Back to sudoers…file { "/etc/sudoers": owner => root, group => root, mode => "440", source => "puppet:///modules/sudo/sudoers", }…
  • Sudoers File itselfHost_Alias WEB=sjc-web*Host_Alias DB=sjc-db*Host_Alias API=sjc-api*database_guy DB=(all) NOPASSWD: ALLops_guy WEB=(all) NOPASSWD: ALLapi_team API=(all) NOPASSWD: ALL
  • Hypergrowth Users (millions)120100 80 60 40 20 0 0 2 3 5 Users (millions)
  • Chapter 3: An Expanding Infrastructure•  Lots of new hires!•  A bunch more developers•  Some PMs•  Some Designers
  • All Kinds Of Problems…•  Boxes of same hardware class running different services•  Boxes serving more than one role (remember sudoers?)•  Deploying or moving hosts quickly
  • Renaming a host•  Update dns•  Update dhcpd.conf•  Push both•  Update puppet configs•  Update code
  • OMG I JUST RENAME HOSTS!
  • Sudoers File From Chapter Two…Host_Alias WEB= sjc-web*Host_Alias API=sjc-api*Host_Alias DB=sjc-db*database_guy DB=(all) NOPASSWD: ALLops_guy WEB=(all) NOPASSWD: ALLapi_team API(all) NOPASSWD: ALL
  • Sudoers File in Chapter 3Host_Alias WEB= sjc-web*Host_Alias API=sjc-api*,sjc-web550,sjc-web551,sjc-web552,sjc-web553Host_Alias DB=sjc-db*database_guy DB=(all) NOPASSWD: ALLops_guy WEB=(all) NOPASSWD: ALLapi_team API(all) NOPASSWD: ALL
  • Dropbox•  We did all that.•  We’re still paying the taxes for doing that.•  But there is a light at the end of the tunnel…
  • ABSTRACT THE SERVICEFROM THE HOST!
  • So what does that mean?• Make hosts role agnostic• Do not require invasive changes• Simple interfaces
  • Making hosts role agnostic• Positional• Serial Numbers• Anything that doesnt change
  • The Dropbox Plan• Positional names• Custom Machine Database• External Node Classifier• Transitioning Puppet configs• Naming service(s) for convenient names
  • Service/Machine Management Database• Universal Source of Truth• Manage roles / attributes• Generated configs - Gmond, Nagios, etc
  • What exactly is the ENC• External Node Classifier• Inject variables (and other) from externalprocess• YAML Output
  • Part 2: External Node Encoders
  • Sudoers++• Move from monolithic to modular• Includes! (Weird caveats)• Just use ALL for Host_Lists
  • Sudoers at Dropbox
  • Part 3: Helper Functions
  • Sudoers with tags
  • Sudoers with tags
  • Provisioning• Preload MDB, DNS, DHCPD, etc. - Set it and forget it• Have spares ready for any roles• Assigning a role is one command• No more renames!
  • Dynamic Naming w/ PowerDNS
  • Dynamic Naming w/ PowerDNS
  • Zookeeper• ZKNS included with the Vitess project• ZK is in use at various different companies(YouTube, Twitter, AirBnB)
  • Q&A FAQ #1: Are you hiring?λ  - Yes! Come talk to us. :)