The Kitchen Cloud How To: Automating Joyent SmartMachines with Chef
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

The Kitchen Cloud How To: Automating Joyent SmartMachines with Chef

on

  • 2,924 views

Learning a new OS can be intimidating, especially one with less support in terms of open source Chef cookbooks. At Wanelo we’ve found the rewards of using Chef with Joyent’s SmartOS to be well ...

Learning a new OS can be intimidating, especially one with less support in terms of open source Chef cookbooks. At Wanelo we’ve found the rewards of using Chef with Joyent’s SmartOS to be well worth the effort.

SmartOS is an open source fork of Illumos (think Solaris) that runs in the Joyent Public Cloud. Over the last year we’ve grown to love SmartOS as a deployment environment, and with the help of Chef have grown Wanelo’s infrastructure more than ten times in six months to meet the demands our exponential user growth. In the next year, we expect to grow our infrastructure by another factor of ten. On another public cloud, our business growth would have required a significantly larger infrastructure at every step.

In this session I’ll explain why we appreciate SmartOS so much and how you can get started. What’s the terminology? What plugins do you need, and how do you use them? What providers should you learn and where can you find them? I’ll provide bootstrap scripts, basic roles and cookbooks on Github to get people provisioning and using SmartMachines immediately. For larger infrastructures, I’ll walk through some of the dependencies that have made our lives easier, and explain why.

By the end, you should have the code at your fingertips to deploy a Ruby or Rails application to the Joyent Public Cloud, with all of the dependent services up and running.

Statistics

Views

Total Views
2,924
Views on SlideShare
2,449
Embed Views
475

Actions

Likes
0
Downloads
30
Comments
0

3 Embeds 475

http://www.opscode.com 287
http://www.getchef.com 169
https://www.getchef.com 19

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

The Kitchen Cloud How To: Automating Joyent SmartMachines with Chef Presentation Transcript

  • 1. Proprietary andConfidentialAutomating JoyentSmartMachines with ChefChef on SmartOSEric Saxby@sax @ecdysone @sax
  • 2. Who am I?Proprietary andConfidential■ Application developeroperational experience with manytechnologies, project by project■ BSD/AIX/UbuntuSolaris in 2002, but I was very muchout of my element■ Switched to DevOps-y team 18 months agoMultiple back end services for a large e-commerce site,transitioning to SmartOS■ Now I’m at Wanelo
  • 3. From a certain point of view...Proprietary andConfidential
  • 4. What is Wanelo?Proprietary andConfidential■ Wanelo (“Wah-nee-lo” from Want, NeedLove) is a global platform for shopping.
  • 5. Proprietary andConfidentialMarketing-free shopping across 100s ofthousands of unique stores
  • 6. Proprietary andConfidentialPersonal feed of products from anystore on the internet
  • 7. Technology overviewProprietary andConfidential■ MRI Ruby 1.9.3 & Rails 3.2■ PostgreSQL 9.2.4, Solr 3.6■ Joyent Cloud, SmartOSZFS, ARC, raw IO performance, SmartOS, CPU bursting, dTrace■ Circonus, Chef + OpscodeMonitoring, graphing, alerting, automation■ Amazon S3 + Fastly CDN■ NewRelic, statsd, Graphite, nagios
  • 8. What’s SmartOS?Proprietary andConfidential■ Illumos branch optimized for cloudcomputing■ Developed by Joyent for their publiccloud
  • 9. What’s Illumos?Proprietary andConfidential■ It’s what OpenSolaris became after Oraclekilled the project■ Umbrella for various distributions, eachcommitted to pushing their improvementsupstream■ http://wiki.illumos.org/display/illumos/About+illumos
  • 10. What does SmartOS lookProprietary andConfidential■ Compute Node — physical server■ Global Zone — host OS (SmartOS)■ Non-Global Zone — like a virtual machine, withnative system calls (no fake hardware layer)■ Very secure■ Can run KVM for guest OS (Ubuntu, Centos)
  • 11. How is it deployed?Proprietary andConfidential■ Can manage from global zone (imgadm,zoneadm)■ Tools provide APIs■ Smart Data Center (Joyent’s tools, can be licensed)■ Project FIFO (SDC API in free package)■ Joyent Public Cloud■ Many compute nodes working in a cluster,PXE booted from a head node
  • 12. ■ Service Management Facility (SMF)If init.d and monit and god were one thing, andactually awesomeWhy should I care?Proprietary andConfidential■ Visibility toolsdtrace, kstat, snoop, truss■ ZFSFile system built for speed and data integrity■ Application LatencyZones are OS virtualization, so fasterProcesses are scheduled in global zone kernel,not in a hardware virtualization layer
  • 13. ■ # cores, RAM required =~# processesLower latency == less costProprietary andConfidential■ # processes required =~requests/second of site■ Requests/second of single process =~request latency$$$
  • 14. On to Chef!Proprietary andConfidential
  • 15. TerminologyProprietary andConfidential■ Image / Dataset — OS at a particular version,snapshotted at base state■ Flavor / Package— RAM, CPU shares■ API URL — Each data center has its own URL■ Server ID / Zonename — Each zone gets aUUID
  • 16. knife-joyentProprietary andConfidential
  • 17. Installation/ConfigurationProprietary andConfidential■ Update knife.rb■ Add to Gemfileknife[:joyent_username] = saxknife[:joyent_keyname] = EricSaxbyknife[:joyent_keyfile] = "#{ENV[HOME]}/.ssh/id_rsa"knife[:joyent_api_url] = https://us-sw-1.api.joyentcloud.com/■ Add first public key in cloud APIhttps://my.joyentcloud.comgem knife-joyent
  • 18. Managing keysProprietary andConfidential■ No role based access, but at least you canmake each user upload their own keyknife joyent key add -f ~/.ssh/id_rsa -k KeyNameknife joyent key delete KeyName■ Passphrase protected keys are annoyingEach API request includes data signed with the privatekey. Ruby does not have a good way of signing privatekeys with ssh-agent.
  • 19. Creating servers!Proprietary andConfidential■ See what images are availableknife joyent image listcf7e2f40-9276-11e2-af9a-0bad2233fb0b base64 1.9.1 smartosf4bc70ca-5e2c-11e1-8380-fb28785857cb smartosplus64 3.1.0 smartosda144ada-a558-11e2-8762-538b60994628 ubuntu-12.04 2.4.1 linux■ base / base64 — minimal install, you add whatyou need■ smartosplus — many more things pre-installed, but can get in the way13328c9a-9173-11e2-a9a5-2ff43d306c21 ws2008ent-r2-sp1 2.0.2 windows
  • 20. Creating servers!Proprietary andConfidential■ See what flavors are availableknife joyent flavor listName RAM Disk SwapExtra Small 512 MB 0 GB 15 GB 1 GBSmall 1GB 1 GB 30 GB 2 GBMedium 2GB 2 GB 60 GB 4 GBMedium 4GB 4 GB 120 GB 8 GBLarge 8GB 8 GB 240 GB 16 GBLarge 16GB 16 GB 480 GB 32 GB■ Custom networking can be done in a customflavor (ie public or private VLAN, routes)
  • 21. Creating servers already!Proprietary andConfidentialknife joyent server create--image cf7e2f40-9276-11e2-af9a-0bad2233fb0b--flavor Medium 2GB-N server.domain.com-E environment-d distro-r run_list■ No Omnibus, so you have to provide your owndistro bootstrap templatehttps://gist.github.com/sax/5457464
  • 22. knife joyent server listSee whats there...Proprietary andConfidentiala597a3a7-3fdf-481f-af08-e7c1e0ae7dca admin.prod running smartmachinesdc:sdc:base64:1.8.1 8.19.1.1 10.100.1.1 8 GB 240 GB5c066e6e-8af2-4d4f-a81e-c8e2691ae8a0 demo.dev running smartmachinesdc:sdc:base64:1.8.1 10.12.1.1 165.225.1.1 8 GB 240 GBb3370d52-3bed-462e-857a-e17eba15ab06 app010.c1.prod running smartmachinesdc:sdc:base64:1.8.1 10.100.1.2 165.225.1.2 8 GB 240 GB■ ID / zonename■ Name■ Run state■ Type■ Image■ IP addresses■ RAM■ Disk
  • 23. Other managementProprietary andConfidentialknife joyent server delete <server_id>knife joyent server start <server_id>knife joyent server stop <server_id>knife joyent server reboot <server_id>knife joyent server resize <server_id> -f <flavor>knife joyent snapshot create <server_id> <snapshot_name>■ Snapshots are full ZFS snapshotsCopy-on-write snapshot of local file system.Each snapshot is locally mounted in zone at/checkpoints
  • 24. So now you have asmartmachine...Proprietary andConfidential
  • 25. Whats different?Proprietary andConfidential■ Things you expect in /usr/local are in /opt/local■ For historical reasons■ If youre used to Linux, this can be annoying■ Joyent is working on a more Linux friendly image■ For now, add /opt/local/bin to PATH■ Many configs are in /opt/local/etc instead of /etc■ Some utilities are different■ This is not the grep youre looking for....■ Symlink your "correct" version into /opt/local/bin■ Add /opt/local/lib to CFLAGS and LDFLAGS
  • 26. Caveats?Proprietary andConfidential■ Zones inside of zones inside of...■ Vagrant does not currently work with SmartOS■ VirtualBox only works in Bridged network mode■ Local integration tests do not work
  • 27. Where are all the things?Proprietary andConfidential■ Services■ svcs -a■ svcadm < enable | disable | clear > service■ Packages■ pkgin search packagename■ pkgin -y install packagename
  • 28. Public vs. Private IPProprietary andConfidential■ ipaddr_extensions gem■ Adds privateaddress attribute to ohai■ Useful to add this to bootstrap■ Smartmachines may have a public IP and aprivate IP■ Recipes can be configured to use ipaddress orprivateaddress
  • 29. System preparationProprietary andConfidential■ smartos cookbook■ https://github.com/modcloth-cookbooks/smartos■ fixes chef providers■ smartmachine_functions■ links nicer utils into /opt/local/bin■ https://github.com/higanworks-cookbooks/smartmachine_functions■ fixes chef providers■ provides access to Joyent metadata APIor
  • 30. Useful LWRPsProprietary andConfidential
  • 31. SMFProprietary andConfidential■ https://github.com/modcloth-cookbooks/smf■ Chef knows how to use SMF, not how to configure it■ Uses nokogiri, which requires libxsltsmf postgres douser postgresgroup postgresproject postgresstart_command postgres-service.sh startstop_command postgres-service.sh stopworking_directory /var/pgsql/dataenvironment PATH => /opt/postgres/binend
  • 32. SMF (cntd)Proprietary andConfidentialsmf postgres douser postgresgroup postgresproject postgresstart_command postgres-service.sh startstop_command postgres-service.sh stopstop_timeout 120restart_command postgres-service.sh restartrefresh_command postgres-service.sh reloadworking_directory /var/pgsql/dataenvironment PATH => /opt/postgres/binendservice postgres dosupports :status => true,:restart => true, :reload => trueend
  • 33. Resource Control /Proprietary andConfidential■ https://github.com/wanelo-chef/resource-control■ configure max file descriptors, shared memory, etc■ Bunch up master/worker processes to view inprstat -Jresource_control_project "postgres" docomment "PostgreSQL 9.2"users "postgres"project_limits "max-shm-memory" => 12000000,"max-lwps" => 6process_limits "max-file-descriptor" => {"value" => 32768, "deny" => true}action :createend
  • 34. Role Based Access ControlProprietary andConfidential■ https://github.com/modcloth-cookbooks/rbac■ Allows delegation of authority without sudo■ Implementation currently too simple, only useful for SMFdelegationrbac solr douser waneloaction :add_management_permissionsend
  • 35. Contributing to cookbooksProprietary andConfidential■ ~95% just require SMF, correct package names■ ~5% of those need a special init script■ The rest usually require custom compile`postgres -D /path/to/data` not granular enough`pg_ctl -D /path/to/data < start | stop | reload | refresh >`--with-libraries=/opt/local/lib--with-includes=/opt/local/includeLDFLAGS=-R/opt/local/lib -L/opt/local/lib
  • 36. Comments? Questions? Findme.https://github.com/wanelohttps://github.com/wanelo-chefhttps://github.com/wanelo-chef/smartos-chef-repoProprietary andConfidential@sax @ecdysone @sax