Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to Puppetize Google Cloud Platform - PuppetConf 2014

2,261 views

Published on

How to Puppetize Google Cloud Platform - Katharina Probst, Google, Matt Bookman, Google & Ryan Coleman, Puppet Labs

Published in: Technology

How to Puppetize Google Cloud Platform - PuppetConf 2014

  1. 1. HOWTO:Puppetizing Google Cloud Platform Katharina Probst, Technical Lead & Manager (Google) Ryan Coleman, Product Manager (Puppet Labs)
  2. 2. Agenda: September 23rd, 2014 Whirlwind tour of Google Cloud Platform Diving into Google Compute Engine How Puppet Labs uses Google Cloud Platform Live Demo 1 2 3 4
  3. 3. Agenda: September 23rd, 2014 Whirlwind tour of Google Cloud Platform Diving into Google Compute Engine How Puppet Labs uses Google Cloud Platform Live Demo 1 2 3 4
  4. 4. For the past 15 years, Google has been building out the world’s fastest, most powerful, highest quality cloud infrastructure on the planet. Images by Connie Zhou Why Google Cloud Platform?
  5. 5. Building Products that Scale Google Maps Gmail Google Drive YouTube
  6. 6. A Network that Spans the Globe
  7. 7. Innovating Software & Driving Technology Forward MapReduce Dremel Spanner Big Table Colossus GFS Compute Engine 2002 2004 2006 2008 2010 2012 2013
  8. 8. “[Google's] ability to build, organize, and operate a huge network of servers and fiber-optic cables with an efficiency and speed that rocks physics on its heels. This is what makes Google Google: its physical network, its thousands of fiber miles, and those many thousands of servers that, in aggregate, add up to the mother of all clouds.” - Wired Images by Connie Zhou
  9. 9. Storage Cloud Storage Cloud SQL Cloud Datastore Compute Compute App Engine Engine App Services BigQuery Cloud Endpoints Google Cloud Platform Cloud DNS
  10. 10. Compute Compute App Engine Engine Storage Cloud Storage Cloud SQL Cloud Datastore Power Computation App Services BigQuery Cloud Endpoints Cloud DNS
  11. 11. Storing What You Want, However You Want Storage Cloud Storage Cloud SQL Cloud Datastore Compute Compute App Engine Engine App Services BigQuery Cloud Endpoints Cloud DNS
  12. 12. App Services BigQuery Cloud Endpoints Building Robust, Intelligent Systems Storage Cloud Storage Cloud SQL Cloud Datastore Compute Compute App Engine Engine Cloud DNS
  13. 13. Agenda: September 23rd, 2014 Whirlwind tour of Google Cloud Platform Diving into Google Compute Engine How Puppet Labs uses Google Cloud Platform Live Demo 1 2 3 4
  14. 14. Google Compute Engine • IaaS: VMs, Network, Storage • Google DNA (speed, scale, reliable, secure) • Fast Provisioning, Consistent Performance • Enterprise Ready • 24x7 Support • 99.95% monthly SLA • ISO 27001, SSAE-16 SOC 1,2,3 • Accessible Through • Web @ https://cloud.google.com/console • gcloud compute command-line utility • REST API • Partners (Commercial and FOSS)
  15. 15. Launching 50 virtual machines
  16. 16. Block Storage Persistent Disks
  17. 17. Persistent Disk 10 TB • SSD and "Standard" • Data and root partitions • Billed only by capacity (GB/month) • Performance caps scale linearly with size • Volume striping is automatic • Differential snapshots • Create new PDs based on snapshots
  18. 18. PD Use-cases Root Stateful root volume Root RW Data User managed data volume RO Data Instant distribution of static content
  19. 19. Global Snapshot and Restore vol1 vol2 vol3 GCS vol1. t2 vol2. t2 vol3. t3 t1 t1 t1 t1 t1 t1 t1 t1 t1 t1 t1 t1 t1 t2 t2 t2 t2 t3 t3 t3 ● Point in time snapshot to Google Cloud Storage (GCS) ● Differential snapshots ● GCS global replication! ● Restore from snapshot anywhere in the world us-central1-a europe-west1-a
  20. 20. Virtual Machines
  21. 21. Machine Types Machine Type Virtual Cores standard • For workloads with balanced CPU and memory highmem • For workloads with higher memory requirements highcpu • For workloads with higher CPU requirements Shared Core Instances • For inexpensive prototyping and staging workloads (g1-small, f1-micro) Memory (GB) n1-standard-1 1 3.75 n1-standard-2 2 7.50 n1-standard-4 4 15.00 n1-standard-8 8 30.00 n1-standard-16 16 60.00 n1-highmem-2 2 13.00 n1-highmem-4 4 26.00 n1-highmem-8 8 52.00 n1-highmem-16 16 104.00 n1-highcpu-2 2 1.80 n1-highcpu-4 4 3.60 n1-highcpu-8 8 7.20 n1-highcpu-16 16 14.40
  22. 22. Images + Custom Kernel Support (debian backports works great for docker)
  23. 23. Sub-hour Billing + Sustained Use Discount 1 minute granularity, 10 minute minimum $ (30%) Full billing cycle Sustained Use Discount ● >25% use, price reduced ● Up to 30% reduction ● No commitment / contract
  24. 24. Enhanced Reliability us-central1-a scheduled maintenance event Live Migration • No downtime during scheduled datacenter maintenance events Automatic Restart • Instances automatically restarted if subjected to system events such as hardware failure
  25. 25. Networking
  26. 26. Images by Connie Zhou Networking • Google's Network • Projects are isolated private networks • TCP, UDP, ICMP only • Multiple private network groups and firewalls • Tags and address ranges • Addresses • public: static or ephemeral • private: ephemeral with DNS • Routes, gateways, VPNs, and IP Forwarding
  27. 27. Region: us-central1 Target Pool (tp-a) Load-balancer • Region based • Session affinity, hashing options on tuples, • Protocol, src ip:port, dst ip:port • Consists of: us-central1-a www0 www1 tcp:80 ➔ tp-a • Target pool: collection of instances • Forwarding rules: protocol:port, pool • HTTP health check: optional us-central1-b www2 www3 Forwarding Rules tcp:443 ➔ tp-a Internet http://googlecloudplatform.blogspot.com/2013/11/compute-engine-load-balancing-hits-1-million-requests-per-second.html
  28. 28. ...and
  29. 29. Noteworthy • Regions • Central United States, Europe, Asia (new!) • Metadata • Startup scripts • SSH Keys • Tags (instance/network) • OAuth2 and Scopes • Access other Google Cloud Platform services • Ecosystem is growing • Partners: RightScale, Scalr, New Relic, MongoLab, MapR, and many more... • Open Source: Chef, Puppet, Salt, Ansible, Vagrant, Docker, CoreOS, fog, libcloud
  30. 30. Agenda: September 23rd, 2014 Whirlwind tour of Google Cloud Platform Diving into Google Compute Engine How Puppet Labs uses Google Cloud Platform Live Demo 1 2 3 4
  31. 31. Puppet DSL for Google Compute Engine https://forge.puppetlabs.com/puppetlabs/gce_compute
  32. 32. The Puppet Forge a real world (meta) example
  33. 33. Abstraction Layers for Infrastructure as Code Infrastructure Layer Role & Profile Layer Component Layer Primitive Layer puppetlabs-gce_compute custom module puppetlabs-postgresql core puppet type/provider
  34. 34. Agenda: September 23rd, 2014 Whirlwind tour of Google Cloud Platform Diving into Google Compute Engine How Puppet Labs uses Google Cloud Platform Live Demo 1 2 3 4
  35. 35. Demo Architecture 1 3 2 4 us-central1-a M us-central1-b allow tcp:80 Load-balancer Target-pool ● [M]aster resides in GCE ● Uses gce_compute manifest to: ○ Spin up 4 VMs ○ Networking (firewall, LB) ● New nodes register with Master ● Install site.pp (apache+php+mysql) ● Custom index.php, connects to Cloud SQL database ● Let's try it out! Cloud SQL
  36. 36. Bring up the Compute Engine environment puppet apply /etc/puppetlabs/puppet/manifests/gce_up.pp # puppet apply /etc/puppetlabs/puppet/manifests/gce_up.pp # ... # cat !$ 1 $prefix = 'pc' 2 $zonea = 'us-central1-a' 3 $zoneb = 'us-central1-b' 4 $region = 'us-central1' 5 6 gce_auth { "graphite-demos": } 7
  37. 37. Bring up the Compute Engine environment (cont) puppet apply /etc/puppetlabs/puppet/manifests/gce_up.pp 8 gce_firewall { "$prefix-firewall": 9 ensure => present, 10 description => 'Allow HTTP', 11 network => 'default', 12 allowed => 'tcp:80', 13 allowed_ip_sources => '0.0.0.0/0', 14 } 15 16 gce_httphealthcheck { "$prefix-http": 17 ensure => present, 18 require => Gce_instance["$prefix-agent-1", "$prefix-agent-2"...], 19 description => 'basic http health check', 20 } 21 ...
  38. 38. Bring up the Compute Engine environment (cont) puppet apply /etc/puppetlabs/puppet/manifests/gce_up.pp 46 gce_disk { [ "$prefix-agent-2", "$prefix-agent-4" ]: 47 ensure => present, 48 size_gb => 10, 49 zone => "$zoneb", 50 source_image => 'debian-7', 51 } 52 ...
  39. 39. Bring up the Compute Engine environment (cont) puppet apply /etc/puppetlabs/puppet/manifests/gce_up.pp 73 define create_instance_zoneb { 74 gce_instance { "${title}": 75 ensure => present, 76 zone => "$zoneb", 77 disks => "${title},boot", 78 require => Gce_disk["${title}"], 79 puppet_service => present, 80 startupscript => 'pe-simplified-agent.sh', 81 metadata => { 82 'pe_role' => 'agent', 83 'pe_master' => "$hostname", 84 'pe_version' => '3.3.2', 85 } 86 } 87 } 88 89 create_instance_zoneb { ["$prefix-agent-2", "$prefix-agent-4"]: }
  40. 40. The Demo App puppet apply /etc/puppetlabs/puppet/manifests/gce_up.pp 1 <html> 2 <title>PuppetConf 2014</title> 3 <body> 4 <h1>PuppetConf 2014: Puppetizing Google Cloud Platform</h1> 5 ... 6 <!-- A couple 'facter' facts --> 7 <ul>This app is being served from host: <b><%= @ hostname %> [<%= @fqdn %>, 7 <%= @ipaddress %>]</b></ul> 8 <p>Please enter a Nick and Message and watch it appear below:</p> 9 ... 10 <?php 11 $db = mysqli_connect("173.194.254.93", "puppetconf", " my_password", "puppetconf"); 12 ... 13 ?>
  41. 41. cloud.google.com • Try out Google Cloud Platform with a $500 Credit • Visit https://cloud.google.com/starterpack and use promo code puppet14-con • Want more? • Try the demo on your own at https://bit.ly/puppet-gce-demo • or see http://googlecloudplatform.github.io/compute-video-demo-puppet • We welcome your help and feedback to make Puppet + GCE even better! • Find us at gce-discussion@googlegroups.com or https://github.com/puppetlabs/puppetlabs-gce_compute
  42. 42. FIN

×