SV Ruby on Rails Meetup, profile picture
Uploaded bySV Ruby on Rails Meetup
KEY, PPTX571 views

How I Learned to Stop Worrying and Love the Cloud - Wesley Beary, Engine Yard

The document discusses Fog, an API-driven cloud services library that allows users to manage cloud resources across various providers like AWS and Rackspace. It emphasizes its on-demand, flexible, and repeatable nature, enabling quick resource allocation and management, along with detailed code examples for setup and operations. The roadmap highlights ongoing improvements in API consistency, error handling, and documentation to enhance user experience.

Related topics:
Ruby on Rails Overview•

Embed presentation

Download as KEY, PPTX
1 / 123
2 / 123
3 / 123
4 / 123
5 / 123
6 / 123
7 / 123
8 / 123
9 / 123
10 / 123
11 / 123
12 / 123
13 / 123
14 / 123
15 / 123
16 / 123
17 / 123
18 / 123
19 / 123
20 / 123
21 / 123
22 / 123
23 / 123
24 / 123
25 / 123
26 / 123
27 / 123
28 / 123
29 / 123
30 / 123
31 / 123
32 / 123
33 / 123
34 / 123
35 / 123
36 / 123
37 / 123
38 / 123
39 / 123
40 / 123
41 / 123
42 / 123
43 / 123
44 / 123
45 / 123
46 / 123
47 / 123
48 / 123
49 / 123
50 / 123
51 / 123
52 / 123
53 / 123
54 / 123
55 / 123
56 / 123
57 / 123
58 / 123
59 / 123
60 / 123
61 / 123
62 / 123
63 / 123
64 / 123
65 / 123
66 / 123
67 / 123
68 / 123
69 / 123
70 / 123
71 / 123
72 / 123
73 / 123
74 / 123
75 / 123
76 / 123
77 / 123
78 / 123
79 / 123
80 / 123
81 / 123
82 / 123
83 / 123
84 / 123
85 / 123
86 / 123
87 / 123
88 / 123
89 / 123
90 / 123
91 / 123
92 / 123
93 / 123
94 / 123
95 / 123
96 / 123
97 / 123
98 / 123
99 / 123
100 / 123
101 / 123
102 / 123
103 / 123
104 / 123
105 / 123
106 / 123
107 / 123
108 / 123
109 / 123
110 / 123
111 / 123
112 / 123
113 / 123
114 / 123
115 / 123
116 / 123
117 / 123
118 / 123
119 / 123
120 / 123
121 / 123
122 / 123
123 / 123
OR: fog HOW I LEARNED TO STOP WORRYING AND LOVE THE CLOUD
geemus (Wesley Beary) web: github.com/geemus twitter: @geemus
employer and sponsor web: engineyard.com twitter: @engineyard
API driven on demand services CLOUD core: compute, dns, storage also: kvs, load balance, ...
What?
What? on demand - pay for only what you actually use
What? on demand - pay for only what you actually use flexible - add and remove resources in minutes (instead of weeks)
What? on demand - pay for only what you actually use flexible - add and remove resources in minutes (instead of weeks) repeatable - code, test, repeat
What? on demand - pay for only what you actually use flexible - add and remove resources in minutes (instead of weeks) repeatable - code, test, repeat resilient - build better systems with transient resources
But, but...
But, but... option overload - which provider should I use
But, but... option overload - which provider should I use intimidating - each service requires separate expertise
But, but... option overload - which provider should I use intimidating - each service requires separate expertise tools - vastly different API’s built on varying levels of quality
Ruby cloud services web: github.com/geemus/fog twitter: @fog
Why?
Why? portable - AWS, Bluebox, Brightbox, Google, Rackspace, Slicehost, Terremark, ...
Why? portable - AWS, Bluebox, Brightbox, Google, Rackspace, Slicehost, Terremark, ... powerful - compute, dns, storage, collections, models, mocks, requests, ...
Why? portable - AWS, Bluebox, Brightbox, Google, Rackspace, Slicehost, Terremark, ... powerful - compute, dns, storage, collections, models, mocks, requests, ... established - 50k downloads, 860 followers, 100 forks, 40 contributors, me, ...
Why? portable - AWS, Bluebox, Brightbox, Google, Rackspace, Slicehost, Terremark, ... powerful - compute, dns, storage, collections, models, mocks, requests, ... established - 50k downloads, 860 followers, 100 forks, 40 contributors, me, ... Fog.mock! - faster, cheaper, simulated cloud behavior
Who?
Who? libraries - carrierwave, chef, deckard, gaff, gemcutter, ... products - DevStructure, Engine Yard, iSwifter, OpenFeint, RowFeeder, ...
What?
What? That’s great and all but I don’t have a use case...
What? That’s great and all but I don’t have a use case... uptime - because who wants a busted web site?
Setup geymus ~ ⌘ gem install fog or geymus ~ ⌘ sudo gem install fog
Get Connected 7 # setup a connection to the service 8 compute = Fog::Compute.new(credentials)
Get Connected 1 credentials = { 2   :provider           => 'Rackspace', 3   :rackspace_api_key  => RACKSPACE_API_KEY, 4   :rackspace_username => RACKSPACE_USERNAME 5 } 6 7 # setup a connection to the service 8 compute = Fog::Compute.new(credentials)
Bootstrap 7 # boot server and setup ssh keys 8 server = compute.servers.bootstrap(server_attributes)
Bootstrap 1 server_attributes = { 2   :image_id         => '49', 3   :private_key_path => PRIVATE_KEY_PATH, 4   :public_key_path  => PUBLIC_KEY_PATH 5 } 6 7 # boot server and setup ssh keys 8 server = compute.servers.bootstrap(server_attributes)
Servers?
Servers? 1 compute.servers # list servers, same as #all
Servers? 1 compute.servers # list servers, same as #all 2 3 compute.servers.get(1234567890) # server by id
Servers? 1 compute.servers # list servers, same as #all 2 3 compute.servers.get(1234567890) # server by id 4 5 compute.servers.reload # update to latest
Servers? 1 compute.servers # list servers, same as #all 2 3 compute.servers.get(1234567890) # server by id 4 5 compute.servers.reload # update to latest 6 7 compute.servers.new(attributes) # local model
Servers? 1 compute.servers # list servers, same as #all 2 3 compute.servers.get(1234567890) # server by id 4 5 compute.servers.reload # update to latest 6 7 compute.servers.new(attributes) # local model 8 9 compute.servers.create(attributes) # remote model
bootstrap?
bootstrap? convenience method for servers
bootstrap? convenience method for servers create - creates a server matching attributes wait_for { ready? } - waits until server responds ssh - sets up ssh keys and disables password authentication
ping
ping 1 # ping target 10 times 2 ssh_results = server.ssh("ping -c 10 #{target}")
ping 1 # ping target 10 times 2 ssh_results = server.ssh("ping -c 10 #{target}") 3 stdout = ssh_results.first.stdout
ping 1 # ping target 10 times 2 ssh_results = server.ssh("ping -c 10 #{target}") 3 stdout = ssh_results.first.stdout 4 5 # parse result, last line is summary 6 # round-trip min/avg/max/stddev = A.A/B.B/C.C/D.D ms
ping 1 # ping target 10 times 2 ssh_results = server.ssh("ping -c 10 #{target}") 3 stdout = ssh_results.first.stdout 4 5 # parse result, last line is summary 6 # round-trip min/avg/max/stddev = A.A/B.B/C.C/D.D ms 7 stats = stdout.split("/n").last.split(' ')[-2] 8 min, avg, max, stddev = stats.split('/')
ping 1 # ping target 10 times 2 ssh_results = server.ssh("ping -c 10 #{target}") 3 stdout = ssh_results.first.stdout 4 5 # parse result, last line is summary 6 # round-trip min/avg/max/stddev = A.A/B.B/C.C/D.D ms 7 stats = stdout.split("/n").last.split(' ')[-2] 8 min, avg, max, stddev = stats.split('/') NOTE: most complex code was string parsing!?!
cleanup
cleanup  1 # shutdown the server  2 server.destroy
cleanup  1 # shutdown the server  2 server.destroy  3  4 # return the data as a hash  5 {  6   :min    => min,  7   :avg    => avg,  8   :max    => max,  9   :stddev => stddev 10 }
geopinging v1  1 # specify a different provider  2 credentials = {  3   :provider => 'AWS',  4   :aws_access_key_id => AWS_ACCESS_KEY_ID,  5   :aws_secret_access_key => AWS_SECRET_ACCESS_KEY  6 }  7  8 server_attributes = {  9   :image_id         => 'ami-1a837773', 10   :private_key_path => PRIVATE_KEY_PATH, 11   :public_key_path  => PUBLIC_KEY_PATH, 12   :username         => 'ubuntu' 13 }
geopinging v2 1 # specify a different aws region 2 # ['ap-southeast-1', 'eu-west-1', 'us-west-1] 3 credentials.merge!({ 4   :region => 'eu-west-1' 5 })
geopinging v... portable - AWS, Bluebox, Brightbox, Rackspace, Slicehost, Terremark, ...
geopinging v... portable - AWS, Bluebox, Brightbox, Rackspace, Slicehost, Terremark, ... lather, rinse, repeat
How? That is awesome, but how did you know...
exploring geymus ~ ⌘ fog   To run as 'default', add the following to ~/.fog :default:   :aws_access_key_id:     INTENTIONALLY_LEFT_BLANK   :aws_secret_access_key: INTENTIONALLY_LEFT_BLANK   :public_key_path:       INTENTIONALLY_LEFT_BLANK   :private_key_path:      INTENTIONALLY_LEFT_BLANK   :rackspace_api_key:     INTENTIONALLY_LEFT_BLANK   :rackspace_username:    INTENTIONALLY_LEFT_BLANK ...
sign posts
sign posts geymus ~ ⌘ fog
sign posts geymus ~ ⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace
sign posts geymus ~ ⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers
sign posts geymus ~ ⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace]
sign posts geymus ~ ⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace] >> Rackspace.collections
sign posts geymus ~ ⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace] >> Rackspace.collections [:directories, :files, :flavors, :images, :servers]
sign posts geymus ~ ⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace] >> Rackspace.collections [:directories, :files, :flavors, :images, :servers] >> Rackspace[:compute]
sign posts geymus ~ ⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace] >> Rackspace.collections [:directories, :files, :flavors, :images, :servers] >> Rackspace[:compute] #<Fog::Rackspace::Compute ...>
sign posts geymus ~ ⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace] >> Rackspace.collections [:directories, :files, :flavors, :images, :servers] >> Rackspace[:compute] #<Fog::Rackspace::Compute ...> >> Rackspace[:compute].requests
sign posts geymus ~ ⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace] >> Rackspace.collections [:directories, :files, :flavors, :images, :servers] >> Rackspace[:compute] #<Fog::Rackspace::Compute ...> >> Rackspace[:compute].requests [:confirm_resized_server, ..., :update_server]
requests?
requests? >> Rackspace[:compute].list_servers
requests? >> Rackspace[:compute].list_servers #<Excon::Response:0x________
requests? >> Rackspace[:compute].list_servers #<Excon::Response:0x________ @body = { "servers" => [] },
requests? >> Rackspace[:compute].list_servers #<Excon::Response:0x________ @body = { "servers" => [] }, @headers = { "X-PURGE-KEY"=>"/______/servers", ..., "Connection"=>"keep-alive" },
requests? >> Rackspace[:compute].list_servers #<Excon::Response:0x________ @body = { "servers" => [] }, @headers = { "X-PURGE-KEY"=>"/______/servers", ..., "Connection"=>"keep-alive" }, @status=200>
sanity check
sanity check >> Rackspace.servers.select {|server| server.ready?}
sanity check >> Rackspace.servers.select {|server| server.ready?} <Fog::Rackspace::Compute::Servers filters={} [] >
sanity check >> Rackspace.servers.select {|server| server.ready?} <Fog::Rackspace::Compute::Servers filters={} [] > >> AWS.servers.select {|server| server.ready?}
sanity check >> Rackspace.servers.select {|server| server.ready?} <Fog::Rackspace::Compute::Servers filters={} [] > >> AWS.servers.select {|server| server.ready?} <Fog::AWS::Compute::Servers [] >
sanity check >> Rackspace.servers.select {|server| server.ready?} <Fog::Rackspace::Compute::Servers filters={} [] > >> AWS.servers.select {|server| server.ready?} <Fog::AWS::Compute::Servers [] > >> exit
finding images
finding images >> Rackspace.images.table([:id, :name])
finding images >> Rackspace.images.table([:id, :name]) +--------- +------------------------------------------+ | id | name | +--------- +------------------------------------------+ | 49 | Ubuntu 10.04 LTS (lucid) | +--------- +------------------------------------------+ ...
finding images >> Rackspace.images.table([:id, :name]) +--------- +------------------------------------------+ | id | name | +--------- +------------------------------------------+ | 49 | Ubuntu 10.04 LTS (lucid) | +--------- +------------------------------------------+ ... >> AWS.images # I use alestic.com listing
finding images >> Rackspace.images.table([:id, :name]) +--------- +------------------------------------------+ | id | name | +--------- +------------------------------------------+ | 49 | Ubuntu 10.04 LTS (lucid) | +--------- +------------------------------------------+ ... >> AWS.images # I use alestic.com listing ...
exploring...
exploring... It takes forever!
exploring... It takes forever! It’s so expensive!
exploring... It takes forever! It’s so expensive! A warm welcome for Fog.mock!
Mocks! geymus ~ ⌘ FOG_MOCK=true fog or require ‘fog’ Fog.mock!
simulation
simulation Most functions just work!
simulation Most functions just work! Unimplemented mocks? Errors keep you on track.
simulation Most functions just work! Unimplemented mocks? Errors keep you on track. Tests run against both, so it is either consistent or a bug.
Back to Business I have a bunch of data now what?
Back to Business I have a bunch of data now what? storage - aggregating cloud data
Get Connected 7 # setup a connection to the service 8 storage = Fog::Storage.new(credentials)
Get Connected 1 credentials = { 2   :provider => 'AWS', 3   :aws_access_key_id => AWS_ACCESS_KEY_ID, 4   :aws_secret_access_key => AWS_SECRET_ACCESS_KEY 5 } 6 7 # setup a connection to the service 8 storage = Fog::Storage.new(credentials)
directories 1 # create a directory 2 directory = storage.directories.create( 3   :key    => directory_name, 4   :public => true 5 )
files
files 1 # store the file 2 file = directory.files.create( 3   :body   => File.open(path), 4   :key    => name, 5   :public => true 6 )
files 1 # store the file 2 file = directory.files.create( 3   :body   => File.open(path), 4   :key    => name, 5   :public => true 6 ) 7 8 # return the public url for the file 9 file.public_url
geostorage  1 # specify a different provider  2 credentials = {  3   :provider           => 'Rackspace',  4   :rackspace_api_key  => RACKSPACE_API_KEY,  5   :rackspace_username => RACKSPACE_USERNAME  6 }
cleanup
cleanup geymus ~ ⌘ fog ...
cleanup geymus ~ ⌘ fog ... >> directory = AWS.directories.get(DIRECTORY_NAME) ...
cleanup geymus ~ ⌘ fog ... >> directory = AWS.directories.get(DIRECTORY_NAME) ... >> directory.files.each {|file| file.destroy} ...
cleanup geymus ~ ⌘ fog ... >> directory = AWS.directories.get(DIRECTORY_NAME) ... >> directory.files.each {|file| file.destroy} ... >> directory.destroy ...
cleanup geymus ~ ⌘ fog ... >> directory = AWS.directories.get(DIRECTORY_NAME) ... >> directory.files.each {|file| file.destroy} ... >> directory.destroy ... >> exit
geoaggregating portable - AWS, Google, Local, Rackspace
geoaggregating portable - AWS, Google, Local, Rackspace lather, rinse, repeat
Phase 3: Profit I’ve got the data, but how do I freemium?
Phase 3: Profit I’ve got the data, but how do I freemium? dns - make your cloud (premium) accessible
Get Connected 7 # setup a connection to the service 8 dns = Fog::DNS.new(credentials)
Get Connected 1 credentials = { 2   :provider  => 'Zerigo', 3   :zerigo_email => AWS_ACCESS_KEY_ID, 4   :zerigo_token => AWS_SECRET_ACCESS_KEY 5 } 6 7 # setup a connection to the service 8 dns = Fog::DNS.new(credentials)
zones 1 # create a zone 2 zone = dns.zone.create( 3   :domain => domain_name, 4   :email  => "admin@#{domain_name}" 5 )
records 1 # create a record 2 record = zones.records.create( 3   :ip   => '1.2.3.4', 4   :name => "#{customer_name}.#{domain_name}", 5   :type => 'A' 6 )
cleanup geymus ~ ⌘ fog ... >> zone = Zerigo.zones.get(ZONE_ID) ... >> zone.records.each {|record| record.destroy} ... >> zone.destroy ... >> exit
geofreemiuming portable - AWS, Linode, Slicehost, Zerigo
geofreemiuming portable - AWS, Linode, Slicehost, Zerigo lather, rinse, repeat
Roadmap
Roadmap compute - consolidating and refining the shared api
Roadmap compute - consolidating and refining the shared api completion - fill in remaining requests and mocks for core services
Roadmap compute - consolidating and refining the shared api completion - fill in remaining requests and mocks for core services errors - more consistent error raising and handling
Roadmap compute - consolidating and refining the shared api completion - fill in remaining requests and mocks for core services errors - more consistent error raising and handling documentation - blog posts, examples, fog.io
Thanks!
Thanks! Questions? (see also: README) examples - http://gist.github.com/729992 slides - http://slidesha.re/hR8sP9 repo - http://github.com/geemus/fog bugs - http://github.com/geemus/fog/issues irc - irc://irc.freenode.net/ruby-fog mailing list - http://groups.google.com/group/ruby-fog twitter - @fog and @geemus

More Related Content

KEY
fog or: How I Learned to Stop Worrying and Love the Cloud (OpenStack Edition)
byWesley Beary
 
ODP
Integrating icinga2 and the HashiCorp suite
byBram Vogelaar
 
PPTX
Streamline Hadoop DevOps with Apache Ambari
byAlejandro Fernandez
 
PPTX
Creating Reusable Puppet Profiles
byBram Vogelaar
 
PDF
Refactoring terraform
byNell Shamrell-Harrington
 
PDF
Autoscaling with hashi_corp_nomad
byBram Vogelaar
 
PPT
Triple Blitz Strike
byDenis Zhdanov
 
PPTX
Tuning Apache Ambari Performance for Big Data at Scale with 3,000 Agents
byAlejandro Fernandez
 
fog or: How I Learned to Stop Worrying and Love the Cloud (OpenStack Edition)
byWesley Beary
 
Integrating icinga2 and the HashiCorp suite
byBram Vogelaar
 
Streamline Hadoop DevOps with Apache Ambari
byAlejandro Fernandez
 
Creating Reusable Puppet Profiles
byBram Vogelaar
 
Refactoring terraform
byNell Shamrell-Harrington
 
Autoscaling with hashi_corp_nomad
byBram Vogelaar
 
Triple Blitz Strike
byDenis Zhdanov
 
Tuning Apache Ambari Performance for Big Data at Scale with 3,000 Agents
byAlejandro Fernandez
 

What's hot

PDF
Observability with Consul Connect
byBram Vogelaar
 
PDF
Puppet and the HashiStack
byBram Vogelaar
 
PDF
Integrating icinga2 and the HashiCorp suite
byBram Vogelaar
 
PPTX
PSGI and Plack from first principles
byPerl Careers
 
PDF
Charla - SharePoint en la Nube (17Jul2013)
byJuan Andrés Valenzuela
 
PDF
Bootstrapping multidc observability stack
byBram Vogelaar
 
PDF
Bootstrapping multidc observability stack
byBram Vogelaar
 
PDF
Scaling Mapufacture on Amazon Web Services
byAndrew Turner
 
KEY
Operation Oriented Web Applications / Yokohama pm7
byMasahiro Nagano
 
PDF
Testing your infrastructure with litmus
byBram Vogelaar
 
PDF
AnyMQ, Hippie, and the real-time web
byclkao
 
PDF
Rhebok, High Performance Rack Handler / Rubykaigi 2015
byMasahiro Nagano
 
PDF
Terraform at Scale - All Day DevOps 2017
byJonathon Brouse
 
PPTX
An intro to Docker, Terraform, and Amazon ECS
byYevgeniy Brikman
 
PDF
VUG5: Varnish at Opera Software
byCosimo Streppone
 
ODP
Asynchronous Threads in Lasso 8.5
bybilcorry
 
PPTX
Ansible fest Presentation slides
byAaron Carey
 
PDF
Ansible : what's ansible & use case by REX
bySaewoong Lee
 
PDF
Securing Prometheus exporters using HashiCorp Vault
byBram Vogelaar
 
PDF
Using Terraform.io (Human Talks Montpellier, Epitech, 2014/09/09)
byStephane Jourdan
 
Observability with Consul Connect
byBram Vogelaar
 
Puppet and the HashiStack
byBram Vogelaar
 
Integrating icinga2 and the HashiCorp suite
byBram Vogelaar
 
PSGI and Plack from first principles
byPerl Careers
 
Charla - SharePoint en la Nube (17Jul2013)
byJuan Andrés Valenzuela
 
Bootstrapping multidc observability stack
byBram Vogelaar
 
Bootstrapping multidc observability stack
byBram Vogelaar
 
Scaling Mapufacture on Amazon Web Services
byAndrew Turner
 
Operation Oriented Web Applications / Yokohama pm7
byMasahiro Nagano
 
Testing your infrastructure with litmus
byBram Vogelaar
 
AnyMQ, Hippie, and the real-time web
byclkao
 
Rhebok, High Performance Rack Handler / Rubykaigi 2015
byMasahiro Nagano
 
Terraform at Scale - All Day DevOps 2017
byJonathon Brouse
 
An intro to Docker, Terraform, and Amazon ECS
byYevgeniy Brikman
 
VUG5: Varnish at Opera Software
byCosimo Streppone
 
Asynchronous Threads in Lasso 8.5
bybilcorry
 
Ansible fest Presentation slides
byAaron Carey
 
Ansible : what's ansible & use case by REX
bySaewoong Lee
 
Securing Prometheus exporters using HashiCorp Vault
byBram Vogelaar
 
Using Terraform.io (Human Talks Montpellier, Epitech, 2014/09/09)
byStephane Jourdan
 

Viewers also liked

DOCX
Windows 市集註冊及上架
byMeng-Ru (Raymond) Tsai
 
DOCX
Getting started windows phone unity
byMeng-Ru (Raymond) Tsai
 
PPTX
20131122 台北遊戲開發者論壇
byMeng-Ru (Raymond) Tsai
 
PPTX
20140222 Unity Windows lab 移轉實作營
byMeng-Ru (Raymond) Tsai
 
KEY
SV Ruby on Rails Meetup - FlickMunk
bySV Ruby on Rails Meetup
 
PPT
O.N.E. Glamour Brand Identity
byShakara Hinds
 
DOCX
Windows phone app 上架說明書
byMeng-Ru (Raymond) Tsai
 
Windows 市集註冊及上架
byMeng-Ru (Raymond) Tsai
 
Getting started windows phone unity
byMeng-Ru (Raymond) Tsai
 
20131122 台北遊戲開發者論壇
byMeng-Ru (Raymond) Tsai
 
20140222 Unity Windows lab 移轉實作營
byMeng-Ru (Raymond) Tsai
 
SV Ruby on Rails Meetup - FlickMunk
bySV Ruby on Rails Meetup
 
O.N.E. Glamour Brand Identity
byShakara Hinds
 
Windows phone app 上架說明書
byMeng-Ru (Raymond) Tsai
 

Similar to How I Learned to Stop Worrying and Love the Cloud - Wesley Beary, Engine Yard

KEY
fog or: How I Learned to Stop Worrying and Love the Cloud
byWesley Beary
 
PDF
Intro to fog and openstack jp
bySatoshi Konno
 
KEY
Geemus
byEngine Yard
 
KEY
Fog
byWesley Beary
 
PDF
Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure
byHabeeb Rahman
 
PDF
Cutting through the fog of cloud
byKyle Rames
 
PDF
A Tale of a Server Architecture (Frozen Rails 2012)
byFlowdock
 
PDF
The Automation Factory
byNathan Milford
 
PDF
OpenStack Deployments with Chef
byMatt Ray
 
PDF
Australian OpenStack User Group August 2012: Chef for OpenStack
byMatt Ray
 
PDF
Swift Install Workshop - OpenStack Conference Spring 2012
byJoe Arnold
 
PDF
Achieving Infrastructure Portability with Chef
byMatt Ray
 
PDF
ChefConf 2012 Spiceweasel
byMatt Ray
 
PPTX
Sonian, Open Source and Sensu
byPete Cheslock
 
PDF
Chef for OpenStack- Fall 2012.pdf
byOpenStack Foundation
 
PDF
Chef for OpenStack - OpenStack Fall 2012 Summit
byMatt Ray
 
KEY
Django deployment with PaaS
byAppsembler
 
PPT
Shmoocon 2013 - OpenStack Security Brief
byopenfly
 
ODP
Deploying Perl apps on dotCloud
bydaoswald
 
PDF
Jeff Lindsay: Building Public Infrastructure with Autosustainable Services
byit-people
 
fog or: How I Learned to Stop Worrying and Love the Cloud
byWesley Beary
 
Intro to fog and openstack jp
bySatoshi Konno
 
Geemus
byEngine Yard
 
Fog
byWesley Beary
 
Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure
byHabeeb Rahman
 
Cutting through the fog of cloud
byKyle Rames
 
A Tale of a Server Architecture (Frozen Rails 2012)
byFlowdock
 
The Automation Factory
byNathan Milford
 
OpenStack Deployments with Chef
byMatt Ray
 
Australian OpenStack User Group August 2012: Chef for OpenStack
byMatt Ray
 
Swift Install Workshop - OpenStack Conference Spring 2012
byJoe Arnold
 
Achieving Infrastructure Portability with Chef
byMatt Ray
 
ChefConf 2012 Spiceweasel
byMatt Ray
 
Sonian, Open Source and Sensu
byPete Cheslock
 
Chef for OpenStack- Fall 2012.pdf
byOpenStack Foundation
 
Chef for OpenStack - OpenStack Fall 2012 Summit
byMatt Ray
 
Django deployment with PaaS
byAppsembler
 
Shmoocon 2013 - OpenStack Security Brief
byopenfly
 
Deploying Perl apps on dotCloud
bydaoswald
 
Jeff Lindsay: Building Public Infrastructure with Autosustainable Services
byit-people
 

How I Learned to Stop Worrying and Love the Cloud - Wesley Beary, Engine Yard

  • 1.
    OR: fog HOW I LEARNED TO STOP WORRYING AND LOVE THE CLOUD
  • 2.
    geemus (Wesley Beary) web:github.com/geemus twitter: @geemus
  • 3.
    employer and sponsor web:engineyard.com twitter: @engineyard
  • 4.
    API driven ondemand services CLOUD core: compute, dns, storage also: kvs, load balance, ...
  • 5.
  • 6.
    What? on demand - pay for only what you actually use
  • 7.
    What? on demand - pay for only what you actually use flexible - add and remove resources in minutes (instead of weeks)
  • 8.
    What? on demand - pay for only what you actually use flexible - add and remove resources in minutes (instead of weeks) repeatable - code, test, repeat
  • 9.
    What? on demand - pay for only what you actually use flexible - add and remove resources in minutes (instead of weeks) repeatable - code, test, repeat resilient - build better systems with transient resources
  • 10.
  • 11.
    But, but... option overload- which provider should I use
  • 12.
    But, but... option overload- which provider should I use intimidating - each service requires separate expertise
  • 13.
    But, but... option overload- which provider should I use intimidating - each service requires separate expertise tools - vastly different API’s built on varying levels of quality
  • 14.
    Ruby cloud services web:github.com/geemus/fog twitter: @fog
  • 15.
  • 16.
    Why? portable - AWS, Bluebox, Brightbox, Google, Rackspace, Slicehost, Terremark, ...
  • 17.
    Why? portable - AWS, Bluebox, Brightbox, Google, Rackspace, Slicehost, Terremark, ... powerful - compute, dns, storage, collections, models, mocks, requests, ...
  • 18.
    Why? portable - AWS, Bluebox, Brightbox, Google, Rackspace, Slicehost, Terremark, ... powerful - compute, dns, storage, collections, models, mocks, requests, ... established - 50k downloads, 860 followers, 100 forks, 40 contributors, me, ...
  • 19.
    Why? portable - AWS, Bluebox, Brightbox, Google, Rackspace, Slicehost, Terremark, ... powerful - compute, dns, storage, collections, models, mocks, requests, ... established - 50k downloads, 860 followers, 100 forks, 40 contributors, me, ... Fog.mock! - faster, cheaper, simulated cloud behavior
  • 20.
  • 21.
    Who? libraries - carrierwave, chef, deckard, gaff, gemcutter, ... products - DevStructure, Engine Yard, iSwifter, OpenFeint, RowFeeder, ...
  • 22.
  • 23.
    What? That’s great andall but I don’t have a use case...
  • 24.
    What? That’s great andall but I don’t have a use case... uptime - because who wants a busted web site?
  • 25.
    Setup geymus~ ⌘ gem install fog or geymus ~ ⌘ sudo gem install fog
  • 26.
    Get Connected 7 #setup a connection to the service 8 compute = Fog::Compute.new(credentials)
  • 27.
    Get Connected 1 credentials = { 2   :provider           => 'Rackspace', 3   :rackspace_api_key  => RACKSPACE_API_KEY, 4   :rackspace_username => RACKSPACE_USERNAME 5 } 6 7 # setup a connection to the service 8 compute = Fog::Compute.new(credentials)
  • 28.
    Bootstrap 7 # bootserver and setup ssh keys 8 server = compute.servers.bootstrap(server_attributes)
  • 29.
    Bootstrap 1 server_attributes = { 2   :image_id         => '49', 3   :private_key_path => PRIVATE_KEY_PATH, 4   :public_key_path  => PUBLIC_KEY_PATH 5 } 6 7 # boot server and setup ssh keys 8 server = compute.servers.bootstrap(server_attributes)
  • 30.
  • 31.
    Servers? 1 compute.servers #list servers, same as #all
  • 32.
    Servers? 1 compute.servers #list servers, same as #all 2 3 compute.servers.get(1234567890) # server by id
  • 33.
    Servers? 1 compute.servers #list servers, same as #all 2 3 compute.servers.get(1234567890) # server by id 4 5 compute.servers.reload # update to latest
  • 34.
    Servers? 1 compute.servers # list servers, same as #all 2 3 compute.servers.get(1234567890) # server by id 4 5 compute.servers.reload # update to latest 6 7 compute.servers.new(attributes) # local model
  • 35.
    Servers? 1 compute.servers # list servers, same as #all 2 3 compute.servers.get(1234567890) # server by id 4 5 compute.servers.reload # update to latest 6 7 compute.servers.new(attributes) # local model 8 9 compute.servers.create(attributes) # remote model
  • 36.
  • 37.
  • 38.
    bootstrap? convenience method for servers create - creates a server matching attributes wait_for { ready? } - waits until server responds ssh - sets up ssh keys and disables password authentication
  • 39.
  • 40.
    ping 1 # pingtarget 10 times 2 ssh_results = server.ssh("ping -c 10 #{target}")
  • 41.
    ping 1 # pingtarget 10 times 2 ssh_results = server.ssh("ping -c 10 #{target}") 3 stdout = ssh_results.first.stdout
  • 42.
    ping 1 # ping target 10 times 2 ssh_results = server.ssh("ping -c 10 #{target}") 3 stdout = ssh_results.first.stdout 4 5 # parse result, last line is summary 6 # round-trip min/avg/max/stddev = A.A/B.B/C.C/D.D ms
  • 43.
    ping 1 # ping target 10 times 2 ssh_results = server.ssh("ping -c 10 #{target}") 3 stdout = ssh_results.first.stdout 4 5 # parse result, last line is summary 6 # round-trip min/avg/max/stddev = A.A/B.B/C.C/D.D ms 7 stats = stdout.split("/n").last.split(' ')[-2] 8 min, avg, max, stddev = stats.split('/')
  • 44.
    ping 1 # ping target 10 times 2 ssh_results = server.ssh("ping -c 10 #{target}") 3 stdout = ssh_results.first.stdout 4 5 # parse result, last line is summary 6 # round-trip min/avg/max/stddev = A.A/B.B/C.C/D.D ms 7 stats = stdout.split("/n").last.split(' ')[-2] 8 min, avg, max, stddev = stats.split('/') NOTE: most complex code was string parsing!?!
  • 45.
  • 46.
    cleanup  1 # shutdownthe server  2 server.destroy
  • 47.
    cleanup  1 # shutdownthe server  2 server.destroy  3  4 # return the data as a hash  5 {  6   :min    => min,  7   :avg    => avg,  8   :max    => max,  9   :stddev => stddev 10 }
  • 48.
    geopinging v1  1 #specify a different provider  2 credentials = {  3   :provider => 'AWS',  4   :aws_access_key_id => AWS_ACCESS_KEY_ID,  5   :aws_secret_access_key => AWS_SECRET_ACCESS_KEY  6 }  7  8 server_attributes = {  9   :image_id         => 'ami-1a837773', 10   :private_key_path => PRIVATE_KEY_PATH, 11   :public_key_path  => PUBLIC_KEY_PATH, 12   :username         => 'ubuntu' 13 }
  • 49.
    geopinging v2 1 # specify a different aws region 2 # ['ap-southeast-1', 'eu-west-1', 'us-west-1] 3 credentials.merge!({ 4   :region => 'eu-west-1' 5 })
  • 50.
    geopinging v... portable - AWS, Bluebox, Brightbox, Rackspace, Slicehost, Terremark, ...
  • 51.
    geopinging v... portable - AWS, Bluebox, Brightbox, Rackspace, Slicehost, Terremark, ... lather, rinse, repeat
  • 52.
    How? That is awesome,but how did you know...
  • 53.
    exploring geymus ~ ⌘fog   To run as 'default', add the following to ~/.fog :default:   :aws_access_key_id:     INTENTIONALLY_LEFT_BLANK   :aws_secret_access_key: INTENTIONALLY_LEFT_BLANK   :public_key_path:       INTENTIONALLY_LEFT_BLANK   :private_key_path:      INTENTIONALLY_LEFT_BLANK   :rackspace_api_key:     INTENTIONALLY_LEFT_BLANK   :rackspace_username:    INTENTIONALLY_LEFT_BLANK ...
  • 54.
  • 55.
  • 56.
    sign posts geymus ~⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace
  • 57.
    sign posts geymus ~⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers
  • 58.
    sign posts geymus ~⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace]
  • 59.
    sign posts geymus ~⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace] >> Rackspace.collections
  • 60.
    sign posts geymus ~⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace] >> Rackspace.collections [:directories, :files, :flavors, :images, :servers]
  • 61.
    sign posts geymus ~⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace] >> Rackspace.collections [:directories, :files, :flavors, :images, :servers] >> Rackspace[:compute]
  • 62.
    sign posts geymus ~⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace] >> Rackspace.collections [:directories, :files, :flavors, :images, :servers] >> Rackspace[:compute] #<Fog::Rackspace::Compute ...>
  • 63.
    sign posts geymus ~⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace] >> Rackspace.collections [:directories, :files, :flavors, :images, :servers] >> Rackspace[:compute] #<Fog::Rackspace::Compute ...> >> Rackspace[:compute].requests
  • 64.
    sign posts geymus ~⌘ fog Welcome to fog interactive! :default credentials provide AWS and Rackspace >> providers [AWS, Rackspace] >> Rackspace.collections [:directories, :files, :flavors, :images, :servers] >> Rackspace[:compute] #<Fog::Rackspace::Compute ...> >> Rackspace[:compute].requests [:confirm_resized_server, ..., :update_server]
  • 65.
  • 66.
  • 67.
  • 68.
  • 69.
    requests? >> Rackspace[:compute].list_servers #<Excon::Response:0x________ @body ={ "servers" => [] }, @headers = { "X-PURGE-KEY"=>"/______/servers", ..., "Connection"=>"keep-alive" },
  • 70.
    requests? >> Rackspace[:compute].list_servers #<Excon::Response:0x________ @body ={ "servers" => [] }, @headers = { "X-PURGE-KEY"=>"/______/servers", ..., "Connection"=>"keep-alive" }, @status=200>
  • 71.
  • 72.
  • 73.
    sanity check >> Rackspace.servers.select{|server| server.ready?} <Fog::Rackspace::Compute::Servers filters={} [] >
  • 74.
    sanity check >> Rackspace.servers.select{|server| server.ready?} <Fog::Rackspace::Compute::Servers filters={} [] > >> AWS.servers.select {|server| server.ready?}
  • 75.
    sanity check >> Rackspace.servers.select{|server| server.ready?} <Fog::Rackspace::Compute::Servers filters={} [] > >> AWS.servers.select {|server| server.ready?} <Fog::AWS::Compute::Servers [] >
  • 76.
    sanity check >> Rackspace.servers.select{|server| server.ready?} <Fog::Rackspace::Compute::Servers filters={} [] > >> AWS.servers.select {|server| server.ready?} <Fog::AWS::Compute::Servers [] > >> exit
  • 77.
  • 78.
  • 79.
    finding images >> Rackspace.images.table([:id,:name]) +--------- +------------------------------------------+ | id | name | +--------- +------------------------------------------+ | 49 | Ubuntu 10.04 LTS (lucid) | +--------- +------------------------------------------+ ...
  • 80.
    finding images >> Rackspace.images.table([:id,:name]) +--------- +------------------------------------------+ | id | name | +--------- +------------------------------------------+ | 49 | Ubuntu 10.04 LTS (lucid) | +--------- +------------------------------------------+ ... >> AWS.images # I use alestic.com listing
  • 81.
    finding images >> Rackspace.images.table([:id,:name]) +--------- +------------------------------------------+ | id | name | +--------- +------------------------------------------+ | 49 | Ubuntu 10.04 LTS (lucid) | +--------- +------------------------------------------+ ... >> AWS.images # I use alestic.com listing ...
  • 82.
  • 83.
    exploring... Ittakes forever!
  • 84.
    exploring... It takes forever! It’s so expensive!
  • 85.
    exploring... It takes forever! It’s so expensive! A warm welcome for Fog.mock!
  • 86.
    Mocks! geymus ~ ⌘FOG_MOCK=true fog or require ‘fog’ Fog.mock!
  • 87.
  • 88.
  • 89.
    simulation Most functions just work! Unimplemented mocks? Errors keep you on track.
  • 90.
    simulation Most functions just work! Unimplemented mocks? Errors keep you on track. Tests run against both, so it is either consistent or a bug.
  • 91.
    Back to Business I have a bunch of data now what?
  • 92.
    Back to Business I have a bunch of data now what? storage - aggregating cloud data
  • 93.
    Get Connected 7 #setup a connection to the service 8 storage = Fog::Storage.new(credentials)
  • 94.
    Get Connected 1 credentials = { 2   :provider => 'AWS', 3   :aws_access_key_id => AWS_ACCESS_KEY_ID, 4   :aws_secret_access_key => AWS_SECRET_ACCESS_KEY 5 } 6 7 # setup a connection to the service 8 storage = Fog::Storage.new(credentials)
  • 95.
    directories 1 # create a directory 2 directory = storage.directories.create( 3   :key    => directory_name, 4   :public => true 5 )
  • 96.
  • 97.
    files 1 # store the file 2 file = directory.files.create( 3   :body   => File.open(path), 4   :key    => name, 5   :public => true 6 )
  • 98.
    files 1 # store the file 2 file = directory.files.create( 3   :body   => File.open(path), 4   :key    => name, 5   :public => true 6 ) 7 8 # return the public url for the file 9 file.public_url
  • 99.
    geostorage  1 # specify a different provider  2 credentials = {  3   :provider           => 'Rackspace',  4   :rackspace_api_key  => RACKSPACE_API_KEY,  5   :rackspace_username => RACKSPACE_USERNAME  6 }
  • 100.
  • 101.
  • 102.
    cleanup geymus ~ ⌘fog ... >> directory = AWS.directories.get(DIRECTORY_NAME) ...
  • 103.
    cleanup geymus ~ ⌘fog ... >> directory = AWS.directories.get(DIRECTORY_NAME) ... >> directory.files.each {|file| file.destroy} ...
  • 104.
    cleanup geymus ~ ⌘fog ... >> directory = AWS.directories.get(DIRECTORY_NAME) ... >> directory.files.each {|file| file.destroy} ... >> directory.destroy ...
  • 105.
    cleanup geymus ~ ⌘fog ... >> directory = AWS.directories.get(DIRECTORY_NAME) ... >> directory.files.each {|file| file.destroy} ... >> directory.destroy ... >> exit
  • 106.
    geoaggregating portable - AWS, Google, Local, Rackspace
  • 107.
    geoaggregating portable - AWS, Google, Local, Rackspace lather, rinse, repeat
  • 108.
    Phase 3: Profit I’vegot the data, but how do I freemium?
  • 109.
    Phase 3: Profit I’vegot the data, but how do I freemium? dns - make your cloud (premium) accessible
  • 110.
    Get Connected 7 #setup a connection to the service 8 dns = Fog::DNS.new(credentials)
  • 111.
    Get Connected 1 credentials = { 2   :provider  => 'Zerigo', 3   :zerigo_email => AWS_ACCESS_KEY_ID, 4   :zerigo_token => AWS_SECRET_ACCESS_KEY 5 } 6 7 # setup a connection to the service 8 dns = Fog::DNS.new(credentials)
  • 112.
    zones 1 # create a zone 2 zone = dns.zone.create( 3   :domain => domain_name, 4   :email  => "admin@#{domain_name}" 5 )
  • 113.
    records 1 # create a record 2 record = zones.records.create( 3   :ip   => '1.2.3.4', 4   :name => "#{customer_name}.#{domain_name}", 5   :type => 'A' 6 )
  • 114.
    cleanup geymus ~ ⌘fog ... >> zone = Zerigo.zones.get(ZONE_ID) ... >> zone.records.each {|record| record.destroy} ... >> zone.destroy ... >> exit
  • 115.
    geofreemiuming portable - AWS, Linode, Slicehost, Zerigo
  • 116.
    geofreemiuming portable - AWS, Linode, Slicehost, Zerigo lather, rinse, repeat
  • 117.
  • 118.
    Roadmap compute - consolidating and refining the shared api
  • 119.
    Roadmap compute - consolidating and refining the shared api completion - fill in remaining requests and mocks for core services
  • 120.
    Roadmap compute - consolidating and refining the shared api completion - fill in remaining requests and mocks for core services errors - more consistent error raising and handling
  • 121.
    Roadmap compute - consolidating and refining the shared api completion - fill in remaining requests and mocks for core services errors - more consistent error raising and handling documentation - blog posts, examples, fog.io
  • 122.
  • 123.
    Thanks! Questions? (see also: README) examples - http://gist.github.com/729992 slides - http://slidesha.re/hR8sP9 repo - http://github.com/geemus/fog bugs - http://github.com/geemus/fog/issues irc - irc://irc.freenode.net/ruby-fog mailing list - http://groups.google.com/group/ruby-fog twitter - @fog and @geemus

Editor's Notes