Usando o Cloud
                      como desenvolvedor!




@fabiokung
fabio.kung@gmail.com
fabio.kung@locaweb.com.br
Como
usar

tudo
isso?
Administração Remota
Gerenciador




                                ...
DHCP     Proxy       Firewall
Gerenciador




WS WS WS   WS WS WS      WS WS WS
                                     ...
 DHCP        Proxy        Firewall
Gerenciador

           WS   WS




WS WS WS   WS WS WS      WS WS WS
                                     ...
 DHCP           Proxy     Firewall
Gerenciador

           WS   WS




WS WS WS   WS WS WS      WS WS WS
                                     ...
 DHCP           Proxy     Firewall
Gerenciador

           WS   WS


            Autenticação?
            Monitoração?
            Criptografia?
                 ...
WS WS WS   WS WS WS         WS WS WS
                                        ...
 DHCP           Proxy        Firewall
Gerenciador

           WS   WS




           ESB
WS WS WS   WS WS WS      WS WS WS
                                     ...
 DHCP           Proxy     Firewall
Gerenciador

           WS   WS




           ESB
WS WS WS   WS WS WS      WS WS WS
                                     ...
 DHCP           Proxy     Firewall
Gerenciador

           WS   WS




           ESB
WS WS WS   WS WS WS      WS WS WS
                                     ...
 DHCP           Proxy     Firewall
KISSH
Gerenciador
          Logica
                          sudoers,
                      pub/priv keypairs




 SSH       SSH         SSH
                                 ...
DHCP     Proxy       Firewall
SSHService
require "net/ssh"

Net::SSH.start("host", "user", options) do |ssh|
  ssh_result = ssh.open_channel do |channel|
    channel.exec command
    channel.on_data do |ch, data|
      ch[:out] << data
    end
    channel.on_request("exit-status") do |ch, req|
      ch[:status] = req.read_long
    end
  end
  ssh.loop
end
one master, several slaves
Gerenciador




Hypervisor   Hypervisor    Hypervisor   ...
Load Balancer
            Proxy Reverso




WebServer    WebServer      WebServer   ...
APIs
http://deltacloud.org
a unified interface to the cloud


libcloud is a standard client library for many popular cloud providers,
                           written in python
http://github.com/geemus/fog
http://github.com/jclouds/jclouds




http://dasein-cloud.sourceforge.net/
http://openstack.org
Gerenciador



    e
com
muitos
nós?


Hypervisor   Hypervisor    Hypervisor   ...
Árvore de requisições
Gerenciador




    Agregador                      Agregador




Worker   Worker     Worker      Worker   Worker   ...
o informante
  canary requests
backup requests
      volume
many internal services
"break large complex systems down into many services"
                      -- Google
self-test boot
     dependências
Comunicações
REST, Protocol Buffers, Thrift
Controle de tarefas
     Mundo Assíncrono




        filas no DB? :-(
process :of => :uninstall do
   must_be :uninstalling

  transition :power_off_if_needed,
    :from => :uninstalling,
    :to => :powered_off

  transition :delete_snapshots,
    :from => :powered_off,
    :to => :snapshots_deleted

  transition :delete_disks,
    :from => :snapshots_deleted,
    :to => :disks_deleted

  transition :delete_machine,


máquinas de estado
    :from => :disks_deleted,
    :to => :machine_deleted
Resque
tolerate failures
Jason McHugh, Principal Engineer at Amazon
back of the envelope
    calculations
       thumbnails
Cache!
disco, virtualização
Instant Messaging
XMPP JIDs


username@domain /resource
fabiokung@servidor
machine01@servidor




                     machine02@servidor

fabiokung@servidor




                     machine03@servidor
installer@servidor




                        cloner@servidor

fabiokung@servidor




                     uninstaller@servidor
XMPP4R
http://home.gna.org/xmpp4r
múltiplos workers


                      installer@servidor




                             ...
frontend@servidor




                      installer@servidor
frontend@servidor   installer@servidor/A




                           ...
                    installer@servidor/B
quem pode?
                    to: installer@servidor

frontend@servidor                            installer@servidor/A




                                                    ...
                                             installer@servidor/B
quem pode?
                      to: installer@servidor

                                                 installer@servidor/A
frontend@servidor            eu posso!
                    from: installer@servidor/A




                                                        ...
                             eu posso!
                    from: installer@servidor/B




                                                 installer@servidor/B
quem pode?
                      to: installer@servidor

                                                 installer@servidor/A
frontend@servidor            eu posso!
                    from: installer@servidor/A




                                                        ...
                             eu posso!
                    from: installer@servidor/B


                          ok, então instala!
                     to: installer@servidor/B
                                                 installer@servidor/B
quem pode?
                      to: installer@servidor

                                                 installer@servidor/A
frontend@servidor            eu posso!
                    from: installer@servidor/A
      ...




                                                        ...
                             eu posso!
                    from: installer@servidor/B


                          ok, então instala!
                     to: installer@servidor/B
frontend@servidor                                installer@servidor/B
presença,
pubsub...
Configuration
Management
•Puppet
•Chef
•CFEngine
Chef
recipes
                            chef-client


               recipes


chef-server       recipes   chef-client




                            chef-client
file "/etc/myapp.conf" do
  owner www-data
  group www-data
  mode "0600"
  action :create
end

package "mysql-server" do
  action :install
end

execute "open http port" do
  command "iptables -I INPUT -p tcp --dport 80 -j ACCEPT"
  creates "/var/run/http_port_opened"
end
package "sudo" do
  action :upgrade
end

user "sshservice" do
  system true
  action :create
end

template "/etc/sudoers" do
  source "sudoers.erb"
  mode 0440
  owner "root"
  group "root"
  variables(:user => "sshservice")
  action :create
end
bundle common control
{
    bundlesequence => { "ssh" };
    inputs => { "cfengine_stdlib.cf" }
}

bundle agent ssh
{
    packages:
        "sudo"
            package_policy => "add"
            package_method => "deb"
    files:
        "/etc/sudoers"
            edit_defaults => empty,
            edit_line => expand_template("sudoers.in"),
            perms => mog("0440", root, root),
            create => "true";
    methods:
        "sshservice" usebundle => create_user("sshservice")
}
http://cookbooks.opscode.com

 http://github.com/37signals/37s_cookbooks

http://github.com/engineyard/ey-cloud-recipes
Snapshots
clones
Testes
multiplos browsers, drivers, ...
Multiplos envs
dev, staging, preproduction, integration, production, ...
Blue Green
           deployments
http://martinfowler.com/bliki/BlueGreenDeployment.html
continuous   Deployment
Capistrano
 maven, ant, make, ...
Chef?
deploy "/srv/my-cms" do
  repo "git://github.com/radiant/radiant.git"
  revision "HEAD"
  user "cms"
  migrate true
  migration_command "rake db:migrate"
  environment "production"
  action :deploy
  restart_command "touch tmp/restart.txt"
end
Pacotes!
 deb, rpm, ...
require'drb';F,D,C,P,M,U,*O=File,Class,Dir,*ARGV;def s(p)F.split(p[/[^|].*/])[-1
]end;def c(u);DRbObject.new((),u)end;def x(u)[P,u].hash;end;M=="client"&&c(U).f(
x(U)).each{|n|p,c=x(n),c(n);(c.f(p,O[0],0).map{|f|s f}-D["*"]).each{|f|F.open(f,
"w"){|o|o<<c.f(p,f,1)}}}||(DRb.start_service U,C.new{def f(c,a=[],t=2)c==x(U)&&(
t==0&&D[s(a)]||t==1&&F.read(s(a))||p(a))end;def y()(p(U)+p).each{|u|c(u).f(x(u),
p(U))rescue()};self;end;private;def p(x=[]);O.push(*x).uniq!;O;end}.new.y;sleep)

              http://ansuz.sooke.bc.ca/software/molester/2004121602.php




                                    P2P
build process generating
         images
     Michael T. Nygard - “Release It”
Multitenancy
chargeback, isolamento, segurança
• novas linguagens (programação e consulta)
• preprocessador/otimizador de queries
• analise continua de logs
• monitoramento de queries executadas no banco
Gregos e Troianos

• clients must be demanding 8 different things;
• doing 6 of them is easy;
• handling 7 of them requires real thought;
• dealing with all 8 usually results in a worse system;
Game Days
  “be prepared”
Dúvidas?




   fabio.kung@gmail.com
fabio.kung@locaweb.com.br      Obrigado!
http://twitter.com/fabiokung

Usando o Cloud

Editor's Notes

  • #4 assincrono
  • #16 autoscaling, API tolerancia a falhas monitora&amp;#xE7;&amp;#xE3;o, estado do sistema
  • #17 autoscaling, API tolerancia a falhas monitora&amp;#xE7;&amp;#xE3;o, estado do sistema
  • #24 o que fazer quando temos muitos n&amp;#xF3;s?
  • #29 divis&amp;#xE3;o de responsabilidade virtual appliance
  • #30 depend&amp;#xEA;ncias
  • #33 processo assincrono retry
  • #34 solido redis tonf of plugins
  • #36 mapreduce comunidade cientifica long running
  • #61 testes para cfmng autoscaling
  • #64 ips el&amp;#xE1;sticos