BATOU         Multi-(Component|Environment|Host|Platform) deployment                            Christian Theune <ct@gocep...
Wednesday, October 31, 12
Wednesday, October 31, 12
INHALT    • Motivation    • Hello World    • Tiefer            in den Kaninchenbau    • AusblickWednesday, October 31, 12
WARUM?Wednesday, October 31, 12
Wednesday, October 31, 12
Wednesday, October 31, 12
Wednesday, October 31, 12
Wednesday, October 31, 12
Wednesday, October 31, 12
PARTY LIKE ITS ...    • 1999: PHP             FTP-Uploads    • 2001: Shell-Skripte         um Zope-Instanzen zu verwalten ...
BATOU                                VS               Puppet, Chef    Fabric, Salt         JuJu      Systemkonfiguration   ...
HELLO WORLD                                  http://www.flickr.com/photos/oskay/472097903Wednesday, October 31, 12
./.hg                     ./buildout.cfg                     ./bootstrap.py                     ./components              ...
from batou.component import Component   from batou.lib.file import File   class Hello(Component):          def configure(s...
[hosts]   localhost = hello                            environments/dev.cfgWednesday, October 31, 12
$ bin/batou-local dev localhost   Updating Hello > File(hello) > Presence(hello)   Updating Hello > File(hello) > Content(...
./                     ./components                     ./environments                     ...                     ./work ...
$ bin/batou-local dev localhostWednesday, October 31, 12
$ cat "foo" > work/hello/hello   $ bin/batou-local dev localhost   Updating Hello > File(hello) > Content(hello)Wednesday,...
[environment]   service_user = test   host_domain = gocept.net   [hosts]   test01 = hello   test02 = hello                ...
$ bin/batou-remote prod   test02.gocept.net: connecting   test01.gocept.net: connecting   test01.gocept.net: bootstrapping...
$ bin/batou-remote prod   test02.gocept.net: connecting   test01.gocept.net: connecting   test01.gocept.net: bootstrapping...
HELLO WORLD    • Komponenten              und Environments    • batou-local           im Konfiguration eines Hosts lokal zu...
IN DEN KANINCHENBAU    • ein        Programm    • verwaltet             mit SupervisorWednesday, October 31, 12
DAS PROGRAMM                            #!/usr/bin/env python                            import time                      ...
KOMPONENTE from batou.component import Component from batou.lib.file import File class Hello(Component):               def...
ENVIRONMENT  [hosts]  localhost = hello                             environments/dev.cfgWednesday, October 31, 12
INSTALLATION   $ bin/batou-local dev localhost   Updating Hello > File(hello) > Presence(hello)   Updating Hello > File(he...
LÄUFT?                 $ ./work/hello/hello                 Hello, world!                 Hello, world!                 ^C...
Wednesday, October 31, 12
BUILDOUT               [buildout]               parts = supervisor               [supervisor]               recipe = colle...
BUILDOUT            from batou.component import Component            from batou.lib.buildout import Buildout            cl...
SERVER-ADRESSE    [buildout]    parts = supervisor    [supervisor]    recipe = collective.recipe.supervisor    port = {{co...
SERVER-ADRESSE            from batou.component import Component            from batou.lib.buildout import Buildout        ...
NETZWERKADRESSEN                        a = Address(localhost:631)                        a = Address(localhost, 631)     ...
SERVER-ADRESSE            from batou.component import Component            from batou.lib.buildout import Buildout        ...
SERVER-ADRESSE   [hosts]   localhost = program, supervisor   [component:supervisor]   port = 6000                         ...
WELCHE PROGRAMME                           STARTEN? from batou.component import Component from batou.lib.file import File ...
WELCHE PROGRAMME                           STARTEN? class Supervisor(Component):        def configure(self):          self...
WELCHE PROGRAMME                           STARTEN?    [buildout]    parts = supervisor    [supervisor]    recipe = collec...
RESSOURCEN UND                            ABHÄNGIGKEITEN    • provide(key,             value)    • require(key,           ...
RESSOURCEN UND                             ABHÄNGIGKEITEN    • provide               ohne require    • require            ...
HOOK KOMPONENTEN from batou.component import Component from batou.lib.file import File class Hello(Component):        def ...
class Program(HookComponent):  namevar = name  key = supervisor:Program      command = None      program_template = {name}...
HOOK COMPONENTS class Program(HookComponent):   key = supervisor:Program                         Kann eigentlich nicht   ....
WELCHE PROGRAMME                            STARTEN[buildout]parts = supervisor[supervisor]recipe = collective.recipe.supe...
SUPERVISOR STARTEN class Supervisor(Component):   def configure(self):     ...   def verify(self):     self.assert_file_is...
@REBOOT   class Service(Component):     namevar = executable       Nur ein "bag".     pidfile = None   class Supervisor(Co...
PLATTFORMEN@platform(flyingcircus.io, Service)class UserInit(Component):       def configure(self):         target = /var/...
PLATTFORMEN  [environment]  host_domain = flyingcircus.io  platform = flyingcircus.io  $ bin/batou-local --platform=     p...
REVIEW    • File              taucht häufig auf    • Address    • Jinja2-Templates    • Environment             overrides  ...
TIPPS    • Komponenten            unabhängig von Host    • Gemeinsamkeiten           in Komponenten bündeln    • verify/up...
VORDEFINIERTE                                   KOMPONENTEN                            File    VirtualEnv    Buildout    z...
FAZIT    • modell-getrieben             aber pragmatisch    • Konvergenz             und Idempotenz    • Der Teufel       ...
AUSSICHT    • Dokumentation    • Bessere               Fehlermeldungen    • Besserer              Status-Output    • Alter...
http://mobro.co/theuniWednesday, October 31, 12
FRAGEN?    • http://bitbucket.org/gocept/batou    • http://packages.python.org/batou    • Bitte                 gebt mir B...
Upcoming SlideShare
Loading in …5
×

Batou - multi-(host|component|environment|version|platform) deployment

490
-1

Published on

Batou ist ein Open-Source Werkzeug um mit Python das Deployment von komplexen Anwendungen zu beschreiben und zu automatisieren.
Das Konzept besteht darin eine passende allgemeine Modellsprache in Form einer API zu definieren. Dabei stehen Wiederverwendung, Toleranz gegenüber unbekannten Zuständen und Ausdrucksstärke im Vordergrund.
Der Vortrag zeigt den Werdegang und unsere Motivation zur Entwicklung von batou auf und zeigt die konzeptionellen Stärken anhand praktischer Beispiele.
Wir haben in den letzten Jahren eine Reihe großer und kleiner Projekte beim Deployment betreut.
Im Rahmen unserer Plattform "gocept.net" bereiten wir unsere Erfahrungen in Dokumentation und offenen Werkzeugen auf und haben daher batou entwickelt.
batou ist sowohl in Python geschrieben und nutzt Python um Deployment-Strukturen zu beschreiben.
Wir möchten Batou gerne vorstellen um unsere Erfahrungen weiterzugeben und (auch auf den Sprints) anzuregen sich mit dem Werkzeug auseinanderzusetzen, Feedback zu bekommen, und daran weiterzuarbeiten.

Siehe auch: http://bitbucket.org/gocept/batou

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
490
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Batou - multi-(host|component|environment|version|platform) deployment

  1. 1. BATOU Multi-(Component|Environment|Host|Platform) deployment Christian Theune <ct@gocept.com>Wednesday, October 31, 12
  2. 2. Wednesday, October 31, 12
  3. 3. Wednesday, October 31, 12
  4. 4. INHALT • Motivation • Hello World • Tiefer in den Kaninchenbau • AusblickWednesday, October 31, 12
  5. 5. WARUM?Wednesday, October 31, 12
  6. 6. Wednesday, October 31, 12
  7. 7. Wednesday, October 31, 12
  8. 8. Wednesday, October 31, 12
  9. 9. Wednesday, October 31, 12
  10. 10. Wednesday, October 31, 12
  11. 11. PARTY LIKE ITS ... • 1999: PHP FTP-Uploads • 2001: Shell-Skripte um Zope-Instanzen zu verwalten • 2003: Zope 2.4 “mkinstance” • 2006: zc.buildout • 2009: FabricWednesday, October 31, 12
  12. 12. BATOU VS Puppet, Chef Fabric, Salt JuJu Systemkonfiguration versus Service Modellierung Laufzeitumgebung deploymentWednesday, October 31, 12
  13. 13. HELLO WORLD http://www.flickr.com/photos/oskay/472097903Wednesday, October 31, 12
  14. 14. ./.hg ./buildout.cfg ./bootstrap.py ./components ./components/hello ./components/hello/component.py ./environments ./environments/dev.cfg ./environments/prod.cfg $ find .Wednesday, October 31, 12
  15. 15. from batou.component import Component from batou.lib.file import File class Hello(Component):    def configure(self):        self += File(hello, content=Hello world) components/hello/component.pyWednesday, October 31, 12
  16. 16. [hosts] localhost = hello environments/dev.cfgWednesday, October 31, 12
  17. 17. $ bin/batou-local dev localhost Updating Hello > File(hello) > Presence(hello) Updating Hello > File(hello) > Content(hello)Wednesday, October 31, 12
  18. 18. ./ ./components ./environments ... ./work ./work/hello ./work/hello/hello $ find .Wednesday, October 31, 12
  19. 19. $ bin/batou-local dev localhostWednesday, October 31, 12
  20. 20. $ cat "foo" > work/hello/hello $ bin/batou-local dev localhost Updating Hello > File(hello) > Content(hello)Wednesday, October 31, 12
  21. 21. [environment] service_user = test host_domain = gocept.net [hosts] test01 = hello test02 = hello environments/prod.cfgWednesday, October 31, 12
  22. 22. $ bin/batou-remote prod test02.gocept.net: connecting test01.gocept.net: connecting test01.gocept.net: bootstrapping test02.gocept.net: bootstrapping OK OK Deploying test01.gocept.net/hello Updating Hello > File(hello) > Presence(hello) Updating Hello > File(hello) > Content(hello) OK Deploying test02.gocept.net/hello Updating Hello > File(hello) > Presence(hello) Updating Hello > File(hello) > Content(hello) OKWednesday, October 31, 12
  23. 23. $ bin/batou-remote prod test02.gocept.net: connecting test01.gocept.net: connecting test01.gocept.net: bootstrapping test02.gocept.net: bootstrapping OK OK Deploying test01.gocept.net/hello OK Deploying test02.gocept.net/hello OKWednesday, October 31, 12
  24. 24. HELLO WORLD • Komponenten und Environments • batou-local im Konfiguration eines Hosts lokal zu realisieren • batou-remote um eine ganze Umgebung zu deployen • Konvergenz und IdempotenzWednesday, October 31, 12
  25. 25. IN DEN KANINCHENBAU • ein Programm • verwaltet mit SupervisorWednesday, October 31, 12
  26. 26. DAS PROGRAMM #!/usr/bin/env python import time while True: print Hello, world! time.sleep(1)Wednesday, October 31, 12
  27. 27. KOMPONENTE from batou.component import Component from batou.lib.file import File class Hello(Component): def configure(self): self += File(hello, source=hello.py, mode=0o755) components/hello/component.pyWednesday, October 31, 12
  28. 28. ENVIRONMENT [hosts] localhost = hello environments/dev.cfgWednesday, October 31, 12
  29. 29. INSTALLATION $ bin/batou-local dev localhost Updating Hello > File(hello) > Presence(hello) Updating Hello > File(hello) > Content(hello) Updating Hello > File(hello) > Mode(hello)Wednesday, October 31, 12
  30. 30. LÄUFT? $ ./work/hello/hello Hello, world! Hello, world! ^CWednesday, October 31, 12
  31. 31. Wednesday, October 31, 12
  32. 32. BUILDOUT [buildout] parts = supervisor [supervisor] recipe = collective.recipe.supervisor components/supervisor/buildout.cfgWednesday, October 31, 12
  33. 33. BUILDOUT from batou.component import Component from batou.lib.buildout import Buildout class Supervisor(Component): def configure(self): self += Buildout(supervisor, python=2.7) components/supervisor/component.pyWednesday, October 31, 12
  34. 34. SERVER-ADRESSE [buildout] parts = supervisor [supervisor] recipe = collective.recipe.supervisor port = {{component.address.listen}} serverurl = http://{{component.address.listen}} components/supervisor/buildout.cfgWednesday, October 31, 12
  35. 35. SERVER-ADRESSE from batou.component import Component from batou.lib.buildout import Buildout from batou.utils import Address class Supervisor(Component): port = 9000 def configure(self): self.address = Address( self.host.fqdn, self.port) self += Buildout(supervisor, python=2.7) components/supervisor/component.pyWednesday, October 31, 12
  36. 36. NETZWERKADRESSEN a = Address(localhost:631) a = Address(localhost, 631) str(a.connect) == localhost:631 a.connect.host == localhost a.connect.port == 631 str(a.listen) == 127.0.0.1:631 a.listen.host == 127.0.0.1 a.listen.port == 631Wednesday, October 31, 12
  37. 37. SERVER-ADRESSE from batou.component import Component from batou.lib.buildout import Buildout from batou.utils import Address class Supervisor(Component): port = 9000 def configure(self): self.address = Address( self.host.fqdn, self.port) self += Buildout(supervisor, python=2.7) components/supervisor/component.pyWednesday, October 31, 12
  38. 38. SERVER-ADRESSE [hosts] localhost = program, supervisor [component:supervisor] port = 6000 environments/dev.cfgWednesday, October 31, 12
  39. 39. WELCHE PROGRAMME STARTEN? from batou.component import Component from batou.lib.file import File class Hello(Component): def configure(self): self += File(hello, source=hello.py, mode=0o755) self.provide(program, {}/hello.format(self.workdir)) components/hello/component.pyWednesday, October 31, 12
  40. 40. WELCHE PROGRAMME STARTEN? class Supervisor(Component): def configure(self): self.programs = self.require( w program, host=self.host) self += Buildout(supervisor, python=2.7) components/supervisor/component.pyWednesday, October 31, 12
  41. 41. WELCHE PROGRAMME STARTEN? [buildout] parts = supervisor [supervisor] recipe = collective.recipe.supervisor port = {{component.address.listen}} serverurl = ... programs = {%- for program in component.programs %} {{loop.index0}} prog{{loop.index}} {{program}} true {% endfor -%} components/supervisor/buildout.cfgWednesday, October 31, 12
  42. 42. RESSOURCEN UND ABHÄNGIGKEITEN • provide(key, value) • require(key, host=None) => [x, y, z] • require_one(key, host=None) => xWednesday, October 31, 12
  43. 43. RESSOURCEN UND ABHÄNGIGKEITEN • provide ohne require • require ohne provide • require_one mit != 1 Element • Flapping durch Sortieren vermeiden. • Dependencies werden automatisch gefundenWednesday, October 31, 12
  44. 44. HOOK KOMPONENTEN from batou.component import Component from batou.lib.file import File class Hello(Component): def configure(self): self += File(hello, source=hello.py, mode=0o755) self += Program(hello, command=hello) components/hello/component.pyWednesday, October 31, 12
  45. 45. class Program(HookComponent): namevar = name key = supervisor:Program command = None program_template = {name} {command} true def configure(self): super(Program, self).configure() self.command = os.path.normpath( os.path.join(self.workdir, self.command)) def format(self, supervisor): return self.program_template.format( name=self.name, command=self.command)Wednesday, October 31, 12
  46. 46. HOOK COMPONENTS class Program(HookComponent): key = supervisor:Program Kann eigentlich nicht ... hier leben - muss importierbar sein! class Supervisor(Component): def configure(self): self.programs = self.require( Program.key, host=self.host) self += Buildout(supervisor, python=2.7) components/supervisor/component.pyWednesday, October 31, 12
  47. 47. WELCHE PROGRAMME STARTEN[buildout]parts = supervisor[supervisor]recipe = collective.recipe.supervisorport = {{component.address.listen}}serverurl = ...programs ={%- for program in component.programs %} {{loop.index0}} {{program.format}}{% endfor -%} components/supervisor/buildout.cfgWednesday, October 31, 12
  48. 48. SUPERVISOR STARTEN class Supervisor(Component): def configure(self): ... def verify(self): self.assert_file_is_current( var/supervisord.pid, [.batou.buildout.success]) self.assert_no_subcomponent_changes() def update(self): out, err = self.cmd(bin/supervisorctl pid) try: int(out) except ValueError: self.cmd(bin/supervisord) else: self.cmd(bin/supervisorctl reload)Wednesday, October 31, 12
  49. 49. @REBOOT class Service(Component): namevar = executable Nur ein "bag". pidfile = None class Supervisor(Component): def configure(self): ... self += Service(bin/supervisord, pidfile=var/supervisord.pid)Wednesday, October 31, 12
  50. 50. PLATTFORMEN@platform(flyingcircus.io, Service)class UserInit(Component): def configure(self): target = /var/spool/init.d/{}/{}.format( self.environment.service_user, basename(self.parent.executable)) self += File(target, source=init.sh, is_template=True, mode=0o755, leading=True)Wednesday, October 31, 12
  51. 51. PLATTFORMEN [environment] host_domain = flyingcircus.io platform = flyingcircus.io $ bin/batou-local --platform= production test01 # keine Plattform!Wednesday, October 31, 12
  52. 52. REVIEW • File taucht häufig auf • Address • Jinja2-Templates • Environment overrides • Ressourcen + HookComponents • verify/update-Cycle • Plattform-SpezifikaWednesday, October 31, 12
  53. 53. TIPPS • Komponenten unabhängig von Host • Gemeinsamkeiten in Komponenten bündeln • verify/update so kurz wie nur möglich • je abstrakter eine Komponente desto unwarscheinlicher trifft man verify/updateWednesday, October 31, 12
  54. 54. VORDEFINIERTE KOMPONENTEN File VirtualEnv Buildout zip Secrets Supervisor Download CMMI git svn tar cronWednesday, October 31, 12
  55. 55. FAZIT • modell-getrieben aber pragmatisch • Konvergenz und Idempotenz • Der Teufel liegt im Detail. Immer wieder.Wednesday, October 31, 12
  56. 56. AUSSICHT • Dokumentation • Bessere Fehlermeldungen • Besserer Status-Output • Alternativen zu SSH/HGWednesday, October 31, 12
  57. 57. http://mobro.co/theuniWednesday, October 31, 12
  58. 58. FRAGEN? • http://bitbucket.org/gocept/batou • http://packages.python.org/batou • Bitte gebt mir Bierdeckelfeedback! • http://mobro.co/theuniWednesday, October 31, 12
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×