Puppetconf2011 small

  • 1,127 views
Uploaded on

Jasper Poppe

Jasper Poppe

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,127
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
14
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. PuppetConf Portland 2011Automated deployment With Seedbank and Puppet Puppetconf 2011 by Jasper Poppe
  • 2. Introduction- Who is Jasper Poppe- Current position at the eBay Classifieds Group- eBay Classifieds Group- eBay Classifieds Group platforms Puppetconf 2011 by Jasper Poppe
  • 3. The Talk- Automated live installation of an infrastructure from scratch- Explanation and demo of Infrastucture Anywhere- Explanation of seedBank- Explanation of our Puppet Structure- Open source seedBank, Infrastructure anywhere and a selection of our Puppet Modules- There will be time for questions at the half and at the end of the talk Puppetconf 2011 by Jasper Poppe
  • 4. Infrastructure Anywhere CLouds are so 2011 Puppetconf 2011 by Jasper Poppe
  • 5. Prologue- Infrastructure from Scratch: Live Installation Puppetconf 2011 by Jasper Poppe
  • 6. Terminilogy- Overlord - An Overlord is a server which is running the core applications needed for afully automated infrastructure- Minion - The Overlord deploys automatically to those servers- Seed file - A file which provides answers to the Debian installer, seeding is the coremethod used for Debian package configuration- seedBank - This is a simple CLI tool/daemon to manage Debian and Ubuntu network in-stallations. Since it is based on seed files it will provide the cleanest installations possibleby just using the standard Debian tools. seedBank is written by me in Python.- Fabric - A deployment tool based on Python, in my own words it is BASH in Python withsome additional features to make deployment tasks easier.- Infrastructure Anywhere - Some simple Fabric scripts which are the glue between seed-Bank and Puppet, it provides the components to do fully automated deployments of acore infrastructure from scratch in a production, virtual, cloud based or any otherenvironment. Puppetconf 2011 by Jasper Poppe
  • 7. Infrastructure Anywhere fabfile.py chicken.py powerdns.py dnsmasq.py vmware.py libvirt.py virtualbox.py Puppetconf 2011 by Jasper Poppe
  • 8. Infrastructure Anywhere Debian ISO Infrastructure ISO Modified for Anywhere Overlord Bootstrap Overlay Directoryrc.local Seed File SSH Key Puppet Repository Puppetconf 2011 by Jasper Poppe
  • 9. Infrastructure Anywhere Debian ISO Overlord Infrastructure ISO Modified for Anywhere Overlord Bootstrap Overlay Directoryrc.local Seed File SSH Key Puppet Repository Minions Puppetconf 2011 by Jasper Poppe
  • 10. Infrastructure AnywhereInfrastructure AnywhereOverlord seedBank Gateway TFTP DNS DHCPPuppetmaster MinionsLocal Network Internet Puppetconf 2011 by Jasper Poppe
  • 11. InfraStructure Anywhere overlord001 a c m e . . . . Hostname Platform TLD Zone Location- Zones and locations are used in our platform base modules- Platform base modules configure about 95% of the machine- Makes it easy to override settings from a production to a QA environment while stillusing exactly the same Puppet code- For now extlookup is used Puppetconf 2011 by Jasper Poppe
  • 12. seedbankThe Cleanest way of automated Debian and Ubuntu Network Installations Possible Puppetconf 2011 by Jasper Poppe
  • 13. seedbank- Simplicity- Easy netboot file management- Automatically integrates ‘Debian non free firmware’ to the netboot image- Carefully hand crafted ready to use templated preseed files- Custom enable and disable hooks for easy integration with external tools- Support for stand alone puppet manifests which will be applied during the first boot- Configuration management database hooks- Support for templated file overlays- Makes it easy to do installations over serial consoles- Template based management of seed files- Separation between seed files and disk recipes- No NFS mounts needed, it only needs a Debian repository- No database backend, everything is stored in ‘pxelinux.cfg’ files Puppetconf 2011 by Jasper Poppe
  • 14. seedbank- seedbank_setupDownload and manage Debian/Ubuntu netboot image(s) which are required to do a PXEboot from the network- seedbankSeedbank prepares the ‘pxelinux.cfg’ files which will start the Debian installation process,when the system boots via from the network- seedbank_daemonA simple multithreaded HTTP daemon which generates seed files on the fly based oninformation which is stored in the ‘pxelinux.cfg’ filesAfter a successful installation it disables the related ‘pxelinux.cfg’ file Puppetconf 2011 by Jasper Poppe
  • 15. seedbankMinion Network Debian Installer Puppetconf 2011 by Jasper Poppe
  • 16. seedbankMinion Network Debian Installer seedBank Daemon Pxelinux.cfg Seed File Seed Template Disk Recipe Puppet Manifests Overlay Puppetconf 2011 by Jasper Poppe
  • 17. seedbank Configure DNS Configure DHCP Configure TFTP wget http://infrastructureanywhere.com/seedbank.deb sudo dpkg -i seedbank.deb sudo vi /etc/seedbank/settings.py sudo /etc/init.d/seedbank restart sudo seedbank_setup debian-squeeze-amd64 sudo seedbank -r default minion001.a.c.m.e debian-squeeze-amd64 reboot minion001.a.c.m.e Puppetconf 2011 by Jasper Poppe
  • 18. seedbankseedbank -M 00:11:22:33:44:55 -o minion -r default -m network minion001.a.c.m.e ubuntu-natty-amd64 Puppetconf 2011 by Jasper Poppe
  • 19. Questions Until NowRemember... Stupid questions do not exist Puppetconf 2011 by Jasper Poppe
  • 20. Showtime ebay Classifieds GroupDatacenter suite Amsterdam Puppetconf 2011 by Jasper Poppe
  • 21. Structuring PuppetJust Because you want to avoid the evil faces of puppet! Puppetconf 2011 by Jasper Poppe
  • 22. Structuring PuppetAfter we discovered you could use multiple module paths in Puppet I came up with a topdown structure which is using multiple module pathsIn other words- It is not allowed to include, inherit, etc. from other modules which reside in the samepath, so every module will be isolated- It is only allowed to include, inherit, override etc. from a higher level than the path youare working from Puppetconf 2011 by Jasper Poppe
  • 23. Structuring PuppetProblems I have taken in account with the design- Be able to share code between different platforms in a responsible way- Keep code as isolated as possible- Keep dependencies as clear as possible- Keep it as simple as simple as possible- Give Puppet newcomers the possibility to start developing without any direct danger Puppetconf 2011 by Jasper Poppe
  • 24. Structuring PuppetShared Puppetconf 2011 by Jasper Poppe
  • 25. Structuring PuppetSharedUsers Puppetconf 2011 by Jasper Poppe
  • 26. Structuring PuppetShared UsersPlatform Puppetconf 2011 by Jasper Poppe
  • 27. Structuring PuppetShared UsersPlatformConfigs Puppetconf 2011 by Jasper Poppe
  • 28. The SOurce Code- seedBank, Infrastruture Anywhere and a selection of our Puppet modules have just beenmade public- Everything has been released under the Apache 2 license- Currently I am the only maintainer and author of seedBank- Still lots of room for code/documentation improvements so you are welcome to submitpatches via GitHub- Two Git repositories at GitHubOne with the seedBank code (Infrastructure Anywhere is included in the addons directory)The other will contain all Puppet modules used in this talk Puppetconf 2011 by Jasper Poppe
  • 29. The source Codehttp://infrastructureanywhere.com/ Puppetconf 2011 by Jasper Poppe
  • 30. QuestionsPlease ask your questions Puppetconf 2011 by Jasper Poppe
  • 31. Thanks to ...Roalt Zijlstra - For being a mentor with his endless patience years ago while I still was alinux noobGJ Moed - For supporting me through the whole Pupppet process, and his bright ideas/discussionsDanny van der Meulen - For being Danny and his faith in me becoming a senior Linuxsystem engineerArmand Verstappen - For having the faith and giving the recognition to whatever I amdoingLex van Roon - For giving me the energy with his enthusiastic attitude, and co developingInfrastructure Anywhere Puppetconf 2011 by Jasper Poppe