Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
www.immobilienscout24.deConfiguration Managementand Linux PackagesOSDC Nürnberg | 17.04.2013 | Schlomo Schapiro | @schlomo...
www.ImmobilienScout24.de >2 billion PI per month 2 data center with ~1400 VM total of ~600 employees  ~30 crossfuncti...
Why am I standing here? My Puppet and Chef to only half the job blog posting (2012-07) Packages Doing Too Much? blog pos...
Business Decision To go ...
DevOpsSlide 5 | Config Management & Linux Packages | @schlomoschapiro
Never change a running system Run the changing system Continous Delivery ...
Configuration Build DEV Application YUM ...
“Any relevant file should be either deployed via a package or co...
„Every package must be verifyable – stay away from package scripts.“ “Reducing the config package reduces the ...
Config SVNInfrastructure as post-commit ...
Configuration over Convention TSTWEB05 Location & Instance Environment ...
Big Picture – Static Structureconfig├── host Post-commit hook cre...
Static Structure – Pros and Cons Variables follow same config ├── host...
Example: Apache HTTPDhttpd RPM /etc/httpd/conf/httpd.conf /etc/sysconfig/httpdDesign Goals: Use and extend upstream ...
Example: is24-httpd RPMRequires: httpdRequires(Pre): httpd%postif ! echo # IS24 HTTPD conf framework. Read IS24_README!# T...
Example: is24-httpd RPM%post (continued)if ! echo # HTTPD options can be configured in# additional /etc/httpd/conf/env/*.s...
Example: How To Upgrade Java PackagesRequires:is24-jdk-6is24-jdk-7 Req is24...
Separation of Concerns Keep Use It ...
http://yadt-project.org and http://github.com/YADThttp://github.com/ImmobilienScout24http://bit.ly/is24tech...
Thank you very much!Please contact me for furtherquestions and discussions.Kontakt:Immobilien Scout GmbH Fon: +49 30 2...
Photo URLsRowboot: http://www.photoreview.com.au/features/profiles/work-hard-get-lucky.aspxHolding hands: http://www.thech...
Upcoming SlideShare
Loading in …5
×

OSDC 2013 - Configuration Management and Linux Packages

1,709 views

Published on

What is the connection between configuration management (CM) and Linux packages? Is there a connection? Why do the Linux packages get in the way of CM all the time? Why should I care about this topic?

In the modern world everybody has some tooling for CM, be it one of chef/puppet/cfengine or something else. All CM tools basically serve the same purpose: Automate everything between a blank new machine and a running system that is ready for production. That includes installing some packages and changing configuration files.

Not really surprising, Linux packages actually serve a very similar purpose, though with a completely different objective. Packages also install other packages and also bring some configuration files.

That is the reason why we should talk about CM and packages: Two different tool sets doing the same job with different objectives. That is also the reason why CM was invented and why distro packages tend to get in the way of what you are trying to achieve with CM. Look at sysadvent.blogspot.de/2012/12/day-10-packages-doing-too-much.html for a good example of this conflict of interests.
When faced with the challenge of finding a new deployment and configuration management solution, we decided to try something different and radically new: Use Linux packages for configuration management!

In this talk I will explain the rationale behind that decision and the design choices that allowed us to do this. We believe that this is a way out of the CM-packages conflict because we actually use the same tooling for everything: OS deployment, software rollout and configuration rollout. In our world there cannot be a conflict between distro packages and configuration because we actively design our packages and configuration to work seamlessly together with the distro packages.
If the time permits I will be happy to share a live demo of how we work with configuration packages.
The result of our work is published under the GPL at yadt-project.org, our tooling to create config RPMs from configuration data snippets kept in SVN can be found in https://github.com/yadt/yadt-config-rpm-maker. Take this as an example, the important part is package-based configuration rollout. It doesn't matter how you actually create those packages, it is only important that they are dumb and contain no install-time scripts.

Published in: Technology
License: CC Attribution-NonCommercial-NoDerivs License
no profile picture user

  • Be the first to comment

OSDC 2013 - Configuration Management and Linux Packages

  1. 1. www.immobilienscout24.deConfiguration Managementand Linux PackagesOSDC Nürnberg | 17.04.2013 | Schlomo Schapiro | @schlomoschapiroSystemarchitekt, Open Source Evangelist License: http://creativecommons.org/licenses/by-nc-nd/3.0/
  2. 2. www.ImmobilienScout24.de >2 billion PI per month 2 data center with ~1400 VM total of ~600 employees  ~30 crossfunctional IT teams  ~160 in IT 15 years in business part of Deutsche TelekomSlide 2 | Config Management & Linux Packages | @schlomoschapiro
  3. 3. Why am I standing here? My Puppet and Chef to only half the job blog posting (2012-07) Packages Doing Too Much? blog post on SysAdvent (2012-12) Talking to lots of people at lots of conferences One Tool To Rule Them AllSlide 3 | Config Management & Linux Packages | @schlomoschapiro
  4. 4. Business Decision To go Live All Build Scale Out Humans Config Data Centers are on the Deploy Automation Same Side Test Systems- Management through packagesSlide 4 | Config Management & Linux Packages | @schlomoschapiro
  5. 5. DevOpsSlide 5 | Config Management & Linux Packages | @schlomoschapiro
  6. 6. Never change a running system Run the changing system Continous Delivery Deploy When ReadyYou Build It – You Run It! Fail Fast – Fail Early Run With The Pack – The Pack Will Protect YouSlide 6 | Config Management & Linux Packages | @schlomoschapiro
  7. 7. Configuration Build DEV Application YUM QA Build ReposInfrastructure PRO Build InterfaceSlide 7 | Config Management & Linux Packages | @schlomoschapiro
  8. 8. “Any relevant file should be either deployed via a package or completely managed by an application that is thus deployed.”Slide 8 | Config Management & Linux Packages | @schlomoschapiro
  9. 9. „Every package must be verifyable – stay away from package scripts.“ “Reducing the config package reduces the deployment risk.”Slide 9 | Config Management & Linux Packages | @schlomoschapiro
  10. 10. Config SVNInfrastructure as post-commit → YUM repos Code Slide 10 | Config Management & Linux Packages | @schlomoschapiro
  11. 11. Configuration over Convention TSTWEB05 Location & Instance Environment Function GroupSlide 11 | Config Management & Linux Packages | @schlomoschapiro
  12. 12. Big Picture – Static Structureconfig├── host Post-commit hook creates│ └── tstweb02 is24-config-$hostname RPM│ ● svn export├── loc svn ci│ └── tst Config ● patch VARIABLES fill in metadata SVN ●│ ├── VARIABLES svn co│ │ ├── RPM_REQUIRES│ │ ├── RPM_PROVIDES│ │ ├── DB_HOST│ │ └── DB_USER│ └── etc YUM IS24 software│ └── is24│ ├── web.properties Repository Automated RPM creation│ └── db.properties├── typ│ └── web db.host=@@@DB_HOST@@@│ db.user=@@@DB_USER@@@ is24-config-tstweb01-1.0-$rev.rpm:├── loctyp db.port=3306 /etc/is24/system.properties│ └── tstweb│ └── etc /etc/is24/db.properties│ └── is24 /etc/is24/web.properties│ └── web.properties└── all ├── VARIABLES │ └── SYSLOG_HOST └── etc └── is24 loghost=log.domain.com [root@tstweb01 ~]# yum update └── system.properties Slide 12 | Config Management & Linux Packages | @schlomoschapiro
  13. 13. Static Structure – Pros and Cons Variables follow same config ├── host Post-commit hook creates structure as config data │ └── tstweb02 is24-config-$hostname RPM │ ● svn export ├── loc svn ci │ └── tst Config ● patch VARIABLES fill in metadata SVN ● │ ├── VARIABLES Only one tool (SVN) needed svn co │ │ ├── RPM_REQUIRES │ │ ├── RPM_PROVIDES │ │ ├── DB_HOST │ │ └── DB_USER GUI talks only to SVN │ │ │ └── etc └── is24 ├── web.properties YUM Repository IS24 software Automated RPM creation │ └── db.properties Versioning and change ├── typ │ │ └── web db.host=@@@DB_HOST@@@ db.user=@@@DB_USER@@@ is24-config-tstweb01-1.0-$rev.rpm: tracking for variables ├── loctyp db.port=3306 /etc/is24/system.properties │ └── tstweb │ └── etc /etc/is24/db.properties │ └── is24 /etc/is24/web.properties Variables and config files │ └── web.properties └── all ├── VARIABLES change together (atomic) │ └── SYSLOG_HOST └── etc └── is24 loghost=log.domain.com [root@tstweb01 ~]# yum update → easy rollback └── system.properties Is this simple overlaying structure sufficient? KIS  Simplify the world so that it is good enough! S!Slide 13 | Config Management & Linux Packages | @schlomoschapiro
  14. 14. Example: Apache HTTPDhttpd RPM /etc/httpd/conf/httpd.conf /etc/sysconfig/httpdDesign Goals: Use and extend upstream httpd RPM Configure MPM and service user per application IS24 standard configuration everywhereSlide 14 | Config Management & Linux Packages | @schlomoschapiro
  15. 15. Example: is24-httpd RPMRequires: httpdRequires(Pre): httpd%postif ! echo # IS24 HTTPD conf framework. Read IS24_README!# This file is managed by %{name}­%{version}.%{release}# Put your stuff in /etc/conf/*/is24*.conf files!ServerRoot "/etc/httpd"Include conf/basic/is24*.confInclude conf/main/is24*.confInclude conf/other/is24*.conf >/etc/httpd/conf/httpd.conf ; then    logger ­p user.err ­s ­t %name ­­ "ERROR: …"fiSlide 15 | Config Management & Linux Packages | @schlomoschapiro
  16. 16. Example: is24-httpd RPM%post (continued)if ! echo # HTTPD options can be configured in# additional /etc/httpd/conf/env/*.sh files# This file is managed by %{name}­%{version}.%{release}# Please add extra options to the OPTIONS Bash Arrayshopt ­s nullglobHTTPD=/usr/sbin/httpd.workerfor f in /etc/httpd/conf/env/*.sh ; do    source $fdoneOPTIONS="${OPTIONS[*]}" # flatten array >/etc/sysconfig/httpd ; then    logger ­p user.err ­s ­t %name ­­ "ERROR: ..."fi...Slide 16 | Config Management & Linux Packages | @schlomoschapiro
  17. 17. Example: How To Upgrade Java PackagesRequires:is24-jdk-6is24-jdk-7 Req is24-jdk-7 is24-jdk-6 /etc/profile.d/java.sh Req Req java-1.7.0-oracle-devel java-1.6.0-sun-develSlide 17 | Config Management & Linux Packages | @schlomoschapiro
  18. 18. Separation of Concerns Keep Use It What Simple Is & Already Stupid There The Tool Is You!Slide 18 | Config Management & Linux Packages | @schlomoschapiro
  19. 19. http://yadt-project.org and http://github.com/YADThttp://github.com/ImmobilienScout24http://bit.ly/is24techjobs Slide 19 | Config Management & Linux Packages | @schlomoschapiro
  20. 20. Thank you very much!Please contact me for furtherquestions and discussions.Kontakt:Immobilien Scout GmbH Fon: +49 30 243 01-1229Andreasstraße 10 Email: schlomo.schapiro@immobilienscout24.de10243 Berlin URL: www.immobilienscout24.deSlide 20 | Config Management & Linux Packages | @schlomoschapiro
  21. 21. Photo URLsRowboot: http://www.photoreview.com.au/features/profiles/work-hard-get-lucky.aspxHolding hands: http://www.thechefalliance.com/Top-Chef-ServicesRobots: http://www.t-tek.com/announcements/conventional-palletizer-or-robotEverything else is from OpenClipArt.org or custom made.Slide 21 | Practical DevOps | @schlomoschapiro

×