• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Subversion Day Berlin 2011 Configuration Management With Subversion And Rpm
 

Subversion Day Berlin 2011 Configuration Management With Subversion And Rpm

on

  • 1,930 views

ImmobilienScout24 makes use of SVN as a structured hierarchical configuration ...

ImmobilienScout24 makes use of SVN as a structured hierarchical configuration
management backend that serves configuration files for development, testing and
production platforms. Configuration files are automatically deployed to target systems via
RPMs, which are automatically built from the information contained in a SVN repository.
The components that comprise the solution (mostly a sophisticated post-commit hook) are
open sourced under the GPL.

See http://code.google.com/p/yadt for open source project homepage.

Statistics

Views

Total Views
1,930
Views on SlideShare
1,859
Embed Views
71

Actions

Likes
0
Downloads
15
Comments
0

4 Embeds 71

http://localhost 24
http://yadt.github.com 22
http://www.scoop.it 13
http://www.yadt-project.org 12

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Subversion Day Berlin 2011 Configuration Management With Subversion And Rpm Subversion Day Berlin 2011 Configuration Management With Subversion And Rpm Presentation Transcript

    • www.immobilienscout24.deConfiguration Managementwith Subversion and RPMThe Linux Way of Doing ThingsBerlin | 19.05.2011 | Schlomo SchapiroSystemarchitekt, Open Source Evangelist License: http://creativecommons.org/licenses/by-nc-nd/3.0/
    • http://code.google.com/p/yadt/Agenda www.immobilienscout24.de: Behind the scenes The Linux Way of Doing Things RPM Deployment Configuration Management in Subversion YADT – an Augmented Deployment ToolSlide 2 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/www.immobilienscout24.de 1.7 billion page impressions per month http://www.immobilienscout24.de/de/anbieten/kennzahlen.jsp Linux and standard open source toolchain  RHEL  LAMP server  Tomcat, Spring, Hibernate …  MySQL and Oracle 2 data centres with async WAN, >600 VM, SAN, NAS … 24/7 operations Agile development, now going toward „Continuous Delivery“ Automate, Automate, AutomateSlide 3 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/Slide 4 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/What problem do we solve? Standalone Web Stand PHP App alone Syslog SNMP Container PHP/Perl Java VM Apache MySQL Postfix Linux OS Physical/Virtual HW Virtualization Layer Manage entire platform above (virtual) hardware layerSlide 5 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/The UNIX Way of Thinking Web WebOn the Linux level PHP/Perl Web Tomcat App Apache Java VM If we look very closely MySQL Syslog Net-SNMP Standalone SSH Postfix And think UNIX Linux OSEverything is just a file! physical/virtual HW Programs Environmental configuration Libraries Even Startup dependencies Software configuration And also what to run and when Operational configuration Reboot is the final test!Slide 6 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/Linux tools we like to use Software Management:  RPM Packages  Versioning & Transactions  Validation and Rollback  Deploy & run actions: post-install, pre-remove, pre-install …  YUM Repositories  Proven to work with millions of systems  Release Upgrades, Repo Mirrors, Downgrades, Provisioning … Bash & Python scripts:  /etc/init.d scripts for service management  YADT for multi-server system management  Automate, Automate, Automate :-)Slide 7 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/Configuration = Software Same tools usable for software and configuration One tool to rule them all  Software & Configuration  Dependencies between software and configuration RPM file conflicts → enforce separation of concerns Doing this with a 100% coverage is probably a new idea!Slide 8 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/Configuration Management in SVN Config SVN post-commit YUM reposSlide 9 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/ Big Picture – Static Structureconfig├── host Post-commit hook creates│ └── tstweb02 is24-config-$hostname RPM│ ● svn export├── loc svn ci│ └── tst Config ● patch VARIABLES fill in metadata SVN ●│ ├── VARIABLES svn co│ │ ├── RPM_REQUIRES│ │ ├── RPM_PROVIDES│ │ ├── DB_HOST│ │ └── DB_USER│ └── etc YUM IS24 software│ └── is24│ ├── web.properties Repository Automated RPM creation│ └── db.properties├── typ│ └── web db.host=@@@DB_HOST@@@│ db.user=@@@DB_USER@@@ is24-config-tstweb01-1.0-$rev.rpm:├── loctyp db.port=3306 /etc/is24/system.properties│ └── tstweb│ └── etc /etc/is24/db.properties│ └── is24 /etc/is24/web.properties│ └── web.properties└── all ├── VARIABLES │ └── SYSLOG_HOST └── etc └── is24 loghost=log.domain.com [root@tstweb01 ~]# yum update └── system.properties Slide 10 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/Static Structure – Pros and Cons config ├── host Post-commit hook creates │ └── tstweb02 is24-config-$hostname RPM │ ● svn export ├── loc svn ci │ └── tst Config ● patch VARIABLES fill in metadata SVN ● │ ├── VARIABLES svn co │ │ ├── RPM_REQUIRES │ │ ├── RPM_PROVIDES │ │ ├── DB_HOST │ │ └── DB_USER │ └── etc YUM IS24 software Variables follow same │ └── is24 Repository Automated RPM │ ├── web.properties creation │ └── db.properties ├── typ structure as config data │ │ └── web ├── loctyp db.host=@@@DB_HOST@@@ db.user=@@@DB_USER@@@ is24-config-tstweb01-1.0-$rev.rpm: │ └── tstweb db.port=3306 /etc/is24/system.properties Only one tool (SVN) needed │ └── etc /etc/is24/db.properties │ └── is24 /etc/is24/web.properties │ └── web.properties └── all GUI talks only to SVN ├── VARIABLES │ └── SYSLOG_HOST └── etc loghost=log.domain.com [root@tstweb01 ~]# yum update └── is24 Versioning and change └── system.properties tracking for variables Variables and config files change together (atomic) → easy rollback Is this simple overlaying structure sufficient? KIS  Simplify the world so that it is good enough! S!Slide 11 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/ Big Picture – Group Assignment meta ├── hosts │ ├── ttestserver-263 │ │ ├── SPECIAL_KEY=value │ │ └── memberOf=apache │ └── tstweb02config │ ├── memberOf=webapp├── all │ └── memberOf=tst│ └── etc └── groups│ └── is24 ├── webapp│ └── system.properties │ ├── member=tstweb02├── groups │ ├── memberOf=tomcat│ ├── tst │ └── RPM_REQUIRES=is24-webapp-env, is24-jcr│ │ └── etc └── tst│ │ └── is24 ├── member=tstweb02│ │ ├── web.properties ├── DB_USER=tuv-db-user│ │ └── db.properties └── DB_HOST=dbserver-20│ ├── apache db.host=@@@DB_HOST@@@│ │ └── etc db.user=@@@DB_USER@@@ db.port=3306│ │ └── is24 svn ci│ │ └── httpd.conf│ └── webapp│ └── etc│ └── is24 svn ci│ └── web.properties Config YUM IS24 software SVN└── host Post-commit Repository Automated RPM hook creation ├── testserver-263 creates └── tstweb02 is24-config-$hostname RPM Slide 12 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/Group Assignment – Pros and Cons meta ├── hosts │ ├── srv001 │ │ ├── SPECIAL_KEY=value │ │ └── memberOf=apache │ └── testserver-263 config │ ├── memberOf=webapp Variables follow same ├── all │ └── memberOf=tuv │ └── etc └── groups │ └── is24 ├── webapp │ ├── member=testserver-263 structure as config data │ └── system.properties ├── groups │ ├── memberOf=tomcat │ ├── tuv │ └── RPM_REQUIRES=is24-webapp-env, is24-jcr │ │ └── etc └── tuv │ │ └── is24 ├── member=testserver-263 Only one tool (SVN) needed │ │ │ │ │ ├── apache ├── web.properties └── db.properties ├── DB_USER=tuv-db-user └── DB_HOST=dbserver-20 db.host=@@@DB_HOST@@@ │ │ └── etc db.user=@@@DB_USER@@@ GUI talks only to SVN db.port=3306 │ │ └── is24 svn ci │ │ └── httpd.conf │ └── webapp │ └── etc Versioning and change │ │ └── host └── is24 └── web.properties svn ci Config SVN Post-commit YUM Repository IS24 software Automated RPM tracking for variables hook creation ├── testserver-263 creates └── tstweb02 is24-config-$hostname RPM Variables and config files change together (atomic) → easy rollback Free assignment of hosts to groups, nested groups Groups define function and pull in configuration and software Final result is combination of all groups (in nesting order) Not any more so much KEEP IT SIMPLESlide 13 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/ System Personalisation: RPM Dependencies is24-config-tstweb02 Requires httpd is24-mod_jk is24-tomcat is24-base-env is24-appl-env is24-tomcat-data is24-jcris24-contentrules is24-jcr-base is24-sitemapassembler is24-jtools is24-sitemapassembler-config jdk Slide 14 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/YADT – an Augmented Deployment Tool Meta Control Layer Open Source – GNU General Public License 100% Python / Bash Local file management through RPM Global service managent through YADT Model datacenter dependencies:  RPM packages  Servers and services  Between services on different servers Stateless design – current status is the state, define target states All actions restartable by following the dependency model SVN → RPM post-commit hook: Maintained as part of YADTSlide 15 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/Immobilien Scout GmbH Operate Germanys Leading Real Estate Listing Portal: www.immobilienscout24.de Linux, XEN, Apache, Tomcat, Java, LAMP, Oracle, MySQL, Nagios, DRBD, RPM Open Source & DevOps Company Always seeking to hire new skill:  Admins  System Developers http://bit.ly/is24greatplace  System Engineers  Architects  Java & PHP Developers  Mobile (iOS & Android) DevelopersSlide 16 | Configuration Management with SVN and RPM | Schlomo Schapiro
    • http://code.google.com/p/yadt/Thank you very much!Please contact me for furtherquestions and discussions.Kontakt:Immobilien Scout GmbH Fon: +49 30 243 01-1229Andreasstraße 10 Email: schlomo.schapiro@immobilienscout24.de10243 Berlin URL: www.immobilienscout24.deSlide 17 | Configuration Management with SVN and RPM | Schlomo Schapiro