Download as PDF, PPTX
![http://code.google.com/p/yadt/
Big Picture – Static Structure
config
├── host Post-commit hook creates
│ └── tstweb02 is24-config-$hostname RPM
│ ●
svn export
├── loc svn ci
│ └── tst Config ●
patch VARIABLES
fill in metadata
SVN
●
│ ├── VARIABLES svn co
│ │ ├── RPM_REQUIRES
│ │ ├── RPM_PROVIDES
│ │ ├── DB_HOST
│ │ └── DB_USER
│ └── etc YUM IS24 software
│ └── is24
│ ├── web.properties Repository Automated RPM
creation
│ └── db.properties
├── typ
│ └── web db.host=@@@DB_HOST@@@
│ db.user=@@@DB_USER@@@ is24-config-tstweb01-1.0-$rev.rpm:
├── loctyp db.port=3306 /etc/is24/system.properties
│ └── tstweb
│ └── etc /etc/is24/db.properties
│ └── is24 /etc/is24/web.properties
│ └── web.properties
└── all
├── VARIABLES
│ └── SYSLOG_HOST
└── etc
└── is24
loghost=log.domain.com [root@tstweb01 ~]# yum update
└── system.properties
Slide 10 | Configuration Management with SVN and RPM | Schlomo Schapiro](https://image.slidesharecdn.com/subversiondayberlin2011-configurationmanagementwithsubversionandrpm-111204153013-phpapp02/85/Subversion-Day-Berlin-2011-Configuration-Management-With-Subversion-And-Rpm-10-320.jpg)
![http://code.google.com/p/yadt/
Static Structure – Pros and Cons
config
├── host Post-commit hook creates
│ └── tstweb02 is24-config-$hostname RPM
│ ●
svn export
├── loc svn ci
│ └── tst Config ●
patch VARIABLES
fill in metadata
SVN
●
│ ├── VARIABLES svn co
│ │ ├── RPM_REQUIRES
│ │ ├── RPM_PROVIDES
│ │ ├── DB_HOST
│ │ └── DB_USER
│ └── etc YUM IS24 software
Variables follow same
│ └── is24
Repository Automated RPM
│ ├── web.properties
creation
│ └── db.properties
├── typ
structure as config data │
│
└── web
├── loctyp
db.host=@@@DB_HOST@@@
db.user=@@@DB_USER@@@ is24-config-tstweb01-1.0-$rev.rpm:
│ └── tstweb
db.port=3306 /etc/is24/system.properties
Only one tool (SVN) needed │ └── etc /etc/is24/db.properties
│ └── is24 /etc/is24/web.properties
│ └── web.properties
└── all
GUI talks only to SVN ├── VARIABLES
│ └── SYSLOG_HOST
└── etc
loghost=log.domain.com [root@tstweb01 ~]# yum update
└── is24
Versioning and change
└── system.properties
tracking for variables
Variables and config files change together (atomic) → easy rollback
Is this simple overlaying structure sufficient? KIS
Simplify the world so that it is good enough! S!
Slide 11 | Configuration Management with SVN and RPM | Schlomo Schapiro](https://image.slidesharecdn.com/subversiondayberlin2011-configurationmanagementwithsubversionandrpm-111204153013-phpapp02/85/Subversion-Day-Berlin-2011-Configuration-Management-With-Subversion-And-Rpm-11-320.jpg)
Uploaded bySchlomo Schapiro
Subversion Day Berlin 2011 Configuration Management With Subversion And Rpm
The document discusses configuration management practices at www.immobilienscout24.de, focusing on the use of Subversion (svn) and RPM for software deployment and management within a Linux environment. It emphasizes automation, continuous delivery, and the integration of software and configuration management to ensure efficient platform management across their extensive data centers. Additionally, it introduces YADT, an open-source tool designed to manage and simplify deployment processes in a multi-server context.
More Related Content
Similar to Subversion Day Berlin 2011 Configuration Management With Subversion And Rpm
Subversion Day Berlin 2011 Configuration Management With Subversion And Rpm
- 1. www.immobilienscout24.de Configuration Management with Subversionand RPM The Linux Way of Doing Things Berlin | 19.05.2011 | Schlomo Schapiro Systemarchitekt, Open Source Evangelist License: http://creativecommons.org/licenses/by-nc-nd/3.0/
- 2. http://code.google.com/p/yadt/ Agenda www.immobilienscout24.de:Behind the scenes The Linux Way of Doing Things RPM Deployment Configuration Management in Subversion YADT – an Augmented Deployment Tool Slide 2 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 3. http://code.google.com/p/yadt/ www.immobilienscout24.de 1.7billion page impressions per month http://www.immobilienscout24.de/de/anbieten/kennzahlen.jsp Linux and standard open source toolchain RHEL LAMP server Tomcat, Spring, Hibernate … MySQL and Oracle 2 data centres with async WAN, >600 VM, SAN, NAS … 24/7 operations Agile development, now going toward „Continuous Delivery“ Automate, Automate, Automate Slide 3 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 4. http://code.google.com/p/yadt/ Slide 4 |Configuration Management with SVN and RPM | Schlomo Schapiro
- 5. http://code.google.com/p/yadt/ What problem dowe solve? Standalone Web Stand PHP App alone Syslog SNMP Container PHP/Perl Java VM Apache MySQL Postfix Linux OS Physical/Virtual HW Virtualization Layer Manage entire platform above (virtual) hardware layer Slide 5 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 6. http://code.google.com/p/yadt/ The UNIX Wayof Thinking Web Web On the Linux level PHP/Perl Web Tomcat App Apache Java VM If we look very closely MySQL Syslog Net-SNMP Standalone SSH Postfix And think UNIX Linux OS Everything is just a file! physical/virtual HW Programs Environmental configuration Libraries Even Startup dependencies Software configuration And also what to run and when Operational configuration Reboot is the final test! Slide 6 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 7. http://code.google.com/p/yadt/ Linux tools welike to use Software Management: RPM Packages Versioning & Transactions Validation and Rollback Deploy & run actions: post-install, pre-remove, pre-install … YUM Repositories Proven to work with millions of systems Release Upgrades, Repo Mirrors, Downgrades, Provisioning … Bash & Python scripts: /etc/init.d scripts for service management YADT for multi-server system management Automate, Automate, Automate :-) Slide 7 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 8. http://code.google.com/p/yadt/ Configuration = Software Same tools usable for software and configuration One tool to rule them all Software & Configuration Dependencies between software and configuration RPM file conflicts → enforce separation of concerns Doing this with a 100% coverage is probably a new idea! Slide 8 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 9. http://code.google.com/p/yadt/ Configuration Management inSVN Config SVN post-commit YUM repos Slide 9 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 10. http://code.google.com/p/yadt/ BigPicture – Static Structure config ├── host Post-commit hook creates │ └── tstweb02 is24-config-$hostname RPM │ ● svn export ├── loc svn ci │ └── tst Config ● patch VARIABLES fill in metadata SVN ● │ ├── VARIABLES svn co │ │ ├── RPM_REQUIRES │ │ ├── RPM_PROVIDES │ │ ├── DB_HOST │ │ └── DB_USER │ └── etc YUM IS24 software │ └── is24 │ ├── web.properties Repository Automated RPM creation │ └── db.properties ├── typ │ └── web db.host=@@@DB_HOST@@@ │ db.user=@@@DB_USER@@@ is24-config-tstweb01-1.0-$rev.rpm: ├── loctyp db.port=3306 /etc/is24/system.properties │ └── tstweb │ └── etc /etc/is24/db.properties │ └── is24 /etc/is24/web.properties │ └── web.properties └── all ├── VARIABLES │ └── SYSLOG_HOST └── etc └── is24 loghost=log.domain.com [root@tstweb01 ~]# yum update └── system.properties Slide 10 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 11. http://code.google.com/p/yadt/ Static Structure –Pros and Cons config ├── host Post-commit hook creates │ └── tstweb02 is24-config-$hostname RPM │ ● svn export ├── loc svn ci │ └── tst Config ● patch VARIABLES fill in metadata SVN ● │ ├── VARIABLES svn co │ │ ├── RPM_REQUIRES │ │ ├── RPM_PROVIDES │ │ ├── DB_HOST │ │ └── DB_USER │ └── etc YUM IS24 software Variables follow same │ └── is24 Repository Automated RPM │ ├── web.properties creation │ └── db.properties ├── typ structure as config data │ │ └── web ├── loctyp db.host=@@@DB_HOST@@@ db.user=@@@DB_USER@@@ is24-config-tstweb01-1.0-$rev.rpm: │ └── tstweb db.port=3306 /etc/is24/system.properties Only one tool (SVN) needed │ └── etc /etc/is24/db.properties │ └── is24 /etc/is24/web.properties │ └── web.properties └── all GUI talks only to SVN ├── VARIABLES │ └── SYSLOG_HOST └── etc loghost=log.domain.com [root@tstweb01 ~]# yum update └── is24 Versioning and change └── system.properties tracking for variables Variables and config files change together (atomic) → easy rollback Is this simple overlaying structure sufficient? KIS Simplify the world so that it is good enough! S! Slide 11 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 12. http://code.google.com/p/yadt/ Big Picture – Group Assignment meta ├── hosts │ ├── ttestserver-263 │ │ ├── SPECIAL_KEY=value │ │ └── memberOf=apache │ └── tstweb02 config │ ├── memberOf=webapp ├── all │ └── memberOf=tst │ └── etc └── groups │ └── is24 ├── webapp │ └── system.properties │ ├── member=tstweb02 ├── groups │ ├── memberOf=tomcat │ ├── tst │ └── RPM_REQUIRES=is24-webapp-env, is24-jcr │ │ └── etc └── tst │ │ └── is24 ├── member=tstweb02 │ │ ├── web.properties ├── DB_USER=tuv-db-user │ │ └── db.properties └── DB_HOST=dbserver-20 │ ├── apache db.host=@@@DB_HOST@@@ │ │ └── etc db.user=@@@DB_USER@@@ db.port=3306 │ │ └── is24 svn ci │ │ └── httpd.conf │ └── webapp │ └── etc │ └── is24 svn ci │ └── web.properties Config YUM IS24 software SVN └── host Post-commit Repository Automated RPM hook creation ├── testserver-263 creates └── tstweb02 is24-config-$hostname RPM Slide 12 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 13. http://code.google.com/p/yadt/ Group Assignment –Pros and Cons meta ├── hosts │ ├── srv001 │ │ ├── SPECIAL_KEY=value │ │ └── memberOf=apache │ └── testserver-263 config │ ├── memberOf=webapp Variables follow same ├── all │ └── memberOf=tuv │ └── etc └── groups │ └── is24 ├── webapp │ ├── member=testserver-263 structure as config data │ └── system.properties ├── groups │ ├── memberOf=tomcat │ ├── tuv │ └── RPM_REQUIRES=is24-webapp-env, is24-jcr │ │ └── etc └── tuv │ │ └── is24 ├── member=testserver-263 Only one tool (SVN) needed │ │ │ │ │ ├── apache ├── web.properties └── db.properties ├── DB_USER=tuv-db-user └── DB_HOST=dbserver-20 db.host=@@@DB_HOST@@@ │ │ └── etc db.user=@@@DB_USER@@@ GUI talks only to SVN db.port=3306 │ │ └── is24 svn ci │ │ └── httpd.conf │ └── webapp │ └── etc Versioning and change │ │ └── host └── is24 └── web.properties svn ci Config SVN Post-commit YUM Repository IS24 software Automated RPM tracking for variables hook creation ├── testserver-263 creates └── tstweb02 is24-config-$hostname RPM Variables and config files change together (atomic) → easy rollback Free assignment of hosts to groups, nested groups Groups define function and pull in configuration and software Final result is combination of all groups (in nesting order) Not any more so much KEEP IT SIMPLE Slide 13 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 14. http://code.google.com/p/yadt/ SystemPersonalisation: RPM Dependencies is24-config-tstweb02 Requires httpd is24-mod_jk is24-tomcat is24-base-env is24-appl-env is24-tomcat-data is24-jcr is24-contentrules is24-jcr-base is24-sitemapassembler is24-jtools is24-sitemapassembler-config jdk Slide 14 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 15. http://code.google.com/p/yadt/ YADT – anAugmented Deployment Tool Meta Control Layer Open Source – GNU General Public License 100% Python / Bash Local file management through RPM Global service managent through YADT Model datacenter dependencies: RPM packages Servers and services Between services on different servers Stateless design – current status is the state, define target states All actions restartable by following the dependency model SVN → RPM post-commit hook: Maintained as part of YADT Slide 15 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 16. http://code.google.com/p/yadt/ Immobilien Scout GmbH Operate Germanys Leading Real Estate Listing Portal: www.immobilienscout24.de Linux, XEN, Apache, Tomcat, Java, LAMP, Oracle, MySQL, Nagios, DRBD, RPM Open Source & DevOps Company Always seeking to hire new skill: Admins System Developers http://bit.ly/is24greatplace System Engineers Architects Java & PHP Developers Mobile (iOS & Android) Developers Slide 16 | Configuration Management with SVN and RPM | Schlomo Schapiro
- 17. http://code.google.com/p/yadt/ Thank you verymuch! Please contact me for further questions and discussions. Kontakt: Immobilien Scout GmbH Fon: +49 30 243 01-1229 Andreasstraße 10 Email: schlomo.schapiro@immobilienscout24.de 10243 Berlin URL: www.immobilienscout24.de Slide 17 | Configuration Management with SVN and RPM | Schlomo Schapiro