GPLS/PINES GenaSYS Presentation - EG2012


Published on

Presentation at the 2012 Evergreen International Conference about GenaSYS, a program developed to automate Evergreen ILS server clusters.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

GPLS/PINES GenaSYS Presentation - EG2012

  1. 1. You Dont Have to Be a Mechanic to Drive a Car: Automating Evergreen Cluster Installation 2012 Evergreen International Conference Chris Sharp, Georgia Public Library Service Andy Witter & Josh Lamos, Emerald Data Networks
  2. 2. 2010: The PINES Bucket ProjectGoals were to identify:•code changes•configurations•any other differences between a "vanilla" Evergreen installation and what is running in PINES
  3. 3. Assumptions: You• you have a working familiarity/facility with Linux filesystem structure/navigation• youve installed Evergreen by hand before• you have the skills/knowledge to optimize Evergreen for your setup• youre interested in automating repetitive tasks
  4. 4. Assumptions: 17 Machines• lvs01 • memcache01• lvs02 • memcache02• brick01-head • reporter01• brick02-head • sip01• db01 (master database) • sip02• db02 (master hot spare) • utility01• db03 (reports queries) • admin01• logger01 • monitor01• backup01
  5. 5. Other GenaSYS Assumptions• 3 public IP addresses• a serial cable (to connect lvs machines)• possible to virtualize any machine (though DB is recommended on bare metal)
  6. 6. GenaSYS Benefits• Packaged files that can be used again to rebuild a machine on new hardware• Init scripts that ease/automate startup and shutdown• Convenience features • “attach” and “detach” bash aliases to remove a brick from rotation) • “extras” scripts (e.g., are added to the opensrf user’s search path
  7. 7. GenaSYS Defaults• Templates for OpenSRF/Evergreen configuration files – based on original versions but configured for clustered instance• Templates for Apache configuration files optimized for clustered instance• OS-level configurations for networking, NFS, logging, database, etc.
  8. 8. Code AvailabilityGenaSYS:git://git.evergreen- Packaging:git:// debian.gitLicense: GNU GPL version 3Copyright (C) Georgia Public Library Service
  9. 9. The Genesis of GenaSYSEvergreen ILS Installation Process:
  10. 10. The Genesis of GenaSYSEvergreen ILS Installation:• requires manual changes of multiple configuration files• many repetitive steps• high risk of typos/human error• lots of time required (30 – 45 minutes for experienced admins; 3 or more hours for new admins)• and thats just a single machine - what about building clusters?
  11. 11. The Genesis of GenaSYSEvergreen Server Clustering:• Linux Virtual Server (LVS)• brick heads/drones (OpenSRF, NFS, SSH keys)• utility server (OpenSRF, NFS, cron jobs)• SIP servers (OpenSRF,, cron jobs)• logger servers (client setup, server setup)• database servers (local configurations)• reports server (NFS, OpenSRF)
  12. 12. Enter GenaSYS• development began in early 2011• goal was to implement a vanilla Evergreen-ILS installation with a single script• and without manually touching configuration files• script needed to be easily modifiable to accommodate changes/updates/upgrades• builds an Evergreen 2.1.1 cluster on Debian 6.0 “squeeze”
  13. 13. GenaSYS: Two Parts1. Debian packaging of OpenSRF, Evergreen, and unpackaged dependencies (libdbi, spidermonkey, etc.)2. Cluster configuration generation: GenaSYS
  14. 14. Debian Packaging• Goal was to create easily installable software packages to replicate the manual building process.• Debs are built from Evergreen source + any customizations/changes for your setup• Not currently working on any sort of official Debian acceptance of packages
  15. 15. GenaSYS Cluster Configuration• Bash script: easily readable, easily modifiable• utilizes templates of configuration files with markers for changed items• gathers information from the user via command line prompts to automatically populate the OpenSRF and Evergreen configuration files• creates gzipped tar archives of all files required to build each server in the cluster, including custom Debian packages• Archives are re-usable for recreating the same machine on new hardware
  16. 16. GenaSYS in Action
  17. 17. GenaSYS: RecapGenaSYS Installation:• Doesn’t require manual changes of configuration files• Automates repetitive steps• Significantly decreases risk of typos/human error• Reduces number of steps required to create a cluster• Creates packages that can be saved and used again to rebuild each server
  18. 18. Questions?
  19. 19. GenaSYS Future Directions• Project will remain active, primarily for GPLS needs• Feature requests/bug reports will be accepted (method for this TBD)• Plan for Ubuntu LTS support