Your SlideShare is downloading. ×
  • Like
  • Save
JUC NYC 2012: Yale Build and Deployment with Jenkins
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

JUC NYC 2012: Yale Build and Deployment with Jenkins

  • 1,457 views
Published

Yale is a diverse place with a wide variety of technologies and a wide range of developer skillsets. This talk will walk you through the journey that we took to standardize where we could and to …

Yale is a diverse place with a wide variety of technologies and a wide range of developer skillsets. This talk will walk you through the journey that we took to standardize where we could and to bring Yale software build and deploy under control. Our goals are to reduce complexity, increase security, increase agility, accept responsibility for what should be ours and otherwise get out of the developer's way. Jenkins is an integral piece of meeting these goals.

Published in Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,457
On SlideShare
0
From Embeds
0
Number of Embeds
5

Actions

Shares
Downloads
0
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Build and Deploy E Camden Fisher Yale University ITS
  • 2. Jenkins User Conference New York, May 17 2012 #jenkinsconf Who am I? E Camden Fisher Technical Lead Unix Infrastructure and Virtualization Information Technology Services Yale University camden.fisher@yale.edu @fishnix
  • 3. Jenkins User Conference New York, May 17 2012 #jenkinsconfWho Uses Jenkins @Yale?
  • 4. Jenkins User Conference New York, May 17 2012 #jenkinsconf Let’s Not Forget Continuous Integration! !   Maintain a Single Source Repository !   Automate the Build !   Make Your Build Self-Testing !   Everyone Commits to the Mainline Every Day !   Every Commit Should Build the Mainline on an Integration Machine !   Keep the Build Fast !   Test in a Clone of the Production Environment !   Make it Easy for Anyone to Get the Latest Executable !   Everyone can see whats happening !   Automate Deployment Avoid "Integration Hell!!”
  • 5. Jenkins User Conference New York, May 17 2012 #jenkinsconf Why we chose Jenkins? !   Easy! !   Extensible !   Scalable !   Flexible !   Open Source !   Supported!
  • 6. Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale History Lesson Application lifecycle is a progression !   Source Code Management !   Maven and Artifactory !   Building and Testing with Jenkins !   Container Configurable artifacts !   Runtime Configurable Containers !   Managed deployments with Jenkins
  • 7. Jenkins User Conference New York, May 17 2012 #jenkinsconf SCM SCCS cp –pr RCS Subversion CVS VSS
  • 8. Jenkins User Conference New York, May 17 2012 #jenkinsconf MVN Maven !   Project Object Model (POM) !   Simplifies dependency resolution ("oops I forgot that .jar!") !   Makes the build process easy and uniform Artifactory (Maven Repository) !   Where do I put my built artifacts? !   Makes it easy for everyone to get the latest build!
  • 9. Jenkins User Conference New York, May 17 2012 #jenkinsconf Builds: Before Jenkins !   We have a single source repo + a place to store built artifacts, but….. –  Builds still take a long time –  Testing takes longer! –  Code coverage take even longer! –  Build environments are not standardized –  Mistakes are caught later, hard to debug
  • 10. Jenkins User Conference New York, May 17 2012 #jenkinsconf Builds: Enter Jenkins !   SCM commits automatically kick off a build !   Testing and code coverage is automated and is run on every commit. !   Broken builds immediately notify the team and the committer !   Tests run in a Clone of the Production Environment! !   Everyone can easily see whats happening! !   Developers can get back to coding instead of building and testing. !   Releases are quick and easy
  • 11. Jenkins User Conference New York, May 17 2012 #jenkinsconf Deployment: The problem. !   Configuration must be different per environment !   Choice between loss of control or loss of agility –  Often devs configure the container –  Either devs can edit deployables or they cant !   If they can... things are bad –  Code is deployed, edited and removed w/out Change Control –  Moving quickly… "Ill clean it up later!" –  Dev environments quickly diverge from Production –  Security is compromised !   If they cant... things are worse –  Change requests are "slow” –  Systems groups must do everything
  • 12. Jenkins User Conference New York, May 17 2012 #jenkinsconf Deployments: How do we fix them? !   Standardize the process! !   Promote SANE change control !   Normalize deployables/artifacts !   Single location for app configuration data !   Standardize the container !   Externalize what makes an environment unique and special
  • 13. Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Application Installer Plugin !   Standardize the process! !   Why? –  SSH creds + Delegation –  Manage keys outside –  Node name substitution !   How? –  Installer is co’d, scp’d –  install.properties on dest –  Pre-Install –  SSH exec installer –  Post-Install
  • 14. Jenkins User Conference New York, May 17 2012 #jenkinsconf Container Configurable Artifacts !   Artifacts were built with embedded configuration. –  ie. datasources, SSO, service endpoints, etc –  Artifacts are different per environment! WAT!? –  "Oops, I forgot to update that parameter!" !   Externalize configuration parameters! –  deployable XML using JNDIBindingServiceMgr –  Apps self configure with JNDI !   With Container Configurable artifacts… –  The SAME artifact migrates between environments –  XML configuration data (stored as build parameters in Jenkins) is all that differentiates environments
  • 15. Jenkins User Conference New York, May 17 2012 #jenkinsconf Container Configurable Artifacts <server> <mbean code="org.jboss.naming.JNDIBindingServiceMgr" name="org.jasig:service=CasConfig"> <attribute name="BindingsConfig" serialDataType="jbxb"> <jndi:bindings xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" xmlns:jndi="urn:jboss:jndi-binding-service:1.0" xs:schemaLocation="urn:jboss:jndi-binding-service:1.0 resource:jndi-binding- service_1_0.xsd"> <jndi:binding name="jasig/cas/config"> <java:properties xmlns:java="urn:jboss:java-properties" xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" xs:schemaLocation="urn:jboss:java-properties resource:java-properties_1_0.xsd"> <!-- Properties --> <java:property> <java:key>loginUrl</java:key> <java:value>https://secure.its.yale.edu/cas/login</java:value> </java:property> <java:property> <java:key>casServerUrl</java:key> <java:value>https://secure.its.yale.edu/cas</java:value> </java:property> </java:properties> </jndi:binding> </jndi:bindings> </attribute> <depends>jboss:service=Naming</depends> </mbean> </server>
  • 16. Jenkins User Conference New York, May 17 2012 #jenkinsconf Container Configurable ArtifactsString jndiBinding = “JNDI_PATH_HERE”;Context ctx = null;try { logger.info("Loading config from JNDI path: " + jndiBinding);   ctx = new InitialContext();   Map<String, String> config = (Map<String, String>) ctx.lookup(jndiBinding);   doSomethingWithConfig(config);} catch (Exception e) {   throw new RuntimeException("Error getting configuration from JNDI. ", e);} finally {  try { ctx.close(); } catch (Exception e) { }}
  • 17. Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Standard Java Container ! JBoss EAP (5.1 now) !   Minimal customization to externalize configuration into runtime !   JAAS provider for CAS SSO !   Packaged in RPM !   Configuration Management to install + manage what’s “special” !   Meets 100% of use cases (so far) !   Runtime configurable container is key!
  • 18. Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale Standard Java Container $JBOSS_HOME/deploy/jbossweb.sar/server.xml <!-- Connector for SSL 20100624 ECF --> <Connector protocol="HTTP/1.1" SSLEnabled="true" port="${jboss.bind.httpsport}" address="${jboss.bind.address}” secure="true" clientAuth="false" scheme="https” proxyName="${jboss.proxyname}" proxyPort="${jboss.proxyport}” SSLCertificateFile="${jboss.server.home.dir}/conf/server.crt" SSLCertificateKeyFile="${jboss.server.home.dir}/conf/server.pem” URIEncoding="UTF-8” SSLProtocol="TLS”> /etc/init.d/jboss_nodexx # JBoss variables export JBOSS_OPTS=“-Djboss.bind.httpport=${HTTPPORT} -Djboss.bind.httpsport=${HTTPSPORT} ${ADDLJBOPTS}” ${JBOSS_HOME}/bin/run.sh ${JBOSS_OPTS} -c ${JBOSS_NODE} -g $ {JBOSS_PARTITION} -b ${JBOSS_BIND} -u ${CLSTR_ADDR}
  • 19. Jenkins User Conference New York, May 17 2012 #jenkinsconf
  • 20. Jenkins User Conference New York, May 17 2012 #jenkinsconf
  • 21. Jenkins User Conference New York, May 17 2012 #jenkinsconf
  • 22. Jenkins User Conference New York, May 17 2012 #jenkinsconf
  • 23. Jenkins User Conference New York, May 17 2012 #jenkinsconf
  • 24. Jenkins User Conference New York, May 17 2012 #jenkinsconf
  • 25. Jenkins User Conference New York, May 17 2012 #jenkinsconf Deployments: The result. !   Consistency! !   Jenkins writes configuration XML: $human_error-- !   Eliminate shells, and elevated privileges on servers !   Container is managed by infrastructure with the O/S !   Developers can deploy to DEV at will !   Empowers developers to GTD !   Puts gates at appropriate places !   Changes to the Jenkins jobs, containers + deploys to Test/Prod require change control !   Frees Systems folks to work on more interesting things !   Standard Container + Cloudy IaaS + Container Configurable artifacts = Vendor-lock-in-less PaaS!
  • 26. Jenkins User Conference New York, May 17 2012 #jenkinsconf Drupal Continuous Deployment !   Yale has a large shared Drupal infrastructure !   Shared = needs change control !   Migration process looks similar to Java !   Some end users want to edit themes! !   Jenkins to the rescue!
  • 27. Jenkins User Conference New York, May 17 2012 #jenkinsconf What’s New? !   Workflow integration through web services !   Build + Release of Apache Servicemix bundles and “features” !   Spawning and Destroying Servicemix child instances !   Deployment of Features and OSGI bundles to Apache Servicemix !   Deploying Drupal 7 –  git + pull
  • 28. Jenkins User Conference New York, May 17 2012 #jenkinsconf Other Languages ! Php –  Code coverage and unit testing available !   Python –  CherryPy + Filelocker deployments !   Ruby –  Automated unit testing –  Automated code coverage –  Automated deployment coming soon ! .Net –  Build, unit test, archive creation
  • 29. Jenkins User Conference New York, May 17 2012 #jenkinsconf The Future !   RBAC + folders –  delegate responsibility to other systems groups !   Ruby deployments !   Enterprise Service Bus !   Centralized SSH mangagement –  the tools are better now !   Managing/Provisioning with cfg mgmt !   Testing and release of cfg mgmt “code” !   Cloud deployments
  • 30. Jenkins User Conference New York, May 17 2012 #jenkinsconf We’re hiring! http://www.yale.edu/jobs
  • 31. Jenkins User Conference New York, May 17 2012 #jenkinsconf Questions? E Camden Fisher camden.fisher@yale.edu @fishnix
  • 32. Jenkins User Conference New York, May 17 2012 #jenkinsconf Thank You To Our Sponsors Platinum Sponsor Gold Sponsors Silver Sponsors Bronze Sponsors