Pragmatic Continuous Delivery          Neeme Praks           @nemecec      LiveRebel Product Lead          ZeroTurnaround
About me•Java developer 10+ years•Java apps support!•Java dev infra engineer•Analyst, architect•Team Lead•Product Lead
About ZeroTurnarond  Developer productivity    tool:   Java app deployment     tool:
Today•Reasons for the pipeline•Tools used•Overview of the pipeline•Review the details•Run it!
Fedex•Package•Dropoff•Transfer•Delivery•Profit!
Fedex       Java EE•Package    •Package•Dropoff    •Test•Transfer   •Approve•Delivery   •Deploy•Profit!     •Profit!
Questions•How do you package the application?•Where did it come from?•Where does it go?•How does it get deployed?•What exa...
A pipeline?              Source: http://startupblog.files.wordpress.com/2008/09/pipeline1.jpg
Continuous Deliverypipeline      Source: http://studentthinktank.eu/wp-content/uploads/2012/02/03_TURKMEN-PIPELINE.jpg
Philosophy•Automate•Record•Test and monitoring•Recover
The sample pipeline•Currently available tools•Preferably open-source•3 weeks to build•DIY pipeline   google for “pragmatic...
The tools•Orchestration Platform•Artifact Repository•Delivery Manager
Jenkins   +plugins(OSS Continuous Integration Server)
Nexus(OSS/Commercial Artifact Repository)
LiveRebel(Commercial Delivery Manager for Java EE)
The tools in the pipeline
Artifacts in the pipeline• WAR• Trace file   [BUILD]   Build: 221   Jenkins URL: http://localhost:2001/job/build/221/   Hg ...
Pipelinephases
Build phase
Test phase
QA phase
Production phase
Dirty details in Jenkins
Pipeline in  actionhttp://cddemo.zeroturnaround.com/lr-demo/
Questions revisited•How do you package the application?•Where did it come from?•Where does it go?•How does it get deployed...
Questions revisited   Build pipeline has all the           answers!
Things Not Covered•Database•Configuration & Environment•Tests & Monitoring
No way my boss let’s me do this!
No way my boss let’s me do this!•Changing process is hard
No way my boss let’s me do this!•Changing process is hard•SOLUTION: Sneak it in :)
No way my boss let’s me do this!•Changing process is hard•SOLUTION: Sneak it in :)•Create a workflow that captures current ...
No way my boss let’s me do this!•Changing process is hard•SOLUTION: Sneak it in :)•Create a workflow that captures current ...
Conclusions•Jenkins jobs represent the workflow•Nexus is a sync-point for long- running workflows•LiveRebel does updates•Man...
Pragmatic Continuous Delivery             Neeme Praks               @nemecec        LiveRebel Product Lead            Zero...
Q&A
Upcoming SlideShare
Loading in...5
×

Pragmatic Continuous Delivery - ReaktorDevDay 2012

461
-1

Published on

Talk delivered @ http://reaktordevday.fi/2012/

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
461
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Continuous Delivery - a way to automate your development and production environment so you can deliver (small) batches of functionality to production in a safe and predictable way.\n\n\n
  • Foremost: problem solver\nStarted as a developer and have been a Java developer for 10+ years and occasionally also handling the operations side of software delivery - I have seen and felt the pain of manual updates. After all that, I wanted to help out operations and joined ZeroTurnaround to work on LiveRebel product - the deployment tool for Java applications.\n\nThis talk is based on a talk given by ZT CEO, Jevgeni Kabanov\n
  • Shameless plug!\n\nJRebel - no more waiting for redeploys while developing!\nLiveRebel - predictable application updates without users noticing!\n\nApproach me during or after the conference:\nI can give a quick demo\nfree trial licenses available! Free licenses for Scala devs!\n\nBTW, we’re hiring! If you like hacking the JVM - get in touch!\n
  • Why we built the pipeline in this particular way\n
  • Every step is trackable, every failure is recoverable\n
  • On a high level, Java EE seems similar. And easy.\n\nYet everyone complains, why?\n\nUsually, new applications are rushed into production\n\nAfter a while, you start to get questions like...(next slide)\n
  • We did a survey, asking ~700 people, if they knew answers to those questions.\n\nConclusion:\nThere is no standard solution - each environment is different.\nWorse yet, typically these processes are not automated!\n\n
  • Doing all these steps manually: each segment is automated, but human error creeps in between segments\n
  • Automated pipeline - humans are gatekeepers, but no manual processes.\n\nYou push code in from one end and functionality is pushed to users from the other end. \n\nHow do we achieve this “pipe dream”?\n
  • Automate - eliminate human error (human is still in control).\nRecord - to be able to learn from failures, to figure out where something went wrong.\nTests - not just unit, integration or smoke tests. Also monitoring! Little difference between good tests and good monitoring?\nRecover - always have a plan B. If you cannot roll back automatically, you need to know how to roll back manually.\n\n
  • All tools are available now.\nYou can use your own tools, same concept still applies.\n\nIt took us 3 weeks to build, but a lot of that was wrestling with Jenkins quirks.\n\nTo help you in building your own pipeline, we have prepared a step-by-step tutorial, available on our homepage.\n
  • Orchestration - something to run the pipeline steps\nRepository - something to hold the artifacts\nDelivery Manager - something to deploy the artifacts\n\nTesting and monitoring is out of scope.\n\n
  • Jenkins - freely available, open source. Most popular.\nTeamcity/Bamboo are just as good\n\nPlugins: build pipeline plugin, email-ext plugin\nBuild pipeline is the best there is (currently). There is also “flow” plugin, but it lacks documentation.\n\n
  • Artifactory is an alternative\n\nWe have set up different repositories for BUILD, QA, RC and TEST - this is used to keep different pipeline stages as independent as possible.\n
  • You can also use Cargo or just custom scripting.\nLiveRebel can do on-the-fly updates or rolling restarts.\n\n
  • Small arrows signify artifact movement.\n\nEach time we upload an artifact, we upload it to a different repository: build, test, QA, release candidate\n
  • Artifacts flowing through the pipeline\n\nNotice the different stages in trace file.\n
  • Pipeline phases in detail - often they consist of more than one job in Jenkins.\n
  • This is the beginning of a trail of how artifact progresses through the pipeline.\n\ndemo.chat.version is defined in pom.xml as artifact version\nBUILD_NUMBER is built-in Jenkins environment variable.\n\nUpload WAR and trail.\n\nUpload artifacts to BUILD repository - separate repository to catch any errors early (later steps fail to download artifact).\nYou can also use Jenkins to propagate the artifact, but: no outside access and no permanent storage.\n\nTrigger parameterized build - so we can use same build number throughout the pipeline.\n\n
  • two jobs: deploy-test and automatic-tests\n\nthis and all subsequent jobs have no source - they just operate with Nexus\n\ndownload artifacts from BUILD repository\nWAR and build.txt\n\nLiveRebel Jenkins plugin\npreviously downloaded WAR\noverrides application name and adds build number - to keep different versions separate\nuploads also build.txt\ncustom context path\ntest server\n\ntrigger next job: passes on current parameters (original build number)\n\nautomatic-tests:\nexecute very simple test - typically you would do Selenium tests instead.\nrecord the result in trace\n\nget WAR from BUILD repository\nupload it to TEST repository (with trace file, notice that classifier is different! otherwise local repo caches it forever)\n\n
  • build pipeline plugin is buggy - environment is not always propagated!\nwe still need the connection, so that the pipeline plugin shows it as one single pipeline\n\nsend e-mail with URL to trigger next build through API to make it easier for casual users (no need to go to Jenkins UI)\ngotcha: e-mail plugin uses a different syntax for referring to environment variables\n\nthe job triggered via API needs to be parameterized\n\n
  • \n
  • \n
  • \n
  • \n
  • Information formalized in the pipeline.\n\nAnd, for each build, you know where it has been!\n
  • \n
  • Sneak it in by not changing the current process - automate it instead!\n
  • Sneak it in by not changing the current process - automate it instead!\n
  • Sneak it in by not changing the current process - automate it instead!\n
  • Sneak it in by not changing the current process - automate it instead!\n
  • \n
  • \n
  • \n
  • Pragmatic Continuous Delivery - ReaktorDevDay 2012

    1. 1. Pragmatic Continuous Delivery Neeme Praks @nemecec LiveRebel Product Lead ZeroTurnaround
    2. 2. About me•Java developer 10+ years•Java apps support!•Java dev infra engineer•Analyst, architect•Team Lead•Product Lead
    3. 3. About ZeroTurnarond Developer productivity tool: Java app deployment tool:
    4. 4. Today•Reasons for the pipeline•Tools used•Overview of the pipeline•Review the details•Run it!
    5. 5. Fedex•Package•Dropoff•Transfer•Delivery•Profit!
    6. 6. Fedex Java EE•Package •Package•Dropoff •Test•Transfer •Approve•Delivery •Deploy•Profit! •Profit!
    7. 7. Questions•How do you package the application?•Where did it come from?•Where does it go?•How does it get deployed?•What exactly is in prod now?
    8. 8. A pipeline? Source: http://startupblog.files.wordpress.com/2008/09/pipeline1.jpg
    9. 9. Continuous Deliverypipeline Source: http://studentthinktank.eu/wp-content/uploads/2012/02/03_TURKMEN-PIPELINE.jpg
    10. 10. Philosophy•Automate•Record•Test and monitoring•Recover
    11. 11. The sample pipeline•Currently available tools•Preferably open-source•3 weeks to build•DIY pipeline google for “pragmatic continuous delivery”
    12. 12. The tools•Orchestration Platform•Artifact Repository•Delivery Manager
    13. 13. Jenkins +plugins(OSS Continuous Integration Server)
    14. 14. Nexus(OSS/Commercial Artifact Repository)
    15. 15. LiveRebel(Commercial Delivery Manager for Java EE)
    16. 16. The tools in the pipeline
    17. 17. Artifacts in the pipeline• WAR• Trace file [BUILD] Build: 221 Jenkins URL: http://localhost:2001/job/build/221/ Hg revision: f78504a525a617ad319e75bb288c24bdcb325794 Hg log: changeset: 40:f78504a525a6 tag: tip user: Jevgeni Kabanov <jevgeni@zeroturnaround.com> date: Tue Oct 09 13:16:26 2012 +0000 summary: commented out check for HOTPATCH mode - we can now make health checks from localhost [TEST] Jenkins URL: http://localhost:2001/job/automatic-tests/161/ Automated Tests Passed!!! [QA] Manual tests passed!!! [RC] Marked as RC
    18. 18. Pipelinephases
    19. 19. Build phase
    20. 20. Test phase
    21. 21. QA phase
    22. 22. Production phase
    23. 23. Dirty details in Jenkins
    24. 24. Pipeline in actionhttp://cddemo.zeroturnaround.com/lr-demo/
    25. 25. Questions revisited•How do you package the application?•Where did it come from?•Where does it go?•How does it get deployed?•What exactly is in prod now?
    26. 26. Questions revisited Build pipeline has all the answers!
    27. 27. Things Not Covered•Database•Configuration & Environment•Tests & Monitoring
    28. 28. No way my boss let’s me do this!
    29. 29. No way my boss let’s me do this!•Changing process is hard
    30. 30. No way my boss let’s me do this!•Changing process is hard•SOLUTION: Sneak it in :)
    31. 31. No way my boss let’s me do this!•Changing process is hard•SOLUTION: Sneak it in :)•Create a workflow that captures current process
    32. 32. No way my boss let’s me do this!•Changing process is hard•SOLUTION: Sneak it in :)•Create a workflow that captures current process•Then Automate!
    33. 33. Conclusions•Jenkins jobs represent the workflow•Nexus is a sync-point for long- running workflows•LiveRebel does updates•Manual flows with email/REST•Tracking with scripts & text files
    34. 34. Pragmatic Continuous Delivery Neeme Praks @nemecec LiveRebel Product Lead ZeroTurnaround Want more? http://zeroturnaround.comGoogle: “pragmatic continuous delivery”
    35. 35. Q&A

    ×