Scaling Continuous IntegrationPractices to Teams with Parallel         Development          January 19, 2012
Introduction to Today’s SpeakersChris Lucca,Technical Evangelist, AccuRev  Chris experience dates back to his days as a bu...
Introduction to Today’s Speakers Eric Minick, Lead Consultant, UrbanCodeEric Minick is a lead consultant at UrbanCode wher...
Agenda•   Intro to CI•   Overview of Parallel Development•   Adapting CI to the Parallel Dev Challenge•   Q&A             ...
Agenda•   Intro to CI•   Overview of Parallel Development•   Adapting CI to the Parallel Dev Challenge•   Q&A             ...
Traditional Development                                                                                                   ...
Traditionally Devs Work in Isolation             Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
Too Much Isolation Leads to Integration Hell               Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights ...
CI: Minimizing Integration Pain• Integrating frequently is less painful than  infrequently  • But, If we commit frequently...
CI: The Simple Implementation• How do we determine code quality? We test it• How do we test code? We build it and run test...
So Simple but Very Important• A build & unit test is a rudimentary check• Gives important feedback• The heartbeat of the d...
CI at Scale?• With too many committers into the same code  line, the likelihood something is broken right now  is high• We...
Agenda•   Intro to CI•   Overview of Parallel Development•   Adapting CI to the Parallel Dev Challenge•   Q&A             ...
Reasons People Work In ParallelIt’s often a business reason…not for optimization• Maintenance Release – Patches• Customer ...
Technical Debt GrowsAs code lines diverge, integration becomes a biggereventual problem.… But we don’t always want to merg...
Agenda•   Intro to CI•   Overview of Parallel Development•   Adapting CI to the Parallel Dev Challenge•   Q&A             ...
Finding the Isolation / Integration Balance             Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Res...
Integration Is Easy With a Few People Chapter 1-2   Chapter 1-2         Chapter 2-3                              Chapter 1...
It’s hard at scale Chapter 1-2   Chapter 1-2         Chapter 2-3                              Chapter 1-3                 ...
Little Hiccup• Race condition: I could update to the latest code before  the build/test notifies me it doesn’t compile• Co...
Cost of IntegrationPer week:• # of check-ins per developer?• # of developers?• % of check-ins that break things?• # of pro...
Multi-Stage Continuous Integration        ws                     Team A        ws        ws                      CI       ...
•Continuous Integration•Database Deploy                                  •Automated Tests                          •Databa...
•Continuous Integration                                             •Automated Tests•Database Deploy                      ...
A Feature Branch ExampleUrbancode’s Website Today…                                     Next (not ready for prime time)We n...
Building and Testing Multiple Code Lines• Build and test every active code line  (continuously)• Automatically run code sc...
CI for a New Code Line Must Be Cheap• Build and deployment configuration  • Maximize reuse  • Potentially create dynamical...
Deploy Time Integration Is the Same• In the team’s environment: • Continuously deploy and execute functional tests • Regul...
In Closing• Compile is the first test• Unit test, deploy, automation can come after• Get Feedback as Fast as possible• Dif...
What Is AccuRev?  Much more than Version Control…  AccuRevenables Software Process Automation                    Best Prac...
Urbancode ProductsuBuild(AnthillPro) • Continuous integration and deliveryuDeploy • Deployment and release managementuClou...
Q&ACall:   1-800-383-8170 or                                         Call:                   +1 216 858-9000        +1-781...
Upcoming SlideShare
Loading in...5
×

Scaling Continuous Integration Practices to Teams with Parallel Development

2,671

Published on

Slides from an Urbancode and Accurev joint webinar: http://www.accurev.com/webinar/20120119-Scaling-CI-Parallel-Development

Continuous integration is simple with a single development team. But when software projects grow to multiple teams and dependencies, continuous integration loses effectiveness due to parallel projects, varying release schedules, and differing cadences between teams. As a result, many teams unknowingly lose the benefits of continuous integration, and therefore suffer from a lack of feedback and poor quality.

In this webinar, UrbanCode’s Eric Minick and AccuRev’s Chris Lucca will explain how to:

- Scale continuous integration builds across multiple development teams working on parallel projects
- Share only code that has passed continuous integration from other teams to avoid broken builds and confusion
- Automate the configuration of your test environment to handle fluid projects done in parallel

Published in: Technology, Business
1 Comment
4 Likes
Statistics
Notes
  • calculation at slide 21 is wrong:
    200 devs, 2 commits each, 1% are bad = 4 breaks/day --> OK
    If each breakage cuts productivity 50% for 15 minutes, we lose 100 dev hours / day (@$50/hr = $5000/day) --> WRONG
    If each breakage cuts productivity 50% for 15 minutes = 200*50%*15min = 1500dev min = 25 dev hours --> $1250/day
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
2,671
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
75
Comments
1
Likes
4
Embeds 0
No embeds

No notes for slide
  • Isolation is good. Nobody breaks our stuff.
  • CI is based on the observation that the longer development streams diverge, the more painful the eventual integration will be. Taking the converse of the above statement, the more frequently developers integrate their changes, the more painless those integrations will be
  • The compile is arguably the most important test. Non-compiling code, really hurts the rest of your teamRace condition gets worse every way we scale. More people, sloppier commits, slower builds, slower tests, faster commits.
  • Tell story around 400 components branching at once and reconfigured in Hudson.
  • Transcript of "Scaling Continuous Integration Practices to Teams with Parallel Development"

    1. 1. Scaling Continuous IntegrationPractices to Teams with Parallel Development January 19, 2012
    2. 2. Introduction to Today’s SpeakersChris Lucca,Technical Evangelist, AccuRev Chris experience dates back to his days as a build / release specialist and software engineer for IBM-Rational. He has also filled the roles of developer, release engineer and SCM consultant for numerous organizations. In addition Chris is a certified scrum master and a frequent speaker at the AccuRev seminar series. Now Chris brings the experience and knowledge he has acquired to help keep others in the industry up-to-date on the latest developments, and how they can impact them directly. Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    3. 3. Introduction to Today’s Speakers Eric Minick, Lead Consultant, UrbanCodeEric Minick is a lead consultant at UrbanCode where he helps customers get themost out of their build, deploy and release processes. He has 8 years of automationexperience throughout the application life-cycle in roles as a developer, testautomation engineer, and support engineer.Eric has been at the forefront of continuous integration for 5+ years and has workedon all three generations of UrbanCode’s Anthill. Keep up with Eric’s latest insightson UrbanCode’s blog. Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    4. 4. Agenda• Intro to CI• Overview of Parallel Development• Adapting CI to the Parallel Dev Challenge• Q&A Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    5. 5. Agenda• Intro to CI• Overview of Parallel Development• Adapting CI to the Parallel Dev Challenge• Q&A Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    6. 6. Traditional Development Write tests Specify Design Plan Code Integrate Test DocJan Feb Mar Apr May Jun Jul Aug Sep Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    7. 7. Traditionally Devs Work in Isolation Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    8. 8. Too Much Isolation Leads to Integration Hell Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    9. 9. CI: Minimizing Integration Pain• Integrating frequently is less painful than infrequently • But, If we commit frequently blindly, we have a broken code base• Implied condition: each integration will not decrease the code quality • This is the key to CI • The rest is implementation details Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    10. 10. CI: The Simple Implementation• How do we determine code quality? We test it• How do we test code? We build it and run tests against it• This is how we get from CI to CI servers and software builds Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    11. 11. So Simple but Very Important• A build & unit test is a rudimentary check• Gives important feedback• The heartbeat of the dev organization Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    12. 12. CI at Scale?• With too many committers into the same code line, the likelihood something is broken right now is high• We stop caring about failures• The feedback is meaningless, and productivity suffers. CI becomes “constantly broken” Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    13. 13. Agenda• Intro to CI• Overview of Parallel Development• Adapting CI to the Parallel Dev Challenge• Q&A Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    14. 14. Reasons People Work In ParallelIt’s often a business reason…not for optimization• Maintenance Release – Patches• Customer Specials - Just for one customer• Development branches – Developer spikes for experiments that will integrate at an unknown time• Code Configurations – Different configurations of code based on environment, such as UAT, QA• Special Content – Images, binaries Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    15. 15. Technical Debt GrowsAs code lines diverge, integration becomes a biggereventual problem.… But we don’t always want to merge immediately• Maintenance Release – Don’t want new features• Customer Specials – One directional integration• Development branches – Dev enjoys a stable playground• Special Content – Reused at different cadence than code Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    16. 16. Agenda• Intro to CI• Overview of Parallel Development• Adapting CI to the Parallel Dev Challenge• Q&A Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    17. 17. Finding the Isolation / Integration Balance Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    18. 18. Integration Is Easy With a Few People Chapter 1-2 Chapter 1-2 Chapter 2-3 Chapter 1-3 Chapter 1-4 Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    19. 19. It’s hard at scale Chapter 1-2 Chapter 1-2 Chapter 2-3 Chapter 1-3 Chapter 1-4 Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    20. 20. Little Hiccup• Race condition: I could update to the latest code before the build/test notifies me it doesn’t compile• Cost associated with this is huge• Leads to “interesting solutions” • Preflight • Build branches • Build Acceleration • Build Avoidance • Componentization Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    21. 21. Cost of IntegrationPer week:• # of check-ins per developer?• # of developers?• % of check-ins that break things?• # of problems encountered per developer?• Length of time per problem encountered? 200 devs, 2 commits each, 1% are bad = 4 breaks/day If each breakage cuts productivity 50% for 15 minutes, we lose 100 dev hours / day (@$50/hr = $5000/day) Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    22. 22. Multi-Stage Continuous Integration ws Team A ws ws CI int Team B CI ws ws CI ws Minutes Hours Daily Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    23. 23. •Continuous Integration•Database Deploy •Automated Tests •Database Deploy•Web Deploy •Unit Tests •Web Deploy•Binary Deploy •Binary Deploy •Binary Deploy•IT Sign-Off •Database Deploy •Customer Sign-Off •Continuous Integration•Rollback (if needed) •Web Deploy x1 x3 x30 x60 x100s x10 x4 x1 x10 x5 x2 x1 5:00 3:00 2:00 0:10
    24. 24. •Continuous Integration •Automated Tests•Database Deploy •Unit Tests•Web Deploy •Database Deploy •Binary Deploy•Binary Deploy •Web Deploy •Database Deploy•IT Sign-Off •Binary Deploy •Customer Sign-Off •Web Deploy •Team Specific CI•Rollback (if needed)
    25. 25. A Feature Branch ExampleUrbancode’s Website Today… Next (not ready for prime time)We need two build configurations, deploy configs, testconfigs, and test environments Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    26. 26. Building and Testing Multiple Code Lines• Build and test every active code line (continuously)• Automatically run code scans, unit tests and functional tests Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    27. 27. CI for a New Code Line Must Be Cheap• Build and deployment configuration • Maximize reuse • Potentially create dynamically• Tests • Core regression suites shared across code lines • Tests for new functionality shared as code is shared• Test Environments • No excuse for a lack of test environments today • Deploy “what’s in production” as a baseline Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    28. 28. Deploy Time Integration Is the Same• In the team’s environment: • Continuously deploy and execute functional tests • Regularly (nightly / weekly) deploy other components that have passed integration tests• When new features work well, deliver to the integration test environment• Releases to Stage & Prod match the full contents of an earlier test environment Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    29. 29. In Closing• Compile is the first test• Unit test, deploy, automation can come after• Get Feedback as Fast as possible• Different teams need different types of builds• Separate code configurations based on process• It’s about people, but tooling helps Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    30. 30. What Is AccuRev? Much more than Version Control… AccuRevenables Software Process Automation Best Practices For: Change Visual Process Agile Management Compliance Automation Parallel Version SCM Control Global Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    31. 31. Urbancode ProductsuBuild(AnthillPro) • Continuous integration and deliveryuDeploy • Deployment and release managementuCloud(soon) • Environment provisioning from your private cloud Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    32. 32. Q&ACall: 1-800-383-8170 or Call: +1 216 858-9000 +1-781-861-8700 Email: sales@urbancode.comEmail: sales@accurev.com Visit: www.urbancode.comVisit: www.accurev.com Learn: blogs.urbancode.comLearn: blog.accurev.com Copyright © 2012 Urbancode, Inc. and AccuRev, Inc.. All Rights Reserved.
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×