MassivelyContinuous IntegrationFrom 3 days to 30 minutes
•Software   as a Service•Project   Management•Collaborative   Work Management
How we use Selenium •1,100   Selenium 1 tests •800   Selenium 2 tests •40,000    hours per month •Release   Acceptance + CI
ThenScripted scenariosSlow, brittleMonthly Releases3-5 days acceptanceManual administration
NowTight unit-like UI testsFast, RobustDaily releases30-45 minutesSign-off and go
“True” Continuous Integration       •How   often do I integrate?       •What   tests tell me I’m integrated?       •What  ...
Pipeline Priorities  •Performance   (Cycle Time)  •Scalability  •Visibility  •Accountability
ATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY
CommitATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY
Commit                   Build Installer                    Build TestsATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY
Selenium Grid                   Grid + AtTask    Commit                   Build Installer                    Build TestsAT...
Selenium Grid                   Grid + AtTask    Commit                           Install                   Build Installe...
Selenium Grid                   Grid + AtTask    Commit                             Install                   Build Instal...
Selenium Grid                        Selenium IE                   Grid + AtTask    Commit                             Ins...
Selenium Grid                        Selenium IE                   Grid + AtTask    Commit                             Ins...
AIRPORT SECURITYNOBODY LIKES TO WAIT IN LINE
Selenium Grid                        Selenium IE           Grid + AtTask  Commit                       Install          Se...
Selenium Grid                        Selenium IE           Grid + AtTask  Commit                       Install          Se...
CYCLE TIME
CYCLE TIME
CYCLE TIME
JENKINS, EC2, AND ATTASKA MATCH MADE IN HEAVEN
JENKINS, EC2, AND ATTASKA MATCH MADE IN HEAVEN
JENKINS DYNAMIC SLAVE ALLOCATION
JENKINS DYNAMIC SLAVE ALLOCATION
SLAVEJENKINS DYNAMIC SLAVE ALLOCATION
SLAVE                    SLAVE                    SLAVE                    SLAVEJENKINS DYNAMIC SLAVE ALLOCATION
JENKINS DYNAMIC SLAVE ALLOCATION
From 3 days to 30 minuteswith Selenium 2
SELENIUM 1 (1800 TESTS)TESTING WITH SELENIUM 1
SELENIUM 1 (1800 TESTS)                               4 HOURS              SLOWTESTING WITH SELENIUM 1
TRANSITIONING TO SELENIUM 2
TRANSITIONING TO SELENIUM 2
TRANSITIONING TO SELENIUM 2
SELENIUM 1 (1800 TESTS)                               4 HOURSTRANSITIONING TO SELENIUM 2
SELENIUM 1 (1800 TESTS)                                    4 HOURS      SELENIUM 2 (750 TESTS)             30 MINUTES     ...
TRANSITIONING TO SELENIUM 2
2.1 MINUTESTRANSITIONING TO SELENIUM 2
2.1 MINUTES             34 SECONDSTRANSITIONING TO SELENIUM 2
2.1 MINUTES                  34 SECONDS                    WITH GRID...TRANSITIONING TO SELENIUM 2
2.1 MINUTES                  34 SECONDS                    WITH GRID...       8 SECONDSTRANSITIONING TO SELENIUM 2
2.1 MINUTES                  34 SECONDS                    WITH GRID...       8 SECONDS                   2.4 SECONDSTRANS...
Scalability  •To   the cloud with Amazon Cloud Formations   •Dynamic   Selenium Grids   •Dynamic   AtTask Environments
JSONPARTLY CLOUDY WITH A CHANCE OF RAIN
JSONPARTLY CLOUDY WITH A CHANCE OF RAIN
Scalability         Selenium Grid                        Selenium IE         Grid + AtTaskCommit                       Ins...
Scalability               Selenium Grid               Grid + AtTask      Commit               Build Installer             ...
SLAVEIT’S RAINING GRIDS
SLAVEIT’S RAINING GRIDS
SELENIUM GRID   SLAVE                     SELENIUM GRIDIT’S RAINING GRIDS
SELENIUM GRID   SLAVE                     SELENIUM GRIDIT’S RAINING GRIDS
SELENIUM GRID   SLAVE                     SELENIUM GRIDIT’S RAINING GRIDS
SLAVE               SELENIUM GRID   SLAVE                                  SLAVE               SELENIUM GRID   SLAVE      ...
Visibility  •Selenium    results in Jenkins  •Description      plugin   •Theres   my commit!  •View    plugin   •All   the...
Visibility  •Selenium    results in Jenkins  •Description      plugin   •Theres   my commit!  •View    plugin   •All   the...
Visibility  •Selenium    results in Jenkins  •Description      plugin   •Theres   my commit!  •View    plugin   •All   the...
Accountability •Claim   / Blame test results using AtTask
Accountability •Claim   / Blame test results using AtTask
Accountability •Claim   / Blame test results using AtTask
Gotchas •Watch    your integration points •Get   the right skill set •Align   process w/ tools •Continuous    Investment
What does it cost? •3   FTE  •Two      human, one machine(s) •Keeping      costs down  •Efficient    machine sizes  •Rock-s...
Return on investment •5   days (40 hours) to certify one release...  •10   engineers  •$40   per hour  •$20,000 •30   minu...
To release 12 times in a year...  $240,000 TO CERTIFY            $3000 TO TEST                          $160,000 FOR TWO F...
100 deployments annually               $2,000,000
100 deployments annually  $265,000     $2,000,000
100 deployments annually     $265,000
Where to go from here •Continuous    Delivery  •Thoughtworks •Application   Lifecycle Management •Infrastructure   CI •Ope...
Tools Manifest•AtTask          contributions... •EC2       -   wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin •Clou...
Q&A
Upcoming SlideShare
Loading in …5
×

Massively Continuous Integration: From 3 days to 30 minutes

3,551 views

Published on

Published in: Technology, Education
1 Comment
6 Likes
Statistics
Notes
No Downloads
Views
Total views
3,551
On SlideShare
0
From Embeds
0
Number of Embeds
280
Actions
Shares
0
Downloads
118
Comments
1
Likes
6
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Massively Continuous Integration: From 3 days to 30 minutes

    1. 1. MassivelyContinuous IntegrationFrom 3 days to 30 minutes
    2. 2. •Software as a Service•Project Management•Collaborative Work Management
    3. 3. How we use Selenium •1,100 Selenium 1 tests •800 Selenium 2 tests •40,000 hours per month •Release Acceptance + CI
    4. 4. ThenScripted scenariosSlow, brittleMonthly Releases3-5 days acceptanceManual administration
    5. 5. NowTight unit-like UI testsFast, RobustDaily releases30-45 minutesSign-off and go
    6. 6. “True” Continuous Integration •How often do I integrate? •What tests tell me I’m integrated? •What must I know to release?
    7. 7. Pipeline Priorities •Performance (Cycle Time) •Scalability •Visibility •Accountability
    8. 8. ATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY
    9. 9. CommitATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY
    10. 10. Commit Build Installer Build TestsATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY
    11. 11. Selenium Grid Grid + AtTask Commit Build Installer Build TestsATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY
    12. 12. Selenium Grid Grid + AtTask Commit Install Build Installer Build TestsATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY
    13. 13. Selenium Grid Grid + AtTask Commit Install Build Installer Build Tests Unit Integration 1/4 3/4 2/4 4/4ATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY
    14. 14. Selenium Grid Selenium IE Grid + AtTask Commit Install Selenium FF Build Installer Build Tests Other tests Unit Integration 1/4 3/4 2/4 4/4ATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY
    15. 15. Selenium Grid Selenium IE Grid + AtTask Commit Install Selenium FF Tear Down Build Installer Build Tests Other tests Unit Integration 1/4 3/4 2/4 4/4ATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY
    16. 16. AIRPORT SECURITYNOBODY LIKES TO WAIT IN LINE
    17. 17. Selenium Grid Selenium IE Grid + AtTask Commit Install Selenium FF Tear Down Build Installer Build Tests Other tests Unit Integration 1/4 3/4 2/4 4/4SERIAL VS PARALLEL
    18. 18. Selenium Grid Selenium IE Grid + AtTask Commit Install Selenium FF Tear Down Build Installer Build Tests Other tests Unit Integration 1/4 3/4 2/4 4/4SERIAL VS PARALLEL
    19. 19. CYCLE TIME
    20. 20. CYCLE TIME
    21. 21. CYCLE TIME
    22. 22. JENKINS, EC2, AND ATTASKA MATCH MADE IN HEAVEN
    23. 23. JENKINS, EC2, AND ATTASKA MATCH MADE IN HEAVEN
    24. 24. JENKINS DYNAMIC SLAVE ALLOCATION
    25. 25. JENKINS DYNAMIC SLAVE ALLOCATION
    26. 26. SLAVEJENKINS DYNAMIC SLAVE ALLOCATION
    27. 27. SLAVE SLAVE SLAVE SLAVEJENKINS DYNAMIC SLAVE ALLOCATION
    28. 28. JENKINS DYNAMIC SLAVE ALLOCATION
    29. 29. From 3 days to 30 minuteswith Selenium 2
    30. 30. SELENIUM 1 (1800 TESTS)TESTING WITH SELENIUM 1
    31. 31. SELENIUM 1 (1800 TESTS) 4 HOURS SLOWTESTING WITH SELENIUM 1
    32. 32. TRANSITIONING TO SELENIUM 2
    33. 33. TRANSITIONING TO SELENIUM 2
    34. 34. TRANSITIONING TO SELENIUM 2
    35. 35. SELENIUM 1 (1800 TESTS) 4 HOURSTRANSITIONING TO SELENIUM 2
    36. 36. SELENIUM 1 (1800 TESTS) 4 HOURS SELENIUM 2 (750 TESTS) 30 MINUTES WHERE DID THE TESTS GO?TRANSITIONING TO SELENIUM 2
    37. 37. TRANSITIONING TO SELENIUM 2
    38. 38. 2.1 MINUTESTRANSITIONING TO SELENIUM 2
    39. 39. 2.1 MINUTES 34 SECONDSTRANSITIONING TO SELENIUM 2
    40. 40. 2.1 MINUTES 34 SECONDS WITH GRID...TRANSITIONING TO SELENIUM 2
    41. 41. 2.1 MINUTES 34 SECONDS WITH GRID... 8 SECONDSTRANSITIONING TO SELENIUM 2
    42. 42. 2.1 MINUTES 34 SECONDS WITH GRID... 8 SECONDS 2.4 SECONDSTRANSITIONING TO SELENIUM 2
    43. 43. Scalability •To the cloud with Amazon Cloud Formations •Dynamic Selenium Grids •Dynamic AtTask Environments
    44. 44. JSONPARTLY CLOUDY WITH A CHANCE OF RAIN
    45. 45. JSONPARTLY CLOUDY WITH A CHANCE OF RAIN
    46. 46. Scalability Selenium Grid Selenium IE Grid + AtTaskCommit Install Selenium FF Tear Down Build Installer Build Tests Other tests Unit Integration 1/4 3/4 2/4 4/4
    47. 47. Scalability Selenium Grid Grid + AtTask Commit Build Installer Build Tests
    48. 48. SLAVEIT’S RAINING GRIDS
    49. 49. SLAVEIT’S RAINING GRIDS
    50. 50. SELENIUM GRID SLAVE SELENIUM GRIDIT’S RAINING GRIDS
    51. 51. SELENIUM GRID SLAVE SELENIUM GRIDIT’S RAINING GRIDS
    52. 52. SELENIUM GRID SLAVE SELENIUM GRIDIT’S RAINING GRIDS
    53. 53. SLAVE SELENIUM GRID SLAVE SLAVE SELENIUM GRID SLAVE SLAVEIT’S RAINING GRIDS
    54. 54. Visibility •Selenium results in Jenkins •Description plugin •Theres my commit! •View plugin •All the jobs on one line, oh the humanity!
    55. 55. Visibility •Selenium results in Jenkins •Description plugin •Theres my commit! •View plugin •All the jobs on one line, oh the humanity!
    56. 56. Visibility •Selenium results in Jenkins •Description plugin •Theres my commit! •View plugin •All the jobs on one line, oh the humanity!
    57. 57. Accountability •Claim / Blame test results using AtTask
    58. 58. Accountability •Claim / Blame test results using AtTask
    59. 59. Accountability •Claim / Blame test results using AtTask
    60. 60. Gotchas •Watch your integration points •Get the right skill set •Align process w/ tools •Continuous Investment
    61. 61. What does it cost? •3 FTE •Two human, one machine(s) •Keeping costs down •Efficient machine sizes •Rock-solid tear down •Elasticity
    62. 62. Return on investment •5 days (40 hours) to certify one release... •10 engineers •$40 per hour •$20,000 •30 minutes to certify one release... •10 engineers •$40 per hour •$250
    63. 63. To release 12 times in a year... $240,000 TO CERTIFY $3000 TO TEST $160,000 FOR TWO FTE $77,000 FOR CITo release 24 times in a year... $480,000 TO CERTIFY $6000 TO TEST $160,000 FOR TWO FTE $77,000 FOR CITo release 100 times in a year... $2,000,000 TO CERTIFY $25000 TO TEST $160,000 FOR TWO FTE $77,000 FOR CI
    64. 64. 100 deployments annually $2,000,000
    65. 65. 100 deployments annually $265,000 $2,000,000
    66. 66. 100 deployments annually $265,000
    67. 67. Where to go from here •Continuous Delivery •Thoughtworks •Application Lifecycle Management •Infrastructure CI •OpenStack
    68. 68. Tools Manifest•AtTask contributions... •EC2 - wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin •CloudFormation - wiki.jenkins-ci.org/display/JENKINS/AWS+Cloudformation+Plugin •Pipeline View - github.com/JoelJ/PipelineDashboard •Description Setter - github.com/JoelJ/DescriptionSetter •GIT - /wiki.jenkins-ci.org/display/JENKINS/Git+Plugin •AtTask for Jenkins - Coming soon...
    69. 69. Q&A

    ×