Massively Continuous Integration: From 3 days to 30 minutes

MassivelyContinuous IntegrationFrom 3 days to 30 minutes

•Software as a Service•Project Management•Collaborative Work Management

How we use Selenium •1,100 Selenium 1 tests •800 Selenium 2 tests •40,000 hours per month •Release Acceptance + CI

ThenScripted scenariosSlow, brittleMonthly Releases3-5 days acceptanceManual administration

NowTight unit-like UI testsFast, RobustDaily releases30-45 minutesSign-off and go

“True” Continuous Integration •How often do I integrate? •What tests tell me I’m integrated? •What must I know to release?

Pipeline Priorities •Performance (Cycle Time) •Scalability •Visibility •Accountability

ATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY

CommitATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY

Commit Build Installer Build TestsATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY

Selenium Grid Grid + AtTask Commit Build Installer Build TestsATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY

Selenium Grid Grid + AtTask Commit Install Build Installer Build TestsATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY

Selenium Grid Grid + AtTask Commit Install Build Installer Build Tests Unit Integration 1/4 3/4 2/4 4/4ATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY

Selenium Grid Selenium IE Grid + AtTask Commit Install Selenium FF Build Installer Build Tests Other tests Unit Integration 1/4 3/4 2/4 4/4ATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY

Selenium Grid Selenium IE Grid + AtTask Commit Install Selenium FF Tear Down Build Installer Build Tests Other tests Unit Integration 1/4 3/4 2/4 4/4ATTASK’S PIPEBUILD -- DEPLOY -- TEST -- DESTROY

AIRPORT SECURITYNOBODY LIKES TO WAIT IN LINE

Selenium Grid Selenium IE Grid + AtTask Commit Install Selenium FF Tear Down Build Installer Build Tests Other tests Unit Integration 1/4 3/4 2/4 4/4SERIAL VS PARALLEL

CYCLE TIME

JENKINS, EC2, AND ATTASKA MATCH MADE IN HEAVEN

JENKINS DYNAMIC SLAVE ALLOCATION

SLAVEJENKINS DYNAMIC SLAVE ALLOCATION

SLAVE SLAVE SLAVE SLAVEJENKINS DYNAMIC SLAVE ALLOCATION

JENKINS DYNAMIC SLAVE ALLOCATION

From 3 days to 30 minuteswith Selenium 2

SELENIUM 1 (1800 TESTS)TESTING WITH SELENIUM 1

SELENIUM 1 (1800 TESTS) 4 HOURS SLOWTESTING WITH SELENIUM 1

TRANSITIONING TO SELENIUM 2

SELENIUM 1 (1800 TESTS) 4 HOURSTRANSITIONING TO SELENIUM 2

SELENIUM 1 (1800 TESTS) 4 HOURS SELENIUM 2 (750 TESTS) 30 MINUTES WHERE DID THE TESTS GO?TRANSITIONING TO SELENIUM 2

TRANSITIONING TO SELENIUM 2

2.1 MINUTESTRANSITIONING TO SELENIUM 2

2.1 MINUTES 34 SECONDSTRANSITIONING TO SELENIUM 2

2.1 MINUTES 34 SECONDS WITH GRID...TRANSITIONING TO SELENIUM 2

2.1 MINUTES 34 SECONDS WITH GRID... 8 SECONDSTRANSITIONING TO SELENIUM 2

2.1 MINUTES 34 SECONDS WITH GRID... 8 SECONDS 2.4 SECONDSTRANSITIONING TO SELENIUM 2

Scalability •To the cloud with Amazon Cloud Formations •Dynamic Selenium Grids •Dynamic AtTask Environments

JSONPARTLY CLOUDY WITH A CHANCE OF RAIN

Scalability Selenium Grid Selenium IE Grid + AtTaskCommit Install Selenium FF Tear Down Build Installer Build Tests Other tests Unit Integration 1/4 3/4 2/4 4/4

Scalability Selenium Grid Grid + AtTask Commit Build Installer Build Tests

SLAVEIT’S RAINING GRIDS

SELENIUM GRID SLAVE SELENIUM GRIDIT’S RAINING GRIDS

SLAVE SELENIUM GRID SLAVE SLAVE SELENIUM GRID SLAVE SLAVEIT’S RAINING GRIDS

Visibility •Selenium results in Jenkins •Description plugin •Theres my commit! •View plugin •All the jobs on one line, oh the humanity!

Accountability •Claim / Blame test results using AtTask

Gotchas •Watch your integration points •Get the right skill set •Align process w/ tools •Continuous Investment

What does it cost? •3 FTE •Two human, one machine(s) •Keeping costs down •Efﬁcient machine sizes •Rock-solid tear down •Elasticity

Return on investment •5 days (40 hours) to certify one release... •10 engineers •$40 per hour •$20,000 •30 minutes to certify one release... •10 engineers •$40 per hour •$250

To release 12 times in a year... $240,000 TO CERTIFY $3000 TO TEST $160,000 FOR TWO FTE $77,000 FOR CITo release 24 times in a year... $480,000 TO CERTIFY $6000 TO TEST $160,000 FOR TWO FTE $77,000 FOR CITo release 100 times in a year... $2,000,000 TO CERTIFY $25000 TO TEST $160,000 FOR TWO FTE $77,000 FOR CI

100 deployments annually $2,000,000

100 deployments annually $265,000 $2,000,000

100 deployments annually $265,000

Where to go from here •Continuous Delivery •Thoughtworks •Application Lifecycle Management •Infrastructure CI •OpenStack

Tools Manifest•AtTask contributions... •EC2 - wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin •CloudFormation - wiki.jenkins-ci.org/display/JENKINS/AWS+Cloudformation+Plugin •Pipeline View - github.com/JoelJ/PipelineDashboard •Description Setter - github.com/JoelJ/DescriptionSetter •GIT - /wiki.jenkins-ci.org/display/JENKINS/Git+Plugin •AtTask for Jenkins - Coming soon...

http://lanyrd.com	72
http://microchipatwork.com	12
http://198.199.79.89	5

Massively Continuous Integration: From 3 days to 30 minutes

by seleniumconf

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 89

Statistics

Massively Continuous Integration: From 3 days to 30 minutes Presentation Transcript